System

Abstract

A system is provided.SOLUTION: A system comprising: means for a user to input a query content by voice or text; means for a server to generate an answer by a generative AI model; means for a user device to display the generated answer; means for obtaining a feedback from the user and obtaining a congestion status of an appropriate contact when a problem is not solved; and means for displaying the congestion status to the user and performing a connection when the user desires to connect to the contact.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Call centers face many problems when responding to user inquiries. Specifically, users often fail to select the appropriate point of contact when making an inquiry, resulting in frequent incorrect transfers and long wait times. Furthermore, even when the inquiry is about basic information listed on a website, call centers can become overloaded with inquiries. This results in increased costs and decreased customer satisfaction. Therefore, a system is needed to resolve these problems and reduce the burden on both users and call centers. [Means for solving the problem]

[0005] In order to solve the above problems, we provide a system that provides the following means: A means is provided for the user to input the content of their inquiry by voice or text, and the server generates an answer to that inquiry using a generative AI model. A means is provided for the generated answer to be displayed on the user's terminal. A means is provided for the user to check the answer and provide feedback based on the result. If the user is unable to solve the problem, a means is provided for the server to obtain the congestion status of the appropriate counter and to display that information on the user's terminal. If the user requests a connection to a counter, a means is provided for the terminal to send a connection request to the server and connect to the counter. This allows the user to quickly receive an appropriate answer and be connected to the correct counter as needed, improving the efficiency of the call center and reducing the burden on both parties.

[0006] A "user" is a person who makes an inquiry using the inquiry support app.

[0007] A "server" is a device or system that passes the query content to a generative AI model, generates an answer, and sends it to the user's terminal.

[0008] A "generative AI model" is an artificial intelligence model used to generate appropriate responses to inquiries.

[0009] A "user terminal" is a device such as a smartphone or computer operated by a user, which is used to input inquiries and display responses.

[0010] "Inquiry content" refers to the question or problem the user wants to solve, and is information provided in the form of audio, text, images, or video.

[0011] An "answer" is information generated by a generative AI model based on the inquiry content, and is intended to help the user resolve the inquiry.

[0012] "Feedback" is information provided by a user regarding the evaluation of the generated answer and whether or not the answer is resolved.

[0013] The "contact point" is a specific department or person in the call center who directly responds to the user's inquiries.

[0014] "Congestion status" is information about the current busyness and waiting time at a particular counter.

[0015] A "connection request" is a signal sent by a user terminal to a server indicating that the user wishes to connect to a particular window. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that uses an inquiry support app to provide solutions to users before they contact a call center. The system uses a generative AI model to generate answers to users' inquiries, aiming to reduce the burden on both users and call centers.

[0038] User query input

[0039] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0040] Server-generated answers

[0041] The device sends the inquiry received from the user to the server, which then passes the inquiry to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information and provide the optimal answer.

[0042] Displaying answers to users

[0043] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen," the terminal displays this answer to the user. At this point, the user decides whether the provided answer solved their problem.

[0044] If the issue persists, further support is available

[0045] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains further appropriate information about the congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0046] Counter connection

[0047] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[0048] Specific examples

[0049] For example, if a user inquires about a problem such as "slow internet speed," the user enters this information into the app. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow" and provides it to the user. If the answer does not solve the problem, the server obtains the congestion status of internet service centers and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0050] In this way, users can quickly obtain the right information and be connected to the right contact point when necessary, improving the efficiency of the inquiry process and reducing the burden on call centers.

[0051] The processing flow will be explained below.

[0052] Step 1:

[0053] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[0054] Step 2:

[0055] The user inputs the inquiry by voice or text. For example, they input "What is the credit card limit?"

[0056] Step 3:

[0057] The terminal sends the input query to the server, which converts the query into an appropriate format and sends it to the server via the network.

[0058] Step 4:

[0059] The server passes the received query to a generative AI model, which then generates the optimal answer based on past data and related information.

[0060] Step 5:

[0061] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer includes specific information, such as "The standard credit card limit is 500,000 yen."

[0062] Step 6:

[0063] The device displays the received answer to the user, who then reviews the answer and determines whether the problem has been resolved.

[0064] Step 7:

[0065] If the user checks the answer and the problem is not solved, the terminal receives feedback from the user, such as "The problem was not solved."

[0066] Step 8:

[0067] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0068] Step 9:

[0069] The server sends the congestion status to the user terminal, which displays the received congestion status to the user and asks whether the user wishes to connect.

[0070] Step 10:

[0071] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[0072] Step 11:

[0073] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available.

[0074] Step 12:

[0075] After the user receives support, the device records the situation and uses it as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] With the number of inquiries being sent to call centers increasing, many companies are seeking efficient responses. However, current inquiry response systems lack sufficient technology to provide prompt and appropriate responses to inquiries, which can lead to a decline in user satisfaction. Furthermore, if an inquiry is not resolved or if users need to check the busy status of the counter, they must manually complete numerous procedures, making it difficult to receive support at the optimal time. This increases the burden on call centers and poses a challenge in terms of reduced operational efficiency.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes: a means for a user to input an inquiry by voice or text; a means for a terminal to transmit the inquiry from the user to the server; a means for the server to generate an answer to the inquiry using a generative AI model; a means for the server to transmit the generated answer to the terminal; a means for the terminal to display the generated answer; a means for receiving feedback from the user and, if the problem is not resolved, obtaining the congestion status of the counter; a means for transmitting the congestion status to the terminal and having the terminal display the congestion status to the user; and a means for the terminal to transmit a connection request to the server when the user requests a connection to the counter, and the server to attempt to connect to the specified counter. This allows users to quickly obtain automatically generated answers, check the congestion status of the counters in real time, and receive support at the optimal time. It also reduces the burden on call centers and improves business efficiency.

[0081] "User" refers to an individual or corporation that uses the system to enter inquiries and receive responses.

[0082] "Terminal" refers to the electronic device used by a User to launch the Inquiry Support App, input inquiry details, and communicate with the Server. Examples include smartphones and computers.

[0083] "Server" refers to the central system that receives the query, generates a response using a generative AI model, and sends the response to the device.

[0084] "Generative AI model" refers to an artificial intelligence algorithm and its software that generates appropriate answers to user inquiries. Specific examples include models that use natural language processing technology.

[0085] "Feedback" refers to the act of a user evaluating whether a problem has been solved by a provided answer or expressing dissatisfaction.

[0086] "Counter congestion status" refers to information that indicates the counter's availability during a specific time period. For example, it includes the current waiting time and the number of available applications.

[0087] A "connection request" refers to an action in which a user requests a connection to a server via a terminal.

[0088] "Means for generating an answer" refers to the function by which the server uses a generative AI model to create an answer to a user's inquiry.

[0089] This invention is a system that uses an inquiry support app to provide solutions before users contact a call center. The aim is to reduce the burden on both users and call centers. This system is composed of devices such as smartphones and computers, a server, and a generative AI model.

[0090] User query input

[0091] The user launches an inquiry support app installed on a device such as a smartphone or computer. The user can input their inquiry using text or voice. For example, they might input a question such as, "What is my credit card limit?" The device is equipped with text analysis software that analyzes the input and converts it into text data. In the case of voice input, voice recognition software is used to convert the voice data into text data.

[0092] Server-generated answers

[0093] The device sends the query entered by the user to the server in the form of an HTTP request using a REST API. The server then performs preprocessing such as text cleaning and tokenization on the received data before passing it to the generative AI model.

[0094] The generative AI model uses an algorithm that uses natural language processing technology, such as OpenAI's GPT-3 (registered trademark). The server inputs the query content into this model and generates an appropriate answer. For example, the server generates an answer such as "The standard credit card limit is 500,000 yen."

[0095] Displaying answers to users

[0096] The server sends the generated answer to the device, and the device displays the answer to the user. For example, the app displays "The standard credit card limit is 500,000 yen." The user checks this answer and decides whether the problem has been resolved.

[0097] If the issue persists, further support is available

[0098] If the user checks the answer and gives feedback that "the problem was not solved," the device sends this feedback to the server. The server uses congestion analysis software to check the congestion status of specific counters in real time. For example, it generates information such as "The current congestion status of the counter is a waiting time of 10 minutes" and sends it to the device.

[0099] Counter connection

[0100] The terminal displays the received congestion information to the user, and if the user wishes to connect to a counter, the terminal sends a connection request to the server. The server uses call center connection software such as a PBX system to attempt to connect to the specified counter. This allows the user to be connected to the correct counter at the appropriate time.

[0101] Specific examples

[0102] Example 1: Credit card inquiry

[0103] The user enters the text "What is the credit card limit?", and the server uses the generative AI model to generate an answer such as "The standard credit card limit is 500,000 yen." This answer is then displayed on the device.

[0104] Example 2: Querying Internet speed

[0105] The user enters the text "My internet speed is slow," and the server uses a generative AI model to generate "solutions for slow internet speeds." This answer is displayed on the device. If the user is unable to resolve the problem, the device sends feedback to the server, which then obtains the congestion status of the counters and displays it to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, which connects the user to the appropriate counter.

[0106] These features allow users to quickly obtain appropriate information and receive support when needed, while also reducing the burden on call centers and improving operational efficiency.

[0107] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0108] Step 1:

[0109] The user launches the inquiry support app and inputs the inquiry content by voice or text. For example, if the user inputs text, such as "What is the credit card limit?", the voice input is converted into text data using voice recognition software.

[0110] Input: Voice or text inquiry

[0111] Output: Inquiry content in text format

[0112] Step 2:

[0113] The terminal sends the user's inquiry to the server. The terminal sends the inquiry as text data to the server in HTTP request format. For example, if the inquiry is text data such as "What is the credit card limit?", it is sent to the server via the REST API.

[0114] Input: Inquiry details in text format

[0115] Output: Text data sent as an HTTP request

[0116] Step 3:

[0117] The server analyzes the query data received from the device and prepares it for passing to the generative AI model. Specifically, it cleans and tokenizes the text data and converts it into an appropriate format, for example, by removing unnecessary characters and splitting words.

[0118] Input: Text data received as an HTTP request

[0119] Output: Preprocessed data to feed into a generative AI model

[0120] Step 4:

[0121] The server inputs the preprocessed data into a generative AI model to generate an answer. For example, a generative AI model such as GPT-3, which uses natural language processing technology, can be used to generate the optimal answer based on the inquiry. For example, it can generate an answer such as, "The standard credit card limit is 500,000 yen."

[0122] Input: Preprocessed data to feed into a generative AI model

[0123] Output: Answer text from the generative AI model

[0124] Step 5:

[0125] The server sends the generated answer to the terminal. The server sends the generated answer text to the terminal in the form of an HTTP response.

[0126] Input: Answer text from the generative AI model

[0127] Output: The answer text sent as the HTTP response

[0128] Step 6:

[0129] The terminal displays the answer received from the server on the user interface. The terminal displays an answer such as "The standard credit card limit is 500,000 yen" on the application screen.

[0130] Input: The answer text received as an HTTP response

[0131] Output: What is displayed in the user interface

[0132] Step 7:

[0133] If the user checks the answer and the problem is not solved, the user inputs feedback such as "The problem was not solved." The user inputs the feedback content in text format, and the terminal transmits it to the server.

[0134] Input: Feedback text if issue persists

[0135] Output: Feedback data sent as an HTTP request

[0136] Step 8:

[0137] The server receives feedback from users and uses a congestion analysis system to obtain the congestion status of the counters. The server monitors the waiting time and number of receptions at specific counters in real time and generates appropriate congestion status information.

[0138] Input: Feedback data received as an HTTP request

[0139] Output: Congestion status information text

[0140] Step 9:

[0141] The server sends congestion information to the terminal, which then displays it on the user interface. For example, the app screen might say, "The current congestion status at the counter is a 10-minute wait."

[0142] Input: Congestion status information text

[0143] Output: What is displayed in the user interface

[0144] Step 10:

[0145] When a user desires to connect to the service desk, the user sends a request for connection to the service desk from the terminal to the server. For example, the request "I would like to connect to the service desk" is sent in text format.

[0146] Input: Request text for a connection

[0147] Output: Connection request data sent as an HTTP request

[0148] Step 11:

[0149] The server receives the connection request and attempts to connect the user to the appropriate agent using call center connection software, such as a PBX system, so that the user is connected to the appropriate agent at the optimal time.

[0150] Input: Connection request data received as an HTTP request

[0151] Output: Connection established with the window

[0152] (Application example 1)

[0153] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0154] Conventional inquiry support systems have the problem that users must contact a call center every time they have an inquiry, which takes time and effort. In particular, inquiries related to content distribution services require a quick response, which places a heavy burden on both users and the support desk. In addition, there are cases where users are kept waiting for a long time depending on the number of calls at the desk, which reduces user satisfaction.

[0155] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0156] In this invention, the server includes a means for a user to input an inquiry related to the content distribution service by voice or text, a means for generating an answer using a generative AI model, a means for a user terminal to display the generated answer, a means for obtaining feedback from the user and obtaining an appropriate congestion status of the support desk if the problem is not resolved, and a means for displaying the congestion status to the user and connecting to the support desk in real time if the user requests a connection to the support desk. This allows the user to quickly obtain the information they need and smoothly connect to the support desk even if the problem is not resolved.

[0157] "Inquiry Content" refers to the specific question or problem that a user enters for support.

[0158] "Voice or text input means" refers to a means by which a user inputs inquiry details into a terminal using voice recognition or text input.

[0159] A "generative AI model" is an artificial intelligence model that uses natural language processing to automatically generate appropriate answers to user inquiries.

[0160] "Means for generating an answer" refers to a means for creating an answer based on the user's inquiry using a generative AI model.

[0161] A "user terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[0162] The "means for displaying the generated answer" is a means for displaying the answer generated by the server on the user terminal in a form that the user can confirm.

[0163] A "means for obtaining feedback" is a means for users to gather ratings and additional opinions about the answers provided.

[0164] The "means for acquiring the appropriate congestion status of the counter" is a means for the server to acquire the current waiting time and congestion status of the call center or support counter in real time.

[0165] The "means for displaying the congestion status to the user" is a means for displaying the acquired information on the congestion status of the counter on the user terminal.

[0166] The "means for establishing a connection when a customer service connection is desired" refers to the means by which the server establishes a connection to an appropriate customer service point when a user desires to connect to a support center.

[0167] A "content distribution service" is a service that provides digital content such as video and audio to users via the Internet.

[0168] "Content distribution service-related inquiries" refer to questions or problems related to content distribution services, such as content playback, distribution schedules, and account management.

[0169] The "means for connecting to a support desk in real time" is a means for instantly establishing a connection with a support person when a user needs support.

[0170] The present invention is a system that provides solutions to inquiries related to content distribution services before users make them using an inquiry support app. The system uses a generative AI model to generate answers to user inquiries, aiming to reduce the burden on both users and support desks.

[0171] User query input

[0172] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, a user can input "When will the next episode be released?" or ask a similar question by voice.

[0173] Server-generated answers

[0174] The server passes the inquiry received from the user to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information to provide the optimal answer. The generative AI model uses natural language processing technology.

[0175] Displaying answers to users

[0176] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The next episode is scheduled to be released next Friday," the terminal displays this answer to the user. At this point, the user can decide whether the provided answer solved their problem.

[0177] If the issue persists, further support is available

[0178] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains more appropriate information about the congestion status of the counter. The server monitors the waiting time and congestion status of the counter in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0179] Counter connection

[0180] If the user checks the congestion status and still wishes to connect to a hotline, the terminal sends a connection request to the server. The server receives this request and attempts to connect to the specified hotline. This ensures that the user is connected to the correct hotline at the optimal time and receives support to resolve the problem. This process reduces waiting times for users and allows them to receive support efficiently.

[0181] Specific examples

[0182] For example, if a user inquires about a problem such as "I can't play the currently streaming episode," the user enters this information into the app. The server uses a generative AI model to generate a response such as "If you can't play the episode, first check your internet connection. Then try restarting your device," and provides it to the user. If the response does not resolve the problem, the server obtains the congestion status of the streaming service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0183] Prompt Sentence Examples

[0184] "How do I resolve episode playback errors?"

[0185] "I want to know when the next episode will be released."

[0186] In this way, the present invention allows users to quickly obtain the appropriate information and connect them to the correct point of contact as needed, thereby increasing the efficiency of the inquiry process and reducing the burden on support points.

[0187] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0188] Step 1: User enters inquiry

[0189] The user launches an inquiry support app on their smartphone or computer and inputs their inquiry by voice or text. The input data is sent to the server by the user's device. For example, if the user inputs "When will the next episode be released?", the text data is sent to the server.

[0190] Step 2: The server passes the query to the generative AI model

[0191] The server passes the inquiry received from the user to the generative AI model. The generative AI model receives the input text data and generates an appropriate answer based on past inquiry data and related information. For example, in response to the text data "When will the next episode be released?", the generative AI model generates an answer such as "The next episode is scheduled to be released next Friday."

[0192] Step 3: The server generates a response and sends it to the user's device.

[0193] The server sends the answer obtained from the generative AI model to the user's device, which receives the answer and displays it to the user. For example, the answer sent from the server, "The next episode is scheduled to be released next Friday," is displayed on the user's device screen.

[0194] Step 4: User reviews answers and provides feedback

[0195] The user checks the answer displayed on the terminal and determines whether the problem has been solved. If the problem has not been solved, the user inputs feedback such as "The problem has not been solved." This feedback data is again sent from the user terminal to the server.

[0196] Step 5: The server obtains the congestion status of the counter.

[0197] The server receives feedback from the user and initiates operations to obtain the appropriate congestion status of the counter. The server monitors the congestion status of the counter in real time and obtains information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0198] Step 6: The server sends the congestion status to the user's device.

[0199] The server sends the obtained congestion status of the counter to the user's terminal. The user terminal receives this information and displays it to the user. For example, the information sent from the server, "The current congestion status of the counter is a waiting time of 10 minutes," is displayed on the user's terminal screen.

[0200] Step 7: If the user wants a direct connection

[0201] If the user checks the congestion status and still wishes to connect through the service window, the terminal sends a connection request to the server. This connection request data is sent from the user terminal to the server.

[0202] Step 8: The server attempts to connect to the gateway

[0203] The server receives a connection request from the user and attempts to connect to the specified support desk. Once the connection is established, the server sends a notification to the user's device that the connection is complete, allowing the user to contact the support desk. For example, the server may notify the user that "Connection has been established. An operator is waiting."

[0204] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0205] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[0206] User query input

[0207] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0208] Emotion recognition by emotion engine

[0209] The device sends the voice or text inquiry received from the user to the emotion engine. The emotion engine recognizes the emotion from the user's voice or text and sends the result to the server. For example, if the user is angry, the emotion engine recognizes it as "anger."

[0210] Server-generated answers

[0211] The device sends the emotion recognition results along with the query content to the server. The server passes this query content to the generative AI model, which then generates an appropriate response. At this time, the generative AI model also takes the emotion recognition results into account. For example, if the user is angry, it will provide a polite response that takes their emotion into consideration.

[0212] Displaying answers to users

[0213] The server sends the generated answer to the user's terminal, and the terminal displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time," the terminal displays this answer to the user. The user checks the answer and determines whether the problem has been solved.

[0214] If the issue persists, further support is available

[0215] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that the problem was not solved, the server obtains the appropriate congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0216] Counter connection

[0217] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[0218] Specific examples

[0219] For example, if a user inquires about a problem such as "slow internet speed," the user inputs this information into the app. The emotion engine recognizes the emotion from the user's voice as "anger." The server uses a generative AI model to generate a response such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the response does not resolve the problem, the server obtains the congestion status of the internet service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0220] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[0221] The processing flow will be explained below.

[0222] Step 1:

[0223] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[0224] Step 2:

[0225] The user inputs the inquiry by voice or text. For example, they might input something like "What is the credit card limit?" If they input the inquiry by voice, it is recorded as is, and if they input the inquiry by text, it is immediately entered into the text area.

[0226] Step 3:

[0227] The device sends the inquiry received from the user to the emotion engine, which analyzes the received voice or text data and recognizes the user's emotions, such as "anger," "sadness," and "joy."

[0228] Step 4:

[0229] The emotion engine sends information including the recognized user emotion to the server. The original query content is also transferred to the server together with the emotion recognition result.

[0230] Step 5:

[0231] The server passes the received inquiry and emotion recognition results to the generative AI model, which then generates the optimal answer based on past data and related information. For example, if the user expresses anger, the generative AI model will generate a polite answer that takes that emotion into consideration.

[0232] Step 6:

[0233] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer may contain specific information, such as "The standard credit card limit is 500,000 yen. Please feel free to contact us with any questions."

[0234] Step 7:

[0235] The device displays the received answer to the user, who can then review the answer and determine whether it has resolved the problem. Specifically, the answer is displayed on the device's screen, and if it was entered via voice, it is played back aloud.

[0236] Step 8:

[0237] If the user checks the answer and the problem is not solved, the device receives feedback from the user. The feedback "The problem was not solved" is sent. This is done by the user providing feedback by pressing a button, etc.

[0238] Step 9:

[0239] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0240] Step 10:

[0241] The server sends the congestion status to the user's terminal. The terminal displays the received congestion status to the user and asks whether the user wishes to connect. Here, the congestion status is displayed visually in a format that is easy for the user to understand.

[0242] Step 11:

[0243] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[0244] Step 12:

[0245] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available. The status is displayed to the user until the connection is complete.

[0246] Step 13:

[0247] After a user receives support at the customer support center, the terminal records the results and uses them as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[0248] By specifying the specific processing steps as described above, it becomes clear how the system operates and what advantages it provides to both the user and the call center.

[0249] Example 2

[0250] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0251] Conventional inquiry support systems often found it difficult to respond appropriately, taking into account the user's emotions. Furthermore, when the response was insufficient and the problem was not resolved, it was difficult to quickly connect the user to the appropriate help desk, which led to a decrease in user satisfaction. Therefore, there was a need for a system that could automate appropriate responses that took the user's emotions into account and quickly connect the user to the appropriate help desk if the problem was not resolved.

[0252] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0253] In this invention, the server includes a means for transmitting the content of an inquiry entered by a user terminal to an emotion recognition engine and analyzing the user's emotions, a means for passing the recognition results from the emotion recognition engine and the content of the inquiry to a generative AI model and generating an answer that takes emotion recognition into consideration, and a means for transmitting the generated answer to the user terminal and for the user terminal to display the generated answer. This enables an appropriate answer that takes the user's emotions into consideration, thereby improving user satisfaction. In addition, by including a means for obtaining feedback from the user, obtaining an appropriate counter congestion status if the problem is not resolved, displaying the congestion status to the user, and connecting to a counter if the user requests it, a prompt and appropriate response can be made.

[0254] "User" means an individual or organization that makes an inquiry using the inquiry support app.

[0255] A "terminal" is an electronic device used by a user, such as a smartphone, computer, or tablet.

[0256] "Inquiry content" refers to questions or requests that a user inputs through voice or text.

[0257] "Speech or text" refers to textual or audio information that a user inputs into a terminal.

[0258] An "emotion recognition engine" is software or hardware that analyzes and recognizes a user's emotions from input voice or text.

[0259] The "server" is a computer system that processes the inquiry content and emotion recognition results, and generates and manages answers using a generative AI model.

[0260] A "generative AI model" is an artificial intelligence algorithm that generates appropriate answers based on input data.

[0261] A "prompt sentence" is an input sentence that prompts a generative AI model to generate an appropriate answer.

[0262] "Feedback" refers to the evaluation or opinion that a user submits after reviewing an answer.

[0263] The "contact point" refers to a department or service provider that handles inquiries from users.

[0264] "Congestion status" is information indicating the current waiting time at the counter and the status of resources available for processing.

[0265] A "connection" is the establishment of communication that allows a user to make a direct inquiry to a customer service center.

[0266] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion recognition engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[0267] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0268] The device sends the voice or text query received from the user to the emotion recognition engine. The emotion recognition engine uses speech recognition software (e.g., the SpeechRecognition library) and text analysis algorithms (e.g., the Python NLTK library) to recognize the user's emotion. For example, if the user types "What's my credit card limit?", the engine recognizes the user's emotion as "neutral."

[0269] After the emotion recognition engine obtains the recognition result, the device sends the query content and the emotion recognition result to the server. The server passes the query content and emotion recognition result to a generative AI model (e.g., OpenAI's GPT-3) and generates an appropriate response. At this time, the generative AI model takes the emotion recognition result into consideration. For example, if the user is angry, it will generate a kind and polite response that takes that emotion into account.

[0270] Example prompt sentence:

[0271] "The user has made the following query: 'What is the credit card limit?' The current sentiment is neutral. Please generate an appropriate answer."

[0272] The server sends the generated answer to the user's terminal, and the terminal displays this answer to the user. For example, if a response such as "The standard credit card limit is 500,000 yen. Please contact us anytime if you have any questions" is generated, this response will be displayed on the terminal. The user can check the response and determine whether the problem has been resolved.

[0273] If the user gives feedback that "the problem was not resolved," the device sends this feedback to the server. The server then obtains the appropriate counter congestion status using a real-time data collection API (for example, Firebase Realtime Database) and generates information such as "The current counter congestion status is a 10-minute wait time." This information is again sent to the user's device and displayed to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, and the server attempts to connect to the specified counter. This allows the user to be connected to the correct counter at the optimal time and receive support to resolve the problem.

[0274] As a concrete example, let's consider the case of an inquiry about the problem of "slow internet speed." The user enters this information into the app, and the emotion recognition engine recognizes "anger" from the voice. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the answer does not solve the problem, the server obtains the congestion status of the internet service center and displays this to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0275] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[0276] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0277] Step 1:

[0278] The user inputs the inquiry content by voice or text. The user starts the inquiry support app and inputs the inquiry content into a device such as a smartphone or computer using a text box or microphone. For example, the user inputs text or voice such as, "What is the credit card limit?"

[0279] Input: Enquiry (e.g. "What is the credit card limit?")

[0280] Output: Inquiry entered into the terminal

[0281] Step 2:

[0282] The device sends the query to the emotion recognition engine, which analyzes the user's emotions. The device sends the input voice or text to the emotion recognition engine, which uses voice recognition software and text analysis algorithms to analyze and recognize the user's emotions.

[0283] Input: User inquiry

[0284] Output: The emotion recognized by the emotion recognition engine (e.g. "Neutral")

[0285] Specific behavior: The terminal sends the text "What is the credit card limit?" to the emotion recognition engine, which recognizes it as "neutral."

[0286] Step 3:

[0287] The device sends the query and emotion recognition results to the server. The device generates an HTTP POST request to send the query and emotion recognition results to the server.

[0288] Input: User's inquiry and emotion recognition result (e.g., "neutral")

[0289] Output: Query content sent to the server and emotion recognition results

[0290] Specific operation: The device sends the query "What is the credit card limit?" and the emotion recognition result "Neutral" to the server as an HTTP POST request.

[0291] Step 4:

[0292] The server generates an answer using a generative AI model. The server inputs the query content and emotion recognition results into the generative AI model to obtain an appropriate answer. For example, OpenAI's GPT-3 is used as the generative AI model.

[0293] Input: Enquiry and emotion recognition result (e.g. "Neutral")

[0294] Output: The answer generated by the generative AI model

[0295] Specific operation: The server inputs the following prompt sentence into the generative AI model: "The user has made the following inquiry: 'What is the credit card limit?' The current emotion is neutral. Please generate an appropriate answer." The server receives the answer from the generative AI model: "The standard credit card limit is 500,000 yen."

[0296] Step 5:

[0297] The server transmits the generated response to the user terminal, and the server generates and transmits data for transmitting the generated response to the user terminal.

[0298] Input: The answer generated by the generative AI model

[0299] Output: The answer sent to the user's terminal

[0300] Specific operation: The server sends the HTTP response "The standard credit card limit is 500,000 yen" to the user's terminal.

[0301] Step 6:

[0302] The device displays the answer to the user. The device displays the answer it receives in the app so that the user can check it.

[0303] Input: Received Answer

[0304] Output: The answer displayed to the user

[0305] Specific operation: The device app displays to the user, "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time."

[0306] Step 7:

[0307] The user sends feedback to the terminal, and the user gives feedback on whether the answer was helpful in solving the problem.

[0308] Input: User feedback

[0309] Output: Feedback sent to the terminal

[0310] Specific operation: The user types "The problem was not resolved" into the terminal and submits it.

[0311] Step 8:

[0312] The device sends the feedback to the server. The device generates an HTTP request to send the user's feedback to the server.

[0313] Input: User feedback

[0314] Output: Feedback sent to the server

[0315] Specific operation: The device sends the user's feedback "The problem was not resolved" to the server.

[0316] Step 9:

[0317] The server obtains the congestion status and sends it to the terminal. The server obtains the congestion status of the appropriate counter through the real-time data collection API and sends it to the terminal.

[0318] Input: Requests to the server and congestion status from the real-time data collection API

[0319] Output: Congestion status information sent to the user terminal

[0320] Specific operation: The server obtains the congestion status using the Firebase Realtime Database API and sends the message "The current congestion status at the counter is a waiting time of 10 minutes" to the user's terminal.

[0321] Step 10:

[0322] The user sends a request for connection to the terminal. The user checks the congestion status and sends a request for connection through the service window to the terminal.

[0323] Input: User connection request

[0324] Output: Connection request sent to the device

[0325] Specific operation: The user enters "I would like to connect through the counter" into the terminal and sends it.

[0326] Step 11:

[0327] The terminal sends a connection request to the server. The terminal generates an HTTP request to send the connection request to the server.

[0328] Input: A user's connection request

[0329] Output: Connection request sent to server

[0330] Specific operation: The terminal sends the user's connection request "I want to connect to the window" to the server.

[0331] Step 12:

[0332] The server connects to the specified gateway. The server receives the connection request and performs the operations to connect the user to the specified gateway.

[0333] Input: Connection request

[0334] Output: Connection established between user and contact

[0335] Specific operation: The server sends a connection request to the call center system and connects the user to the specified contact point.

[0336] The above is the specific flow of the program processing of the invented system and details of each processing step.

[0337] (Application example 2)

[0338] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0339] The present invention aims to improve user satisfaction by combining a system that provides solutions before the user makes an inquiry with an emotion engine that recognizes the user's emotions, enabling responses tailored to the user's emotions. However, current inquiry systems have the problem of difficulty in properly understanding the user's emotions and customizing responses. Furthermore, if a user's inquiry cannot be properly addressed, it is necessary to quickly connect them to an appropriate point of contact. An efficient and effective system is needed to solve this problem.

[0340] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0341] In this invention, the server includes means for the user to input the inquiry content by voice or text, means for the server to generate an answer using a generative AI model, means for the user terminal to display the generated answer, means for obtaining feedback from the user and obtaining an appropriate counter congestion status if the problem is not resolved, means for displaying the congestion status to the user and connecting to a counter if the user requests it, means for recognizing the user's emotion using an emotion engine and passing the recognition result to the generative AI model, and means for the server to generate an answer taking the emotion recognition result into consideration. This makes it possible to provide a system that enables more appropriate and customized responses according to the user's emotion and improves user satisfaction.

[0342] A "user" is an individual or legal entity that uses the system to make an inquiry.

[0343] The "content of inquiry" refers to the specific content of the question or problem that the user wants to solve.

[0344] "Voice or text" refers to the form of means by which a user inputs inquiry content, with voice meaning voice data and text meaning character information.

[0345] A "server" is a computer system that receives, processes, and stores data from user terminals over a network.

[0346] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to generate appropriate answers to user inquiries.

[0347] A "user terminal" is a device that allows a user to access the system, input inquiries, and receive responses.

[0348] "Feedback" refers to the evaluations and comments that users make on the system's suggestions and answers.

[0349] The "contact point" refers to a department or person in charge who responds to user inquiries in person or by telephone or other means.

[0350] "Congestion status" refers to indicators such as the number of inquiries currently being handled at the counter and waiting times.

[0351] An "emotion engine" is an algorithm that recognizes a user's emotions from voice or text.

[0352] "Emotion recognition result" is data indicating the emotional state of the user analyzed by the emotion engine.

[0353] The present invention begins when a user uses an inquiry support app installed on a device such as a smartphone or computer to input their inquiry by voice or text. For example, the user may make an inquiry such as, "What is my credit card limit?" This input is sent to a server, and the emotion engine recognizes the user's emotion.

[0354] The server receives the user's inquiry and the emotion recognition results, inputs them into a generative AI model, and generates an appropriate answer. This generative AI model uses the prompt sentence to perform advanced natural language processing and provide a response that takes emotion into consideration. For example, if the user is angry, the model responds according to the user's emotion by adding expressions of gratitude or apology, such as "We apologize for the inconvenience."

[0355] The generated answer is sent to the user's terminal, where the user can review it. If the user is not satisfied with the answer, the terminal obtains feedback. If the feedback is "the problem was not resolved," the server further obtains the appropriate counter congestion status and provides the user with information such as "The current counter congestion status is a 10-minute wait time."

[0356] If a user checks the congestion status and wishes to connect to a hotline, the user's terminal sends a connection request to the server, and the server connects the user to the appropriate hotline. In this way, the user is connected to the appropriate hotline at the optimal time and can receive support to resolve their problem.

[0357] Specifically, by inputting the following prompt sentence into the generative AI model, an appropriate answer is generated:

[0358] "Please politely respond to angry users with the following question: My internet is slow. What should I do?"

[0359] This system uses the speech_recognition library for speech recognition and the transformers library for emotion recognition. It also uses the latest natural language processing technologies, such as GPT-3, for its generative AI model. It uses communication services such as the Twilio API for notification services. This enables highly efficient and appropriate customer support tailored to the user's emotional state. This system can significantly improve customer service in physical stores and increase user satisfaction.

[0360] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0361] Step 1:

[0362] The user inputs the inquiry by voice or text.

[0363] Input: User's voice or text query

[0364] How it works: A user uses an inquiry support app installed on a device such as a smartphone or computer and enters a specific question, such as "What is my credit card limit?"

[0365] Step 2:

[0366] The terminal sends the inquiry to the server.

[0367] Input: The audio or text data generated in step 1

[0368] Output: Query data arrives at the server

[0369] Operation: The terminal sends the user's inquiry to the server as voice data or text data.

[0370] Step 3:

[0371] The server converts the received voice data into text (in the case of voice input).

[0372] Input: Audio data

[0373] Output: Text data

[0374] How it works: The server uses the speech_recognition library to analyze the audio data and convert it to text data.

[0375] Step 4:

[0376] The server sends the received text data to the emotion engine to recognize the user's emotion.

[0377] Input: Text data of inquiry content

[0378] Output: Emotion recognition result data

[0379] How it works: The server uses the transformers library to parse the text data to determine the user's emotions, and then obtains the results. For example, it identifies emotions such as "anger," "joy," and "sadness."

[0380] Step 5:

[0381] The server inputs the query content and emotion recognition results into a generative AI model to generate an answer.

[0382] Input: Text data of inquiry content, emotion recognition result data

[0383] Output: Generated answer text

[0384] How it works: The server inputs a prompt into the generative AI model, which generates an answer based on the emotion recognition results. An example of a prompt is, "Please politely respond to the following question for an angry user: My internet speed is slow. What should I do?"

[0385] Step 6:

[0386] The server transmits the generated answer to the user terminal.

[0387] Input: Generated answer text

[0388] Output: The answer displayed on the user's terminal

[0389] Operation: The server sends the generated answer data to the user's terminal, which displays it.

[0390] Step 7:

[0391] The user reviews the answer and provides feedback on whether the problem was resolved.

[0392] Input: User feedback

[0393] Output: Feedback data

[0394] Action: The user reviews the answer they received and provides feedback via their device, such as "problem solved" or "not solved."

[0395] Step 8:

[0396] The server receives the feedback data and obtains the appropriate busy status of the counter if the problem is not resolved.

[0397] Input: Feedback data

[0398] Output: Counter congestion data

[0399] Operation: If the user's feedback is "not resolved", the server obtains the appropriate counter congestion status in real time.

[0400] Step 9:

[0401] The server transmits the congestion status of the counter to the user terminal, and when the user requests a counter connection, the connection is made.

[0402] Input: Counter congestion data, user connection requests

[0403] Output: Check connection to the counter

[0404] Operation: The server sends congestion status data to the user terminal, and if the user requests a gateway connection, processes the connection request and connects to the appropriate gateway.

[0405] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0406] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0407] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0408] [Second embodiment]

[0409] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0410] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0411] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0412] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0413] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0414] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0415] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0416] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0417] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0418] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0419] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0420] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0421] This invention is a system that uses an inquiry support app to provide solutions to users before they contact a call center. The system uses a generative AI model to generate answers to users' inquiries, aiming to reduce the burden on both users and call centers.

[0422] User query input

[0423] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0424] Server-generated answers

[0425] The device sends the inquiry received from the user to the server, which then passes the inquiry to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information and provide the optimal answer.

[0426] Displaying answers to users

[0427] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen," the terminal displays this answer to the user. At this point, the user decides whether the provided answer solved their problem.

[0428] If the issue persists, further support is available

[0429] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains further appropriate information about the congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0430] Counter connection

[0431] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[0432] Specific examples

[0433] For example, if a user inquires about a problem such as "slow internet speed," the user enters this information into the app. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow" and provides it to the user. If the answer does not solve the problem, the server obtains the congestion status of internet service centers and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0434] In this way, users can quickly obtain the right information and be connected to the right contact point when necessary, improving the efficiency of the inquiry process and reducing the burden on call centers.

[0435] The processing flow will be explained below.

[0436] Step 1:

[0437] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[0438] Step 2:

[0439] The user inputs the inquiry by voice or text. For example, they input "What is the credit card limit?"

[0440] Step 3:

[0441] The terminal sends the input query to the server, which converts the query into an appropriate format and sends it to the server via the network.

[0442] Step 4:

[0443] The server passes the received query to a generative AI model, which then generates the optimal answer based on past data and related information.

[0444] Step 5:

[0445] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer includes specific information, such as "The standard credit card limit is 500,000 yen."

[0446] Step 6:

[0447] The device displays the received answer to the user, who then reviews the answer and determines whether the problem has been resolved.

[0448] Step 7:

[0449] If the user checks the answer and the problem is not solved, the terminal receives feedback from the user, such as "The problem was not solved."

[0450] Step 8:

[0451] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0452] Step 9:

[0453] The server sends the congestion status to the user terminal, which displays the received congestion status to the user and asks whether the user wishes to connect.

[0454] Step 10:

[0455] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[0456] Step 11:

[0457] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available.

[0458] Step 12:

[0459] After the user receives support, the device records the situation and uses it as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[0460] Example 1

[0461] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0462] With the number of inquiries being sent to call centers increasing, many companies are seeking efficient responses. However, current inquiry response systems lack sufficient technology to provide prompt and appropriate responses to inquiries, which can lead to a decline in user satisfaction. Furthermore, if an inquiry is not resolved or if users need to check the busy status of the counter, they must manually complete numerous procedures, making it difficult to receive support at the optimal time. This increases the burden on call centers and poses a challenge in terms of reduced operational efficiency.

[0463] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0464] In this invention, the server includes: a means for a user to input an inquiry by voice or text; a means for a terminal to transmit the inquiry from the user to the server; a means for the server to generate an answer to the inquiry using a generative AI model; a means for the server to transmit the generated answer to the terminal; a means for the terminal to display the generated answer; a means for receiving feedback from the user and, if the problem is not resolved, obtaining the congestion status of the counter; a means for transmitting the congestion status to the terminal and having the terminal display the congestion status to the user; and a means for the terminal to transmit a connection request to the server when the user requests a connection to the counter, and the server to attempt to connect to the specified counter. This allows users to quickly obtain automatically generated answers, check the congestion status of the counters in real time, and receive support at the optimal time. It also reduces the burden on call centers and improves business efficiency.

[0465] "User" refers to an individual or corporation that uses the system to enter inquiries and receive responses.

[0466] "Terminal" refers to the electronic device used by a User to launch the Inquiry Support App, input inquiry details, and communicate with the Server. Examples include smartphones and computers.

[0467] "Server" refers to the central system that receives the query, generates a response using a generative AI model, and sends the response to the device.

[0468] "Generative AI model" refers to an artificial intelligence algorithm and its software that generates appropriate answers to user inquiries. Specific examples include models that use natural language processing technology.

[0469] "Feedback" refers to the act of a user evaluating whether a problem has been solved by a provided answer or expressing dissatisfaction.

[0470] "Counter congestion status" refers to information that indicates the counter's availability during a specific time period. For example, it includes the current waiting time and the number of available applications.

[0471] A "connection request" refers to an action in which a user requests a connection to a server via a terminal.

[0472] "Means for generating an answer" refers to the function by which the server uses a generative AI model to create an answer to a user's inquiry.

[0473] This invention is a system that uses an inquiry support app to provide solutions before users contact a call center. The aim is to reduce the burden on both users and call centers. This system is composed of devices such as smartphones and computers, a server, and a generative AI model.

[0474] User query input

[0475] The user launches an inquiry support app installed on a device such as a smartphone or computer. The user can input their inquiry using text or voice. For example, they might input a question such as, "What is my credit card limit?" The device is equipped with text analysis software that analyzes the input and converts it into text data. In the case of voice input, voice recognition software is used to convert the voice data into text data.

[0476] Server-generated answers

[0477] The device sends the query entered by the user to the server in the form of an HTTP request using a REST API. The server then performs preprocessing such as text cleaning and tokenization on the received data before passing it to the generative AI model.

[0478] The generative AI model uses an algorithm that uses natural language processing technology, such as OpenAI's GPT-3. The server inputs the query into this model and generates an appropriate answer. For example, the server might generate an answer such as, "The standard credit card limit is 500,000 yen."

[0479] Displaying answers to users

[0480] The server sends the generated answer to the device, and the device displays the answer to the user. For example, the app displays "The standard credit card limit is 500,000 yen." The user checks this answer and decides whether the problem has been resolved.

[0481] If the issue persists, further support is available

[0482] If the user checks the answer and gives feedback that "the problem was not solved," the device sends this feedback to the server. The server uses congestion analysis software to check the congestion status of specific counters in real time. For example, it generates information such as "The current congestion status of the counter is a waiting time of 10 minutes" and sends it to the device.

[0483] Counter connection

[0484] The terminal displays the received congestion information to the user, and if the user wishes to connect to a counter, the terminal sends a connection request to the server. The server uses call center connection software such as a PBX system to attempt to connect to the specified counter. This allows the user to be connected to the correct counter at the appropriate time.

[0485] Specific examples

[0486] Example 1: Credit card inquiry

[0487] The user enters the text "What is the credit card limit?", and the server uses the generative AI model to generate an answer such as "The standard credit card limit is 500,000 yen." This answer is then displayed on the device.

[0488] Example 2: Querying Internet speed

[0489] The user enters the text "My internet speed is slow," and the server uses a generative AI model to generate "solutions for slow internet speeds." This answer is displayed on the device. If the user is unable to resolve the problem, the device sends feedback to the server, which then obtains the congestion status of the counters and displays it to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, which connects the user to the appropriate counter.

[0490] These features allow users to quickly obtain appropriate information and receive support when needed, while also reducing the burden on call centers and improving operational efficiency.

[0491] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0492] Step 1:

[0493] The user launches the inquiry support app and inputs the inquiry content by voice or text. For example, if the user inputs text, such as "What is the credit card limit?", the voice input is converted into text data using voice recognition software.

[0494] Input: Voice or text inquiry

[0495] Output: Inquiry content in text format

[0496] Step 2:

[0497] The terminal sends the user's inquiry to the server. The terminal sends the inquiry as text data to the server in HTTP request format. For example, if the inquiry is text data such as "What is the credit card limit?", it is sent to the server via the REST API.

[0498] Input: Inquiry details in text format

[0499] Output: Text data sent as an HTTP request

[0500] Step 3:

[0501] The server analyzes the query data received from the device and prepares it for passing to the generative AI model. Specifically, it cleans and tokenizes the text data and converts it into an appropriate format, for example, by removing unnecessary characters and splitting words.

[0502] Input: Text data received as an HTTP request

[0503] Output: Preprocessed data to feed into a generative AI model

[0504] Step 4:

[0505] The server inputs the preprocessed data into a generative AI model to generate an answer. For example, a generative AI model such as GPT-3, which uses natural language processing technology, can be used to generate the optimal answer based on the inquiry. For example, it can generate an answer such as, "The standard credit card limit is 500,000 yen."

[0506] Input: Preprocessed data to feed into a generative AI model

[0507] Output: Answer text from the generative AI model

[0508] Step 5:

[0509] The server sends the generated answer to the terminal. The server sends the generated answer text to the terminal in the form of an HTTP response.

[0510] Input: Answer text from the generative AI model

[0511] Output: The answer text sent as the HTTP response

[0512] Step 6:

[0513] The terminal displays the answer received from the server on the user interface. The terminal displays an answer such as "The standard credit card limit is 500,000 yen" on the application screen.

[0514] Input: The answer text received as an HTTP response

[0515] Output: What is displayed in the user interface

[0516] Step 7:

[0517] If the user checks the answer and the problem is not solved, the user inputs feedback such as "The problem was not solved." The user inputs the feedback content in text format, and the terminal transmits it to the server.

[0518] Input: Feedback text if issue persists

[0519] Output: Feedback data sent as an HTTP request

[0520] Step 8:

[0521] The server receives feedback from users and uses a congestion analysis system to obtain the congestion status of the counters. The server monitors the waiting time and number of receptions at specific counters in real time and generates appropriate congestion status information.

[0522] Input: Feedback data received as an HTTP request

[0523] Output: Congestion status information text

[0524] Step 9:

[0525] The server sends congestion information to the terminal, which then displays it on the user interface. For example, the app screen might say, "The current congestion status at the counter is a 10-minute wait."

[0526] Input: Congestion status information text

[0527] Output: What is displayed in the user interface

[0528] Step 10:

[0529] When a user desires to connect to the service desk, the user sends a request for connection to the service desk from the terminal to the server. For example, the request "I would like to connect to the service desk" is sent in text format.

[0530] Input: Request text for a connection

[0531] Output: Connection request data sent as an HTTP request

[0532] Step 11:

[0533] The server receives the connection request and attempts to connect the user to the appropriate agent using call center connection software, such as a PBX system, so that the user is connected to the appropriate agent at the optimal time.

[0534] Input: Connection request data received as an HTTP request

[0535] Output: Connection established with the window

[0536] (Application example 1)

[0537] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0538] Conventional inquiry support systems have the problem that users must contact a call center every time they have an inquiry, which takes time and effort. In particular, inquiries related to content distribution services require a quick response, which places a heavy burden on both users and the support desk. In addition, there are cases where users are kept waiting for a long time depending on the number of calls at the desk, which reduces user satisfaction.

[0539] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0540] In this invention, the server includes a means for a user to input an inquiry related to the content distribution service by voice or text, a means for generating an answer using a generative AI model, a means for a user terminal to display the generated answer, a means for obtaining feedback from the user and obtaining an appropriate congestion status of the support desk if the problem is not resolved, and a means for displaying the congestion status to the user and connecting to the support desk in real time if the user requests a connection to the support desk. This allows the user to quickly obtain the information they need and smoothly connect to the support desk even if the problem is not resolved.

[0541] "Inquiry Content" refers to the specific question or problem that a user enters for support.

[0542] "Voice or text input means" refers to a means by which a user inputs inquiry details into a terminal using voice recognition or text input.

[0543] A "generative AI model" is an artificial intelligence model that uses natural language processing to automatically generate appropriate answers to user inquiries.

[0544] "Means for generating an answer" refers to a means for creating an answer based on the user's inquiry using a generative AI model.

[0545] A "user terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[0546] The "means for displaying the generated answer" is a means for displaying the answer generated by the server on the user terminal in a form that the user can confirm.

[0547] A "means for obtaining feedback" is a means for users to gather ratings and additional opinions about the answers provided.

[0548] The "means for acquiring the appropriate congestion status of the counter" is a means for the server to acquire the current waiting time and congestion status of the call center or support counter in real time.

[0549] The "means for displaying the congestion status to the user" is a means for displaying the acquired information on the congestion status of the counter on the user terminal.

[0550] The "means for establishing a connection when a customer service connection is desired" refers to the means by which the server establishes a connection to an appropriate customer service point when a user desires to connect to a support center.

[0551] A "content distribution service" is a service that provides digital content such as video and audio to users via the Internet.

[0552] "Content distribution service-related inquiries" refer to questions or problems related to content distribution services, such as content playback, distribution schedules, and account management.

[0553] The "means for connecting to a support desk in real time" is a means for instantly establishing a connection with a support person when a user needs support.

[0554] The present invention is a system that provides solutions to inquiries related to content distribution services before users make them using an inquiry support app. The system uses a generative AI model to generate answers to user inquiries, aiming to reduce the burden on both users and support desks.

[0555] User query input

[0556] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, a user can input "When will the next episode be released?" or ask a similar question by voice.

[0557] Server-generated answers

[0558] The server passes the inquiry received from the user to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information to provide the optimal answer. The generative AI model uses natural language processing technology.

[0559] Displaying answers to users

[0560] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The next episode is scheduled to be released next Friday," the terminal displays this answer to the user. At this point, the user can decide whether the provided answer solved their problem.

[0561] If the issue persists, further support is available

[0562] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains more appropriate information about the congestion status of the counter. The server monitors the waiting time and congestion status of the counter in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0563] Counter connection

[0564] If the user checks the congestion status and still wishes to connect to a hotline, the terminal sends a connection request to the server. The server receives this request and attempts to connect to the specified hotline. This ensures that the user is connected to the correct hotline at the optimal time and receives support to resolve the problem. This process reduces waiting times for users and allows them to receive support efficiently.

[0565] Specific examples

[0566] For example, if a user inquires about a problem such as "I can't play the currently streaming episode," the user enters this information into the app. The server uses a generative AI model to generate a response such as "If you can't play the episode, first check your internet connection. Then try restarting your device," and provides it to the user. If the response does not resolve the problem, the server obtains the congestion status of the streaming service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0567] Prompt Sentence Examples

[0568] "How do I resolve episode playback errors?"

[0569] "I want to know when the next episode will be released."

[0570] In this way, the present invention allows users to quickly obtain the appropriate information and connect them to the correct point of contact as needed, thereby increasing the efficiency of the inquiry process and reducing the burden on support points.

[0571] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0572] Step 1: User enters inquiry

[0573] The user launches an inquiry support app on their smartphone or computer and inputs their inquiry by voice or text. The input data is sent to the server by the user's device. For example, if the user inputs "When will the next episode be released?", the text data is sent to the server.

[0574] Step 2: The server passes the query to the generative AI model

[0575] The server passes the inquiry received from the user to the generative AI model. The generative AI model receives the input text data and generates an appropriate answer based on past inquiry data and related information. For example, in response to the text data "When will the next episode be released?", the generative AI model generates an answer such as "The next episode is scheduled to be released next Friday."

[0576] Step 3: The server generates a response and sends it to the user's device.

[0577] The server sends the answer obtained from the generative AI model to the user's device, which receives the answer and displays it to the user. For example, the answer sent from the server, "The next episode is scheduled to be released next Friday," is displayed on the user's device screen.

[0578] Step 4: User reviews answers and provides feedback

[0579] The user checks the answer displayed on the terminal and determines whether the problem has been solved. If the problem has not been solved, the user inputs feedback such as "The problem has not been solved." This feedback data is again sent from the user terminal to the server.

[0580] Step 5: The server obtains the congestion status of the counter.

[0581] The server receives feedback from the user and initiates operations to obtain the appropriate congestion status of the counter. The server monitors the congestion status of the counter in real time and obtains information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0582] Step 6: The server sends the congestion status to the user's device.

[0583] The server sends the obtained congestion status of the counter to the user's terminal. The user terminal receives this information and displays it to the user. For example, the information sent from the server, "The current congestion status of the counter is a waiting time of 10 minutes," is displayed on the user's terminal screen.

[0584] Step 7: If the user wants a direct connection

[0585] If the user checks the congestion status and still wishes to connect through the service window, the terminal sends a connection request to the server. This connection request data is sent from the user terminal to the server.

[0586] Step 8: The server attempts to connect to the gateway

[0587] The server receives a connection request from the user and attempts to connect to the specified support desk. Once the connection is established, the server sends a notification to the user's device that the connection is complete, allowing the user to contact the support desk. For example, the server may notify the user that "Connection has been established. An operator is waiting."

[0588] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0589] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[0590] User query input

[0591] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0592] Emotion recognition by emotion engine

[0593] The device sends the voice or text inquiry received from the user to the emotion engine. The emotion engine recognizes the emotion from the user's voice or text and sends the result to the server. For example, if the user is angry, the emotion engine recognizes it as "anger."

[0594] Server-generated answers

[0595] The device sends the emotion recognition results along with the query content to the server. The server passes this query content to the generative AI model, which then generates an appropriate response. At this time, the generative AI model also takes the emotion recognition results into account. For example, if the user is angry, it will provide a polite response that takes their emotion into consideration.

[0596] Displaying answers to users

[0597] The server sends the generated answer to the user's terminal, and the terminal displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time," the terminal displays this answer to the user. The user checks the answer and determines whether the problem has been solved.

[0598] If the issue persists, further support is available

[0599] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that the problem was not solved, the server obtains the appropriate congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0600] Counter connection

[0601] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[0602] Specific examples

[0603] For example, if a user inquires about a problem such as "slow internet speed," the user inputs this information into the app. The emotion engine recognizes the emotion from the user's voice as "anger." The server uses a generative AI model to generate a response such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the response does not resolve the problem, the server obtains the congestion status of the internet service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0604] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[0605] The processing flow will be explained below.

[0606] Step 1:

[0607] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[0608] Step 2:

[0609] The user inputs the inquiry by voice or text. For example, they might input something like "What is the credit card limit?" If they input the inquiry by voice, it is recorded as is, and if they input the inquiry by text, it is immediately entered into the text area.

[0610] Step 3:

[0611] The device sends the inquiry received from the user to the emotion engine, which analyzes the received voice or text data and recognizes the user's emotions, such as "anger," "sadness," and "joy."

[0612] Step 4:

[0613] The emotion engine sends information including the recognized user emotion to the server. The original query content is also transferred to the server together with the emotion recognition result.

[0614] Step 5:

[0615] The server passes the received inquiry and emotion recognition results to the generative AI model, which then generates the optimal answer based on past data and related information. For example, if the user expresses anger, the generative AI model will generate a polite answer that takes that emotion into consideration.

[0616] Step 6:

[0617] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer may contain specific information, such as "The standard credit card limit is 500,000 yen. Please feel free to contact us with any questions."

[0618] Step 7:

[0619] The device displays the received answer to the user, who can then review the answer and determine whether it has resolved the problem. Specifically, the answer is displayed on the device's screen, and if it was entered via voice, it is played back aloud.

[0620] Step 8:

[0621] If the user checks the answer and the problem is not solved, the device receives feedback from the user. The feedback "The problem was not solved" is sent. This is done by the user providing feedback by pressing a button, etc.

[0622] Step 9:

[0623] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0624] Step 10:

[0625] The server sends the congestion status to the user's terminal. The terminal displays the received congestion status to the user and asks whether the user wishes to connect. Here, the congestion status is displayed visually in a format that is easy for the user to understand.

[0626] Step 11:

[0627] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[0628] Step 12:

[0629] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available. The status is displayed to the user until the connection is complete.

[0630] Step 13:

[0631] After a user receives support at the customer service center, the terminal records the results and uses them as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[0632] By specifying the specific processing steps as described above, it becomes clear how the system operates and what advantages it provides to both the user and the call center.

[0633] Example 2

[0634] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0635] Conventional inquiry support systems often found it difficult to respond appropriately, taking into account the user's emotions. Furthermore, when the response was insufficient and the problem was not resolved, it was difficult to quickly connect the user to the appropriate help desk, which led to a decrease in user satisfaction. Therefore, there was a need for a system that could automate appropriate responses that took the user's emotions into account and quickly connect the user to the appropriate help desk if the problem was not resolved.

[0636] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0637] In this invention, the server includes a means for transmitting the content of an inquiry entered by a user terminal to an emotion recognition engine and analyzing the user's emotions, a means for passing the recognition results from the emotion recognition engine and the content of the inquiry to a generative AI model and generating an answer that takes emotion recognition into consideration, and a means for transmitting the generated answer to the user terminal and for the user terminal to display the generated answer. This enables an appropriate answer that takes the user's emotions into consideration, thereby improving user satisfaction. In addition, by including a means for obtaining feedback from the user, obtaining an appropriate counter congestion status if the problem is not resolved, displaying the congestion status to the user, and connecting to a counter if the user requests it, a prompt and appropriate response can be made.

[0638] "User" means an individual or organization that makes an inquiry using the inquiry support app.

[0639] A "terminal" is an electronic device used by a user, such as a smartphone, computer, or tablet.

[0640] "Inquiry content" refers to questions or requests that a user inputs through voice or text.

[0641] "Speech or text" refers to textual or audio information that a user inputs into a terminal.

[0642] An "emotion recognition engine" is software or hardware that analyzes and recognizes a user's emotions from input voice or text.

[0643] The "server" is a computer system that processes the inquiry content and emotion recognition results, and generates and manages answers using a generative AI model.

[0644] A "generative AI model" is an artificial intelligence algorithm that generates appropriate answers based on input data.

[0645] A "prompt sentence" is an input sentence that prompts a generative AI model to generate an appropriate answer.

[0646] "Feedback" refers to the evaluation or opinion that a user submits after reviewing an answer.

[0647] The "contact point" refers to a department or service provider that handles inquiries from users.

[0648] "Congestion status" is information indicating the current waiting time at the counter and the status of resources available for processing.

[0649] A "connection" is the establishment of communication that allows a user to make a direct inquiry to a customer service center.

[0650] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion recognition engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[0651] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0652] The device sends the voice or text query received from the user to the emotion recognition engine. The emotion recognition engine uses speech recognition software (e.g., the SpeechRecognition library) and text analysis algorithms (e.g., the Python NLTK library) to recognize the user's emotion. For example, if the user types "What's my credit card limit?", the engine recognizes the user's emotion as "neutral."

[0653] After the emotion recognition engine obtains the recognition result, the device sends the query content and the emotion recognition result to the server. The server passes the query content and emotion recognition result to a generative AI model (e.g., OpenAI's GPT-3) and generates an appropriate response. At this time, the generative AI model takes the emotion recognition result into consideration. For example, if the user is angry, it will generate a kind and polite response that takes that emotion into account.

[0654] Example prompt sentence:

[0655] "The user has made the following query: 'What is the credit card limit?' The current sentiment is neutral. Please generate an appropriate answer."

[0656] The server sends the generated answer to the user's terminal, and the terminal displays this answer to the user. For example, if a response such as "The standard credit card limit is 500,000 yen. Please contact us anytime if you have any questions" is generated, this response will be displayed on the terminal. The user can check the response and determine whether the problem has been resolved.

[0657] If the user gives feedback that "the problem was not resolved," the device sends this feedback to the server. The server then obtains the appropriate counter congestion status using a real-time data collection API (for example, Firebase Realtime Database) and generates information such as "The current counter congestion status is a 10-minute wait time." This information is again sent to the user's device and displayed to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, and the server attempts to connect to the specified counter. This allows the user to be connected to the correct counter at the optimal time and receive support to resolve the problem.

[0658] As a concrete example, let's consider the case of an inquiry about the problem of "slow internet speed." The user enters this information into the app, and the emotion recognition engine recognizes "anger" from the voice. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the answer does not solve the problem, the server obtains the congestion status of the internet service center and displays this to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0659] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[0660] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0661] Step 1:

[0662] The user inputs the inquiry content by voice or text. The user starts the inquiry support app and inputs the inquiry content into a device such as a smartphone or computer using a text box or microphone. For example, the user inputs text or voice such as, "What is the credit card limit?"

[0663] Input: Enquiry (e.g. "What is the credit card limit?")

[0664] Output: Inquiry entered into the terminal

[0665] Step 2:

[0666] The device sends the query to the emotion recognition engine, which analyzes the user's emotions. The device sends the input voice or text to the emotion recognition engine, which uses voice recognition software and text analysis algorithms to analyze and recognize the user's emotions.

[0667] Input: User inquiry

[0668] Output: The emotion recognized by the emotion recognition engine (e.g. "Neutral")

[0669] Specific behavior: The terminal sends the text "What is the credit card limit?" to the emotion recognition engine, which recognizes it as "neutral."

[0670] Step 3:

[0671] The device sends the query and emotion recognition results to the server. The device generates an HTTP POST request to send the query and emotion recognition results to the server.

[0672] Input: User's inquiry and emotion recognition result (e.g., "neutral")

[0673] Output: Query content sent to the server and emotion recognition results

[0674] Specific operation: The device sends the query "What is the credit card limit?" and the emotion recognition result "Neutral" to the server as an HTTP POST request.

[0675] Step 4:

[0676] The server generates an answer using a generative AI model. The server inputs the query content and emotion recognition results into the generative AI model to obtain an appropriate answer. For example, OpenAI's GPT-3 is used as the generative AI model.

[0677] Input: Enquiry and emotion recognition result (e.g. "Neutral")

[0678] Output: The answer generated by the generative AI model

[0679] Specific operation: The server inputs the following prompt sentence into the generative AI model: "The user has made the following inquiry: 'What is the credit card limit?' The current emotion is neutral. Please generate an appropriate answer." The server receives the answer from the generative AI model: "The standard credit card limit is 500,000 yen."

[0680] Step 5:

[0681] The server transmits the generated response to the user terminal, and the server generates and transmits data for transmitting the generated response to the user terminal.

[0682] Input: The answer generated by the generative AI model

[0683] Output: The answer sent to the user's terminal

[0684] Specific operation: The server sends the HTTP response "The standard credit card limit is 500,000 yen" to the user's terminal.

[0685] Step 6:

[0686] The device displays the answer to the user. The device displays the answer it receives in the app so that the user can check it.

[0687] Input: Received Answer

[0688] Output: The answer displayed to the user

[0689] Specific operation: The device app displays to the user, "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time."

[0690] Step 7:

[0691] The user sends feedback to the terminal, and the user gives feedback on whether the answer was helpful in solving the problem.

[0692] Input: User feedback

[0693] Output: Feedback sent to the terminal

[0694] Specific operation: The user types "The problem was not resolved" into the terminal and submits it.

[0695] Step 8:

[0696] The device sends the feedback to the server. The device generates an HTTP request to send the user's feedback to the server.

[0697] Input: User feedback

[0698] Output: Feedback sent to the server

[0699] Specific operation: The device sends the user's feedback "The problem was not resolved" to the server.

[0700] Step 9:

[0701] The server obtains the congestion status and sends it to the terminal. The server obtains the congestion status of the appropriate counter through the real-time data collection API and sends it to the terminal.

[0702] Input: Requests to the server and congestion status from the real-time data collection API

[0703] Output: Congestion status information sent to the user terminal

[0704] Specific operation: The server obtains the congestion status using the Firebase Realtime Database API and sends the message "The current congestion status at the counter is a waiting time of 10 minutes" to the user's terminal.

[0705] Step 10:

[0706] The user sends a request for connection to the terminal. The user checks the congestion status and sends a request for connection through the service window to the terminal.

[0707] Input: User connection request

[0708] Output: Connection request sent to the device

[0709] Specific operation: The user enters "I would like to connect through the counter" into the terminal and sends it.

[0710] Step 11:

[0711] The terminal sends a connection request to the server. The terminal generates an HTTP request to send the connection request to the server.

[0712] Input: A user's connection request

[0713] Output: Connection request sent to server

[0714] Specific operation: The terminal sends the user's connection request "I want to connect to the window" to the server.

[0715] Step 12:

[0716] The server connects to the specified gateway. The server receives the connection request and performs the operations to connect the user to the specified gateway.

[0717] Input: Connection request

[0718] Output: Connection established between user and contact

[0719] Specific operation: The server sends a connection request to the call center system and connects the user to the specified contact point.

[0720] The above is the specific flow of the program processing of the invented system and details of each processing step.

[0721] (Application example 2)

[0722] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0723] The present invention aims to improve user satisfaction by combining a system that provides solutions before the user makes an inquiry with an emotion engine that recognizes the user's emotions, enabling responses tailored to the user's emotions. However, current inquiry systems have the problem of difficulty in properly understanding the user's emotions and customizing responses. Furthermore, if a user's inquiry cannot be properly addressed, it is necessary to quickly connect them to an appropriate point of contact. An efficient and effective system is needed to solve this problem.

[0724] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0725] In this invention, the server includes means for the user to input the inquiry content by voice or text, means for the server to generate an answer using a generative AI model, means for the user terminal to display the generated answer, means for obtaining feedback from the user and obtaining an appropriate counter congestion status if the problem is not resolved, means for displaying the congestion status to the user and connecting to a counter if the user requests it, means for recognizing the user's emotion using an emotion engine and passing the recognition result to the generative AI model, and means for the server to generate an answer taking the emotion recognition result into consideration. This makes it possible to provide a system that enables more appropriate and customized responses according to the user's emotion and improves user satisfaction.

[0726] A "user" is an individual or legal entity that uses the system to make an inquiry.

[0727] The "content of inquiry" refers to the specific content of the question or problem that the user wants to solve.

[0728] "Voice or text" refers to the form of means by which a user inputs inquiry content, with voice meaning voice data and text meaning character information.

[0729] A "server" is a computer system that receives, processes, and stores data from user terminals over a network.

[0730] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to generate appropriate answers to user inquiries.

[0731] A "user terminal" is a device that allows a user to access the system, input inquiries, and receive responses.

[0732] "Feedback" refers to the evaluations and comments that users make on the system's suggestions and answers.

[0733] The "contact point" refers to a department or person in charge who responds to user inquiries in person or by telephone or other means.

[0734] "Congestion status" refers to indicators such as the number of inquiries currently being handled at the counter and waiting times.

[0735] An "emotion engine" is an algorithm that recognizes a user's emotions from voice or text.

[0736] "Emotion recognition result" is data indicating the emotional state of the user analyzed by the emotion engine.

[0737] The present invention begins when a user uses an inquiry support app installed on a device such as a smartphone or computer to input their inquiry by voice or text. For example, the user may make an inquiry such as, "What is my credit card limit?" This input is sent to a server, and the emotion engine recognizes the user's emotion.

[0738] The server receives the user's inquiry and the emotion recognition results, inputs them into a generative AI model, and generates an appropriate answer. This generative AI model uses the prompt sentence to perform advanced natural language processing and provide a response that takes emotion into consideration. For example, if the user is angry, the model responds according to the user's emotion by adding expressions of gratitude or apology, such as "We apologize for the inconvenience."

[0739] The generated answer is sent to the user's terminal, where the user can review it. If the user is not satisfied with the answer, the terminal obtains feedback. If the feedback is "the problem was not resolved," the server further obtains the appropriate counter congestion status and provides the user with information such as "The current counter congestion status is a 10-minute wait time."

[0740] If a user checks the congestion status and wishes to connect to a hotline, the user's terminal sends a connection request to the server, and the server connects the user to the appropriate hotline. In this way, the user is connected to the appropriate hotline at the optimal time and can receive support to resolve their problem.

[0741] Specifically, by inputting the following prompt sentence into the generative AI model, an appropriate answer is generated:

[0742] "Please politely respond to angry users with the following question: My internet is slow. What should I do?"

[0743] This system uses the speech_recognition library for speech recognition and the transformers library for emotion recognition. It also uses the latest natural language processing technologies, such as GPT-3, for its generative AI model. It uses communication services such as the Twilio API for notification services. This enables highly efficient and appropriate customer support tailored to the user's emotional state. This system can significantly improve customer service in physical stores and increase user satisfaction.

[0744] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0745] Step 1:

[0746] The user inputs the inquiry by voice or text.

[0747] Input: User's voice or text query

[0748] How it works: A user uses an inquiry support app installed on a device such as a smartphone or computer and enters a specific question, such as "What is my credit card limit?"

[0749] Step 2:

[0750] The terminal sends the inquiry to the server.

[0751] Input: The audio or text data generated in step 1

[0752] Output: Query data arrives at the server

[0753] Operation: The terminal sends the user's inquiry to the server as voice data or text data.

[0754] Step 3:

[0755] The server converts the received voice data into text (in the case of voice input).

[0756] Input: Audio data

[0757] Output: Text data

[0758] How it works: The server uses the speech_recognition library to analyze the audio data and convert it to text data.

[0759] Step 4:

[0760] The server sends the received text data to the emotion engine to recognize the user's emotion.

[0761] Input: Text data of inquiry content

[0762] Output: Emotion recognition result data

[0763] How it works: The server uses the transformers library to parse the text data to determine the user's emotions, and then obtains the results. For example, it identifies emotions such as "anger," "joy," and "sadness."

[0764] Step 5:

[0765] The server inputs the query content and emotion recognition results into a generative AI model to generate an answer.

[0766] Input: Text data of inquiry content, emotion recognition result data

[0767] Output: Generated answer text

[0768] How it works: The server inputs a prompt into the generative AI model, which generates an answer based on the emotion recognition results. An example of a prompt is, "Please politely respond to the following question for an angry user: My internet speed is slow. What should I do?"

[0769] Step 6:

[0770] The server transmits the generated answer to the user terminal.

[0771] Input: Generated answer text

[0772] Output: The answer displayed on the user's terminal

[0773] Operation: The server sends the generated answer data to the user's terminal, which displays it.

[0774] Step 7:

[0775] The user reviews the answer and provides feedback on whether the problem was resolved.

[0776] Input: User feedback

[0777] Output: Feedback data

[0778] Action: The user reviews the answer they received and provides feedback via their device, such as "problem solved" or "not solved."

[0779] Step 8:

[0780] The server receives the feedback data and obtains the appropriate busy status of the counter if the problem is not resolved.

[0781] Input: Feedback data

[0782] Output: Counter congestion data

[0783] Operation: If the user's feedback is "not resolved", the server obtains the appropriate counter congestion status in real time.

[0784] Step 9:

[0785] The server transmits the congestion status of the counter to the user terminal, and when the user requests a counter connection, the connection is made.

[0786] Input: Counter congestion data, user connection requests

[0787] Output: Check connection to the counter

[0788] Operation: The server sends congestion status data to the user terminal, and if the user requests a gateway connection, processes the connection request and connects to the appropriate gateway.

[0789] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0790] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0791] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0792] [Third embodiment]

[0793] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0794] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0795] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0796] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0797] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0798] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0799] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0800] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0801] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0802] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0803] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0804] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0805] This invention is a system that uses an inquiry support app to provide solutions to users before they contact a call center. The system uses a generative AI model to generate answers to users' inquiries, aiming to reduce the burden on both users and call centers.

[0806] User query input

[0807] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0808] Server-generated answers

[0809] The device sends the inquiry received from the user to the server, which then passes the inquiry to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information and provide the optimal answer.

[0810] Displaying answers to users

[0811] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen," the terminal displays this answer to the user. At this point, the user decides whether the provided answer solved their problem.

[0812] If the issue persists, further support is available

[0813] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains further appropriate information about the congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0814] Counter connection

[0815] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[0816] Specific examples

[0817] For example, if a user inquires about a problem such as "slow internet speed," the user enters this information into the app. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow" and provides it to the user. If the answer does not solve the problem, the server obtains the congestion status of internet service centers and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0818] In this way, users can quickly obtain the right information and be connected to the right contact point when necessary, improving the efficiency of the inquiry process and reducing the burden on call centers.

[0819] The processing flow will be explained below.

[0820] Step 1:

[0821] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[0822] Step 2:

[0823] The user inputs the inquiry by voice or text. For example, they input "What is the credit card limit?"

[0824] Step 3:

[0825] The terminal sends the input query to the server, which converts the query into an appropriate format and sends it to the server via the network.

[0826] Step 4:

[0827] The server passes the received query to a generative AI model, which then generates the optimal answer based on past data and related information.

[0828] Step 5:

[0829] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer includes specific information, such as "The standard credit card limit is 500,000 yen."

[0830] Step 6:

[0831] The device displays the received answer to the user, who then reviews the answer and determines whether the problem has been resolved.

[0832] Step 7:

[0833] If the user checks the answer and the problem is not solved, the terminal receives feedback from the user, such as "The problem was not solved."

[0834] Step 8:

[0835] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0836] Step 9:

[0837] The server sends the congestion status to the user terminal, which displays the received congestion status to the user and asks whether the user wishes to connect.

[0838] Step 10:

[0839] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[0840] Step 11:

[0841] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available.

[0842] Step 12:

[0843] After the user receives support, the device records the situation and uses it as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[0844] Example 1

[0845] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0846] With the number of inquiries being sent to call centers increasing, many companies are seeking efficient responses. However, current inquiry response systems lack sufficient technology to provide prompt and appropriate responses to inquiries, which can lead to a decline in user satisfaction. Furthermore, if an inquiry is not resolved or if users need to check the busy status of the counter, they must manually complete numerous procedures, making it difficult to receive support at the optimal time. This increases the burden on call centers and poses a challenge in terms of reduced operational efficiency.

[0847] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0848] In this invention, the server includes: a means for a user to input an inquiry by voice or text; a means for a terminal to transmit the inquiry from the user to the server; a means for the server to generate an answer to the inquiry using a generative AI model; a means for the server to transmit the generated answer to the terminal; a means for the terminal to display the generated answer; a means for receiving feedback from the user and, if the problem is not resolved, obtaining the congestion status of the counter; a means for transmitting the congestion status to the terminal and having the terminal display the congestion status to the user; and a means for the terminal to transmit a connection request to the server when the user requests a connection to the counter, and the server to attempt to connect to the specified counter. This allows users to quickly obtain automatically generated answers, check the congestion status of the counters in real time, and receive support at the optimal time. It also reduces the burden on call centers and improves business efficiency.

[0849] "User" refers to an individual or corporation that uses the system to enter inquiries and receive responses.

[0850] "Terminal" refers to the electronic device used by a User to launch the Inquiry Support App, input inquiry details, and communicate with the Server. Examples include smartphones and computers.

[0851] "Server" refers to the central system that receives the query, generates a response using a generative AI model, and sends the response to the device.

[0852] "Generative AI model" refers to an artificial intelligence algorithm and its software that generates appropriate answers to user inquiries. Specific examples include models that use natural language processing technology.

[0853] "Feedback" refers to the act of a user evaluating whether a problem has been solved by a provided answer or expressing dissatisfaction.

[0854] "Counter congestion status" refers to information that indicates the counter's availability during a specific time period. For example, it includes the current waiting time and the number of available applications.

[0855] A "connection request" refers to an action in which a user requests a connection to a server via a terminal.

[0856] "Means for generating an answer" refers to the function by which the server uses a generative AI model to create an answer to a user's inquiry.

[0857] This invention is a system that uses an inquiry support app to provide solutions before users contact a call center. The aim is to reduce the burden on both users and call centers. This system is composed of devices such as smartphones and computers, a server, and a generative AI model.

[0858] User query input

[0859] The user launches an inquiry support app installed on a device such as a smartphone or computer. The user can input their inquiry using text or voice. For example, they might input a question such as, "What is my credit card limit?" The device is equipped with text analysis software that analyzes the input and converts it into text data. In the case of voice input, voice recognition software is used to convert the voice data into text data.

[0860] Server-generated answers

[0861] The device sends the query entered by the user to the server in the form of an HTTP request using a REST API. The server then performs preprocessing such as text cleaning and tokenization on the received data before passing it to the generative AI model.

[0862] The generative AI model uses an algorithm that uses natural language processing technology, such as OpenAI's GPT-3. The server inputs the query into this model and generates an appropriate answer. For example, the server might generate an answer such as, "The standard credit card limit is 500,000 yen."

[0863] Displaying answers to users

[0864] The server sends the generated answer to the device, and the device displays the answer to the user. For example, the app displays "The standard credit card limit is 500,000 yen." The user checks this answer and decides whether the problem has been resolved.

[0865] If the issue persists, further support is available

[0866] If the user checks the answer and gives feedback that "the problem was not solved," the device sends this feedback to the server. The server uses congestion analysis software to check the congestion status of specific counters in real time. For example, it generates information such as "The current congestion status of the counter is a waiting time of 10 minutes" and sends it to the device.

[0867] Counter connection

[0868] The terminal displays the received congestion information to the user, and if the user wishes to connect to a counter, the terminal sends a connection request to the server. The server uses call center connection software such as a PBX system to attempt to connect to the specified counter. This allows the user to be connected to the correct counter at the appropriate time.

[0869] Specific examples

[0870] Example 1: Credit card inquiry

[0871] The user enters the text "What is the credit card limit?", and the server uses the generative AI model to generate an answer such as "The standard credit card limit is 500,000 yen." This answer is then displayed on the device.

[0872] Example 2: Querying Internet speed

[0873] The user enters the text "My internet speed is slow," and the server uses a generative AI model to generate "solutions for slow internet speeds." This answer is displayed on the device. If the user is unable to resolve the problem, the device sends feedback to the server, which then obtains the congestion status of the counters and displays it to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, which connects the user to the appropriate counter.

[0874] These features allow users to quickly obtain appropriate information and receive support when needed, while also reducing the burden on call centers and improving operational efficiency.

[0875] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0876] Step 1:

[0877] The user launches the inquiry support app and inputs the inquiry content by voice or text. For example, if the user inputs text, such as "What is the credit card limit?", the voice input is converted into text data using voice recognition software.

[0878] Input: Voice or text inquiry

[0879] Output: Inquiry content in text format

[0880] Step 2:

[0881] The terminal sends the user's inquiry to the server. The terminal sends the inquiry as text data to the server in HTTP request format. For example, if the inquiry is text data such as "What is the credit card limit?", it is sent to the server via the REST API.

[0882] Input: Inquiry details in text format

[0883] Output: Text data sent as an HTTP request

[0884] Step 3:

[0885] The server analyzes the query data received from the device and prepares it for passing to the generative AI model. Specifically, it cleans and tokenizes the text data and converts it into an appropriate format, for example, by removing unnecessary characters and splitting words.

[0886] Input: Text data received as an HTTP request

[0887] Output: Preprocessed data to feed into a generative AI model

[0888] Step 4:

[0889] The server inputs the preprocessed data into a generative AI model to generate an answer. For example, a generative AI model such as GPT-3, which uses natural language processing technology, can be used to generate the optimal answer based on the inquiry. For example, it can generate an answer such as, "The standard credit card limit is 500,000 yen."

[0890] Input: Preprocessed data to feed into a generative AI model

[0891] Output: Answer text from the generative AI model

[0892] Step 5:

[0893] The server sends the generated answer to the terminal. The server sends the generated answer text to the terminal in the form of an HTTP response.

[0894] Input: Answer text from the generative AI model

[0895] Output: The answer text sent as the HTTP response

[0896] Step 6:

[0897] The terminal displays the answer received from the server on the user interface. The terminal displays an answer such as "The standard credit card limit is 500,000 yen" on the application screen.

[0898] Input: The answer text received as an HTTP response

[0899] Output: What is displayed in the user interface

[0900] Step 7:

[0901] If the user checks the answer and the problem is not solved, the user inputs feedback such as "The problem was not solved." The user inputs the feedback content in text format, and the terminal transmits it to the server.

[0902] Input: Feedback text if issue persists

[0903] Output: Feedback data sent as an HTTP request

[0904] Step 8:

[0905] The server receives feedback from users and uses a congestion analysis system to obtain the congestion status of the counters. The server monitors the waiting time and number of receptions at specific counters in real time and generates appropriate congestion status information.

[0906] Input: Feedback data received as an HTTP request

[0907] Output: Congestion status information text

[0908] Step 9:

[0909] The server sends congestion information to the terminal, which then displays it on the user interface. For example, the app screen might say, "The current congestion status at the counter is a 10-minute wait."

[0910] Input: Congestion status information text

[0911] Output: What is displayed in the user interface

[0912] Step 10:

[0913] When a user desires to connect to the service desk, the user sends a request for connection to the service desk from the terminal to the server. For example, the request "I would like to connect to the service desk" is sent in text format.

[0914] Input: Request text for a connection

[0915] Output: Connection request data sent as an HTTP request

[0916] Step 11:

[0917] The server receives the connection request and attempts to connect the user to the appropriate agent using call center connection software, such as a PBX system, so that the user is connected to the appropriate agent at the optimal time.

[0918] Input: Connection request data received as an HTTP request

[0919] Output: Connection established with the window

[0920] (Application example 1)

[0921] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0922] Conventional inquiry support systems have the problem that users must contact a call center every time they have an inquiry, which takes time and effort. In particular, inquiries related to content distribution services require a quick response, which places a heavy burden on both users and the support desk. In addition, there are cases where users are kept waiting for a long time depending on the number of calls at the desk, which reduces user satisfaction.

[0923] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0924] In this invention, the server includes a means for a user to input an inquiry related to the content distribution service by voice or text, a means for generating an answer using a generative AI model, a means for a user terminal to display the generated answer, a means for obtaining feedback from the user and obtaining an appropriate congestion status of the support desk if the problem is not resolved, and a means for displaying the congestion status to the user and connecting to the support desk in real time if the user requests a connection to the support desk. This allows the user to quickly obtain the information they need and smoothly connect to the support desk even if the problem is not resolved.

[0925] "Inquiry Content" refers to the specific question or problem that a user enters for support.

[0926] "Voice or text input means" refers to a means by which a user inputs inquiry details into a terminal using voice recognition or text input.

[0927] A "generative AI model" is an artificial intelligence model that uses natural language processing to automatically generate appropriate answers to user inquiries.

[0928] "Means for generating an answer" refers to a means for creating an answer based on the user's inquiry using a generative AI model.

[0929] A "user terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[0930] The "means for displaying the generated answer" is a means for displaying the answer generated by the server on the user terminal in a form that the user can confirm.

[0931] A "means for obtaining feedback" is a means for users to gather ratings and additional opinions about the answers provided.

[0932] The "means for acquiring the appropriate congestion status of the counter" is a means for the server to acquire the current waiting time and congestion status of the call center or support counter in real time.

[0933] The "means for displaying the congestion status to the user" is a means for displaying the acquired information on the congestion status of the counter on the user terminal.

[0934] The "means for establishing a connection when a customer service connection is desired" refers to the means by which the server establishes a connection to an appropriate customer service point when a user desires to connect to a support center.

[0935] A "content distribution service" is a service that provides digital content such as video and audio to users via the Internet.

[0936] "Content distribution service-related inquiries" refer to questions or problems related to content distribution services, such as content playback, distribution schedules, and account management.

[0937] The "means for connecting to a support desk in real time" is a means for instantly establishing a connection with a support person when a user needs support.

[0938] The present invention is a system that provides solutions to inquiries related to content distribution services before users make them using an inquiry support app. The system uses a generative AI model to generate answers to user inquiries, aiming to reduce the burden on both users and support desks.

[0939] User query input

[0940] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, a user can input "When will the next episode be released?" or ask a similar question by voice.

[0941] Server-generated answers

[0942] The server passes the inquiry received from the user to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information to provide the optimal answer. The generative AI model uses natural language processing technology.

[0943] Displaying answers to users

[0944] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The next episode is scheduled to be released next Friday," the terminal displays this answer to the user. At this point, the user can decide whether the provided answer solved their problem.

[0945] If the issue persists, further support is available

[0946] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains more appropriate information about the congestion status of the counter. The server monitors the waiting time and congestion status of the counter in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0947] Counter connection

[0948] If the user checks the congestion status and still wishes to connect to a hotline, the terminal sends a connection request to the server. The server receives this request and attempts to connect to the specified hotline. This ensures that the user is connected to the correct hotline at the optimal time and receives support to resolve the problem. This process reduces waiting times for users and allows them to receive support efficiently.

[0949] Specific examples

[0950] For example, if a user inquires about a problem such as "I can't play the currently streaming episode," the user enters this information into the app. The server uses a generative AI model to generate a response such as "If you can't play the episode, first check your internet connection. Then try restarting your device," and provides it to the user. If the response does not resolve the problem, the server obtains the congestion status of the streaming service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0951] Prompt Sentence Examples

[0952] "How do I resolve episode playback errors?"

[0953] "I want to know when the next episode will be released."

[0954] In this way, the present invention allows users to quickly obtain the appropriate information and connect them to the correct point of contact as needed, thereby increasing the efficiency of the inquiry process and reducing the burden on support points.

[0955] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0956] Step 1: User enters inquiry

[0957] The user launches an inquiry support app on their smartphone or computer and inputs their inquiry by voice or text. The input data is sent to the server by the user's device. For example, if the user inputs "When will the next episode be released?", the text data is sent to the server.

[0958] Step 2: The server passes the query to the generative AI model

[0959] The server passes the inquiry received from the user to the generative AI model. The generative AI model receives the input text data and generates an appropriate answer based on past inquiry data and related information. For example, in response to the text data "When will the next episode be released?", the generative AI model generates an answer such as "The next episode is scheduled to be released next Friday."

[0960] Step 3: The server generates a response and sends it to the user's device.

[0961] The server sends the answer obtained from the generative AI model to the user's device, which receives the answer and displays it to the user. For example, the answer sent from the server, "The next episode is scheduled to be released next Friday," is displayed on the user's device screen.

[0962] Step 4: User reviews answers and provides feedback

[0963] The user checks the answer displayed on the terminal and determines whether the problem has been solved. If the problem has not been solved, the user inputs feedback such as "The problem has not been solved." This feedback data is again sent from the user terminal to the server.

[0964] Step 5: The server obtains the congestion status of the counter.

[0965] The server receives feedback from the user and initiates operations to obtain the appropriate congestion status of the counter. The server monitors the congestion status of the counter in real time and obtains information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[0966] Step 6: The server sends the congestion status to the user's device.

[0967] The server sends the obtained congestion status of the counter to the user's terminal. The user terminal receives this information and displays it to the user. For example, the information sent from the server, "The current congestion status of the counter is a waiting time of 10 minutes," is displayed on the user's terminal screen.

[0968] Step 7: If the user wants a direct connection

[0969] If the user checks the congestion status and still wishes to connect through the service window, the terminal sends a connection request to the server. This connection request data is sent from the user terminal to the server.

[0970] Step 8: The server attempts to connect to the gateway

[0971] The server receives a connection request from the user and attempts to connect to the specified support desk. Once the connection is established, the server sends a notification to the user's device that the connection is complete, allowing the user to contact the support desk. For example, the server may notify the user that "Connection has been established. An operator is waiting."

[0972] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0973] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[0974] User query input

[0975] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[0976] Emotion recognition by emotion engine

[0977] The device sends the voice or text inquiry received from the user to the emotion engine. The emotion engine recognizes the emotion from the user's voice or text and sends the result to the server. For example, if the user is angry, the emotion engine recognizes it as "anger."

[0978] Server-generated answers

[0979] The device sends the emotion recognition results along with the query content to the server. The server passes this query content to the generative AI model, which then generates an appropriate response. At this time, the generative AI model also takes the emotion recognition results into account. For example, if the user is angry, it will provide a polite response that takes their emotion into consideration.

[0980] Displaying answers to users

[0981] The server sends the generated answer to the user's terminal, and the terminal displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time," the terminal displays this answer to the user. The user checks the answer and determines whether the problem has been solved.

[0982] If the issue persists, further support is available

[0983] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that the problem was not solved, the server obtains the appropriate congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[0984] Counter connection

[0985] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[0986] Specific examples

[0987] For example, if a user inquires about a problem such as "slow internet speed," the user inputs this information into the app. The emotion engine recognizes the emotion from the user's voice as "anger." The server uses a generative AI model to generate a response such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the response does not resolve the problem, the server obtains the congestion status of the internet service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[0988] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[0989] The processing flow will be explained below.

[0990] Step 1:

[0991] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[0992] Step 2:

[0993] The user inputs the inquiry by voice or text. For example, they might input something like "What is the credit card limit?" If they input the inquiry by voice, it is recorded as is, and if they input the inquiry by text, it is immediately entered into the text area.

[0994] Step 3:

[0995] The device sends the inquiry received from the user to the emotion engine, which analyzes the received voice or text data and recognizes the user's emotions, such as "anger," "sadness," and "joy."

[0996] Step 4:

[0997] The emotion engine sends information including the recognized user emotion to the server. The original query content is also transferred to the server together with the emotion recognition result.

[0998] Step 5:

[0999] The server passes the received inquiry and emotion recognition results to the generative AI model, which then generates the optimal answer based on past data and related information. For example, if the user expresses anger, the generative AI model will generate a polite answer that takes that emotion into consideration.

[1000] Step 6:

[1001] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer may contain specific information, such as "The standard credit card limit is 500,000 yen. Please feel free to contact us with any questions."

[1002] Step 7:

[1003] The device displays the received answer to the user, who can then review the answer and determine whether it has resolved the problem. Specifically, the answer is displayed on the device's screen, and if it was entered via voice, it is played back aloud.

[1004] Step 8:

[1005] If the user checks the answer and the problem is not solved, the device receives feedback from the user. The feedback "The problem was not solved" is sent. This is done by the user providing feedback by pressing a button, etc.

[1006] Step 9:

[1007] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[1008] Step 10:

[1009] The server sends the congestion status to the user's terminal. The terminal displays the received congestion status to the user and asks whether the user wishes to connect. Here, the congestion status is displayed visually in a format that is easy for the user to understand.

[1010] Step 11:

[1011] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[1012] Step 12:

[1013] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available. The status is displayed to the user until the connection is complete.

[1014] Step 13:

[1015] After a user receives support at the customer service center, the terminal records the results and uses them as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[1016] By specifying the specific processing steps as described above, it becomes clear how the system operates and what advantages it provides to both the user and the call center.

[1017] Example 2

[1018] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1019] Conventional inquiry support systems often found it difficult to respond appropriately, taking into account the user's emotions. Furthermore, when the response was insufficient and the problem was not resolved, it was difficult to quickly connect the user to the appropriate help desk, which led to a decrease in user satisfaction. Therefore, there was a need for a system that could automate appropriate responses that took the user's emotions into account and quickly connect the user to the appropriate help desk if the problem was not resolved.

[1020] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1021] In this invention, the server includes a means for transmitting the content of an inquiry entered by a user terminal to an emotion recognition engine and analyzing the user's emotions, a means for passing the recognition results from the emotion recognition engine and the content of the inquiry to a generative AI model and generating an answer that takes emotion recognition into consideration, and a means for transmitting the generated answer to the user terminal and for the user terminal to display the generated answer. This enables an appropriate answer that takes the user's emotions into consideration, thereby improving user satisfaction. In addition, by including a means for obtaining feedback from the user, obtaining an appropriate counter congestion status if the problem is not resolved, displaying the congestion status to the user, and connecting to a counter if the user requests it, a prompt and appropriate response can be made.

[1022] "User" means an individual or organization that makes an inquiry using the inquiry support app.

[1023] A "terminal" is an electronic device used by a user, such as a smartphone, computer, or tablet.

[1024] "Inquiry content" refers to questions or requests that a user inputs through voice or text.

[1025] "Speech or text" refers to textual or audio information that a user inputs into a terminal.

[1026] An "emotion recognition engine" is software or hardware that analyzes and recognizes a user's emotions from input voice or text.

[1027] The "server" is a computer system that processes the inquiry content and emotion recognition results, and generates and manages answers using a generative AI model.

[1028] A "generative AI model" is an artificial intelligence algorithm that generates appropriate answers based on input data.

[1029] A "prompt sentence" is an input sentence that prompts a generative AI model to generate an appropriate answer.

[1030] "Feedback" refers to the evaluation or opinion that a user submits after reviewing an answer.

[1031] The "contact point" refers to a department or service provider that handles inquiries from users.

[1032] "Congestion status" is information indicating the current waiting time at the counter and the status of resources available for processing.

[1033] A "connection" is the establishment of communication that allows a user to make a direct inquiry to a customer service center.

[1034] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion recognition engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[1035] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[1036] The device sends the voice or text query received from the user to the emotion recognition engine. The emotion recognition engine uses speech recognition software (e.g., the SpeechRecognition library) and text analysis algorithms (e.g., the Python NLTK library) to recognize the user's emotion. For example, if the user types "What's my credit card limit?", the engine recognizes the user's emotion as "neutral."

[1037] After the emotion recognition engine obtains the recognition result, the device sends the query content and the emotion recognition result to the server. The server passes the query content and emotion recognition result to a generative AI model (e.g., OpenAI's GPT-3) and generates an appropriate response. At this time, the generative AI model takes the emotion recognition result into consideration. For example, if the user is angry, it will generate a kind and polite response that takes that emotion into account.

[1038] Example prompt sentence:

[1039] "The user has made the following query: 'What is the credit card limit?' The current sentiment is neutral. Please generate an appropriate answer."

[1040] The server sends the generated answer to the user's terminal, and the terminal displays this answer to the user. For example, if a response such as "The standard credit card limit is 500,000 yen. Please contact us anytime if you have any questions" is generated, this response will be displayed on the terminal. The user can check the response and determine whether the problem has been resolved.

[1041] If the user gives feedback that "the problem was not resolved," the device sends this feedback to the server. The server then obtains the appropriate counter congestion status using a real-time data collection API (for example, Firebase Realtime Database) and generates information such as "The current counter congestion status is a 10-minute wait time." This information is again sent to the user's device and displayed to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, and the server attempts to connect to the specified counter. This allows the user to be connected to the correct counter at the optimal time and receive support to resolve the problem.

[1042] As a concrete example, let's consider the case of an inquiry about the problem of "slow internet speed." The user enters this information into the app, and the emotion recognition engine recognizes "anger" from the voice. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the answer does not solve the problem, the server obtains the congestion status of the internet service center and displays this to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[1043] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[1044] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1045] Step 1:

[1046] The user inputs the inquiry content by voice or text. The user starts the inquiry support app and inputs the inquiry content into a device such as a smartphone or computer using a text box or microphone. For example, the user inputs text or voice such as, "What is the credit card limit?"

[1047] Input: Enquiry (e.g. "What is the credit card limit?")

[1048] Output: Inquiry entered into the terminal

[1049] Step 2:

[1050] The device sends the query to the emotion recognition engine, which analyzes the user's emotions. The device sends the input voice or text to the emotion recognition engine, which uses voice recognition software and text analysis algorithms to analyze and recognize the user's emotions.

[1051] Input: User inquiry

[1052] Output: The emotion recognized by the emotion recognition engine (e.g. "Neutral")

[1053] Specific behavior: The terminal sends the text "What is the credit card limit?" to the emotion recognition engine, which recognizes it as "neutral."

[1054] Step 3:

[1055] The device sends the query and emotion recognition results to the server. The device generates an HTTP POST request to send the query and emotion recognition results to the server.

[1056] Input: User's inquiry and emotion recognition result (e.g., "neutral")

[1057] Output: Query content sent to the server and emotion recognition results

[1058] Specific operation: The device sends the query "What is the credit card limit?" and the emotion recognition result "Neutral" to the server as an HTTP POST request.

[1059] Step 4:

[1060] The server generates an answer using a generative AI model. The server inputs the query content and emotion recognition results into the generative AI model to obtain an appropriate answer. For example, OpenAI's GPT-3 is used as the generative AI model.

[1061] Input: Enquiry and emotion recognition result (e.g. "Neutral")

[1062] Output: The answer generated by the generative AI model

[1063] Specific operation: The server inputs the following prompt sentence into the generative AI model: "The user has made the following inquiry: 'What is the credit card limit?' The current emotion is neutral. Please generate an appropriate answer." The server receives the answer from the generative AI model: "The standard credit card limit is 500,000 yen."

[1064] Step 5:

[1065] The server transmits the generated response to the user terminal, and the server generates and transmits data for transmitting the generated response to the user terminal.

[1066] Input: The answer generated by the generative AI model

[1067] Output: The answer sent to the user's terminal

[1068] Specific operation: The server sends the HTTP response "The standard credit card limit is 500,000 yen" to the user's terminal.

[1069] Step 6:

[1070] The device displays the answer to the user. The device displays the answer it receives in the app so that the user can check it.

[1071] Input: Received Answer

[1072] Output: The answer displayed to the user

[1073] Specific operation: The device app displays to the user, "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time."

[1074] Step 7:

[1075] The user sends feedback to the terminal, and the user gives feedback on whether the answer was helpful in solving the problem.

[1076] Input: User feedback

[1077] Output: Feedback sent to the terminal

[1078] Specific operation: The user types "The problem was not resolved" into the terminal and submits it.

[1079] Step 8:

[1080] The device sends the feedback to the server. The device generates an HTTP request to send the user's feedback to the server.

[1081] Input: User feedback

[1082] Output: Feedback sent to the server

[1083] Specific operation: The device sends the user's feedback "The problem was not resolved" to the server.

[1084] Step 9:

[1085] The server obtains the congestion status and sends it to the terminal. The server obtains the congestion status of the appropriate counter through the real-time data collection API and sends it to the terminal.

[1086] Input: Requests to the server and congestion status from the real-time data collection API

[1087] Output: Congestion status information sent to the user terminal

[1088] Specific operation: The server obtains the congestion status using the Firebase Realtime Database API and sends the message "The current congestion status at the counter is a waiting time of 10 minutes" to the user's terminal.

[1089] Step 10:

[1090] The user sends a request for connection to the terminal. The user checks the congestion status and sends a request for connection through the service window to the terminal.

[1091] Input: User connection request

[1092] Output: Connection request sent to the device

[1093] Specific operation: The user enters "I would like to connect through the counter" into the terminal and sends it.

[1094] Step 11:

[1095] The terminal sends a connection request to the server. The terminal generates an HTTP request to send the connection request to the server.

[1096] Input: A user's connection request

[1097] Output: Connection request sent to server

[1098] Specific operation: The terminal sends the user's connection request "I want to connect to the window" to the server.

[1099] Step 12:

[1100] The server connects to the specified gateway. The server receives the connection request and performs the operations to connect the user to the specified gateway.

[1101] Input: Connection request

[1102] Output: Connection established between user and contact

[1103] Specific operation: The server sends a connection request to the call center system and connects the user to the specified contact point.

[1104] The above is the specific flow of the program processing of the invented system and details of each processing step.

[1105] (Application example 2)

[1106] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1107] The present invention aims to improve user satisfaction by combining a system that provides solutions before the user makes an inquiry with an emotion engine that recognizes the user's emotions, enabling responses tailored to the user's emotions. However, current inquiry systems have the problem of difficulty in properly understanding the user's emotions and customizing responses. Furthermore, if a user's inquiry cannot be properly addressed, it is necessary to quickly connect them to an appropriate point of contact. An efficient and effective system is needed to solve this problem.

[1108] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1109] In this invention, the server includes means for the user to input the inquiry content by voice or text, means for the server to generate an answer using a generative AI model, means for the user terminal to display the generated answer, means for obtaining feedback from the user and obtaining an appropriate counter congestion status if the problem is not resolved, means for displaying the congestion status to the user and connecting to a counter if the user requests it, means for recognizing the user's emotion using an emotion engine and passing the recognition result to the generative AI model, and means for the server to generate an answer taking the emotion recognition result into consideration. This makes it possible to provide a system that enables more appropriate and customized responses according to the user's emotion and improves user satisfaction.

[1110] A "user" is an individual or legal entity that uses the system to make an inquiry.

[1111] The "content of inquiry" refers to the specific content of the question or problem that the user wants to solve.

[1112] "Voice or text" refers to the form of means by which a user inputs inquiry content, with voice meaning voice data and text meaning character information.

[1113] A "server" is a computer system that receives, processes, and stores data from user terminals over a network.

[1114] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to generate appropriate answers to user inquiries.

[1115] A "user terminal" is a device that allows a user to access the system, input inquiries, and receive responses.

[1116] "Feedback" refers to the evaluations and comments that users make on the system's suggestions and answers.

[1117] The "contact point" refers to a department or person in charge who responds to user inquiries in person or by telephone or other means.

[1118] "Congestion status" refers to indicators such as the number of inquiries currently being handled at the counter and waiting times.

[1119] An "emotion engine" is an algorithm that recognizes a user's emotions from voice or text.

[1120] "Emotion recognition result" is data indicating the emotional state of the user analyzed by the emotion engine.

[1121] The present invention begins when a user uses an inquiry support app installed on a device such as a smartphone or computer to input their inquiry by voice or text. For example, the user may make an inquiry such as, "What is my credit card limit?" This input is sent to a server, and the emotion engine recognizes the user's emotion.

[1122] The server receives the user's inquiry and the emotion recognition results, inputs them into a generative AI model, and generates an appropriate answer. This generative AI model uses the prompt sentence to perform advanced natural language processing and provide a response that takes emotion into consideration. For example, if the user is angry, the model responds according to the user's emotion by adding expressions of gratitude or apology, such as "We apologize for the inconvenience."

[1123] The generated answer is sent to the user's terminal, where the user can review it. If the user is not satisfied with the answer, the terminal obtains feedback. If the feedback is "the problem was not resolved," the server further obtains the appropriate counter congestion status and provides the user with information such as "The current counter congestion status is a 10-minute wait time."

[1124] If a user checks the congestion status and wishes to connect to a hotline, the user's terminal sends a connection request to the server, and the server connects the user to the appropriate hotline. In this way, the user is connected to the appropriate hotline at the optimal time and can receive support to resolve their problem.

[1125] Specifically, by inputting the following prompt sentence into the generative AI model, an appropriate answer is generated:

[1126] "Please politely respond to angry users with the following question: My internet is slow. What should I do?"

[1127] This system uses the speech_recognition library for speech recognition and the transformers library for emotion recognition. It also uses the latest natural language processing technologies, such as GPT-3, for its generative AI model. It uses communication services such as the Twilio API for notification services. This enables highly efficient and appropriate customer support tailored to the user's emotional state. This system can significantly improve customer service in physical stores and increase user satisfaction.

[1128] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1129] Step 1:

[1130] The user inputs the inquiry by voice or text.

[1131] Input: User's voice or text query

[1132] How it works: A user uses an inquiry support app installed on a device such as a smartphone or computer and enters a specific question, such as "What is my credit card limit?"

[1133] Step 2:

[1134] The terminal sends the inquiry to the server.

[1135] Input: The audio or text data generated in step 1

[1136] Output: Query data arrives at the server

[1137] Operation: The terminal sends the user's inquiry to the server as voice data or text data.

[1138] Step 3:

[1139] The server converts the received voice data into text (in the case of voice input).

[1140] Input: Audio data

[1141] Output: Text data

[1142] How it works: The server uses the speech_recognition library to analyze the audio data and convert it to text data.

[1143] Step 4:

[1144] The server sends the received text data to the emotion engine to recognize the user's emotion.

[1145] Input: Text data of inquiry content

[1146] Output: Emotion recognition result data

[1147] How it works: The server uses the transformers library to parse the text data to determine the user's emotions, and then obtains the results. For example, it identifies emotions such as "anger," "joy," and "sadness."

[1148] Step 5:

[1149] The server inputs the query content and emotion recognition results into a generative AI model to generate an answer.

[1150] Input: Text data of inquiry content, emotion recognition result data

[1151] Output: Generated answer text

[1152] How it works: The server inputs a prompt into the generative AI model, which generates an answer based on the emotion recognition results. An example of a prompt is, "Please politely respond to the following question for an angry user: My internet speed is slow. What should I do?"

[1153] Step 6:

[1154] The server transmits the generated answer to the user terminal.

[1155] Input: Generated answer text

[1156] Output: The answer displayed on the user's terminal

[1157] Operation: The server sends the generated answer data to the user's terminal, which displays it.

[1158] Step 7:

[1159] The user reviews the answer and provides feedback on whether the problem was resolved.

[1160] Input: User feedback

[1161] Output: Feedback data

[1162] Action: The user reviews the answer they received and provides feedback via their device, such as "problem solved" or "not solved."

[1163] Step 8:

[1164] The server receives the feedback data and obtains the appropriate busy status of the counter if the problem is not resolved.

[1165] Input: Feedback data

[1166] Output: Counter congestion data

[1167] Operation: If the user's feedback is "not resolved", the server obtains the appropriate counter congestion status in real time.

[1168] Step 9:

[1169] The server transmits the congestion status of the counter to the user terminal, and when the user requests a counter connection, the connection is made.

[1170] Input: Counter congestion data, user connection requests

[1171] Output: Check connection to the counter

[1172] Operation: The server sends congestion status data to the user terminal, and if the user requests a gateway connection, processes the connection request and connects to the appropriate gateway.

[1173] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1174] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1175] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1176] [Fourth embodiment]

[1177] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1178] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1179] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1180] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1181] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1182] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1183] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1184] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1185] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1186] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1187] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1188] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1189] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1190] This invention is a system that uses an inquiry support app to provide solutions to users before they contact a call center. The system uses a generative AI model to generate answers to users' inquiries, aiming to reduce the burden on both users and call centers.

[1191] User query input

[1192] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[1193] Server-generated answers

[1194] The device sends the inquiry received from the user to the server, which then passes the inquiry to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information and provide the optimal answer.

[1195] Displaying answers to users

[1196] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen," the terminal displays this answer to the user. At this point, the user decides whether the provided answer solved their problem.

[1197] If the issue persists, further support is available

[1198] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains further appropriate information about the congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[1199] Counter connection

[1200] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[1201] Specific examples

[1202] For example, if a user inquires about a problem such as "slow internet speed," the user enters this information into the app. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow" and provides it to the user. If the answer does not solve the problem, the server obtains the congestion status of internet service centers and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[1203] In this way, users can quickly obtain the right information and be connected to the right contact point when necessary, improving the efficiency of the inquiry process and reducing the burden on call centers.

[1204] The processing flow will be explained below.

[1205] Step 1:

[1206] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[1207] Step 2:

[1208] The user inputs the inquiry by voice or text. For example, they input "What is the credit card limit?"

[1209] Step 3:

[1210] The terminal sends the input query to the server, which converts the query into an appropriate format and sends it to the server via the network.

[1211] Step 4:

[1212] The server passes the received query to a generative AI model, which then generates the optimal answer based on past data and related information.

[1213] Step 5:

[1214] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer includes specific information, such as "The standard credit card limit is 500,000 yen."

[1215] Step 6:

[1216] The device displays the received answer to the user, who then reviews the answer and determines whether the problem has been resolved.

[1217] Step 7:

[1218] If the user checks the answer and the problem is not solved, the terminal receives feedback from the user, such as "The problem was not solved."

[1219] Step 8:

[1220] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[1221] Step 9:

[1222] The server sends the congestion status to the user terminal, which displays the received congestion status to the user and asks whether the user wishes to connect.

[1223] Step 10:

[1224] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[1225] Step 11:

[1226] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available.

[1227] Step 12:

[1228] After the user receives support, the device records the situation and uses it as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[1229] Example 1

[1230] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1231] With the number of inquiries being sent to call centers increasing, many companies are seeking efficient responses. However, current inquiry response systems lack sufficient technology to provide prompt and appropriate responses to inquiries, which can lead to a decline in user satisfaction. Furthermore, if an inquiry is not resolved or if users need to check the busy status of the counter, they must manually complete numerous procedures, making it difficult to receive support at the optimal time. This increases the burden on call centers and poses a challenge in terms of reduced operational efficiency.

[1232] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1233] In this invention, the server includes: a means for a user to input an inquiry by voice or text; a means for a terminal to transmit the inquiry from the user to the server; a means for the server to generate an answer to the inquiry using a generative AI model; a means for the server to transmit the generated answer to the terminal; a means for the terminal to display the generated answer; a means for receiving feedback from the user and, if the problem is not resolved, obtaining the congestion status of the counter; a means for transmitting the congestion status to the terminal and having the terminal display the congestion status to the user; and a means for the terminal to transmit a connection request to the server when the user requests a connection to the counter, and the server to attempt to connect to the specified counter. This allows users to quickly obtain automatically generated answers, check the congestion status of the counters in real time, and receive support at the optimal time. It also reduces the burden on call centers and improves business efficiency.

[1234] "User" refers to an individual or corporation that uses the system to enter inquiries and receive responses.

[1235] "Terminal" refers to the electronic device used by a User to launch the Inquiry Support App, input inquiry details, and communicate with the Server. Examples include smartphones and computers.

[1236] "Server" refers to the central system that receives the query, generates a response using a generative AI model, and sends the response to the device.

[1237] "Generative AI model" refers to an artificial intelligence algorithm and its software that generates appropriate answers to user inquiries. Specific examples include models that use natural language processing technology.

[1238] "Feedback" refers to the act of a user evaluating whether a problem has been solved by a provided answer or expressing dissatisfaction.

[1239] "Counter congestion status" refers to information that indicates the counter's availability during a specific time period. For example, it includes the current waiting time and the number of available applications.

[1240] A "connection request" refers to an action in which a user requests a connection to a server via a terminal.

[1241] "Means for generating an answer" refers to the function by which the server uses a generative AI model to create an answer to a user's inquiry.

[1242] This invention is a system that uses an inquiry support app to provide solutions before users contact a call center. The aim is to reduce the burden on both users and call centers. This system is composed of devices such as smartphones and computers, a server, and a generative AI model.

[1243] User query input

[1244] The user launches an inquiry support app installed on a device such as a smartphone or computer. The user can input their inquiry using text or voice. For example, they might input a question such as, "What is my credit card limit?" The device is equipped with text analysis software that analyzes the input and converts it into text data. In the case of voice input, voice recognition software is used to convert the voice data into text data.

[1245] Server-generated answers

[1246] The device sends the query entered by the user to the server in the form of an HTTP request using a REST API. The server then performs preprocessing such as text cleaning and tokenization on the received data before passing it to the generative AI model.

[1247] The generative AI model uses an algorithm that uses natural language processing technology, such as OpenAI's GPT-3. The server inputs the query into this model and generates an appropriate answer. For example, the server might generate an answer such as, "The standard credit card limit is 500,000 yen."

[1248] Displaying answers to users

[1249] The server sends the generated answer to the device, and the device displays the answer to the user. For example, the app displays "The standard credit card limit is 500,000 yen." The user checks this answer and decides whether the problem has been resolved.

[1250] If the issue persists, further support is available

[1251] If the user checks the answer and gives feedback that "the problem was not solved," the device sends this feedback to the server. The server uses congestion analysis software to check the congestion status of specific counters in real time. For example, it generates information such as "The current congestion status of the counter is a waiting time of 10 minutes" and sends it to the device.

[1252] Counter connection

[1253] The terminal displays the received congestion information to the user, and if the user wishes to connect to a counter, the terminal sends a connection request to the server. The server uses call center connection software such as a PBX system to attempt to connect to the specified counter. This allows the user to be connected to the correct counter at the appropriate time.

[1254] Specific examples

[1255] Example 1: Credit card inquiry

[1256] The user enters the text "What is the credit card limit?", and the server uses the generative AI model to generate an answer such as "The standard credit card limit is 500,000 yen." This answer is then displayed on the device.

[1257] Example 2: Querying Internet speed

[1258] The user enters the text "My internet speed is slow," and the server uses a generative AI model to generate "solutions for slow internet speeds." This answer is displayed on the device. If the user is unable to resolve the problem, the device sends feedback to the server, which then obtains the congestion status of the counters and displays it to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, which connects the user to the appropriate counter.

[1259] These features allow users to quickly obtain appropriate information and receive support when needed, while also reducing the burden on call centers and improving operational efficiency.

[1260] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1261] Step 1:

[1262] The user launches the inquiry support app and inputs the inquiry content by voice or text. For example, if the user inputs text, such as "What is the credit card limit?", the voice input is converted into text data using voice recognition software.

[1263] Input: Voice or text inquiry

[1264] Output: Inquiry content in text format

[1265] Step 2:

[1266] The terminal sends the user's inquiry to the server. The terminal sends the inquiry as text data to the server in HTTP request format. For example, if the inquiry is text data such as "What is the credit card limit?", it is sent to the server via the REST API.

[1267] Input: Inquiry details in text format

[1268] Output: Text data sent as an HTTP request

[1269] Step 3:

[1270] The server analyzes the query data received from the device and prepares it for passing to the generative AI model. Specifically, it cleans and tokenizes the text data and converts it into an appropriate format, for example, by removing unnecessary characters and splitting words.

[1271] Input: Text data received as an HTTP request

[1272] Output: Preprocessed data to feed into a generative AI model

[1273] Step 4:

[1274] The server inputs the preprocessed data into a generative AI model to generate an answer. For example, a generative AI model such as GPT-3, which uses natural language processing technology, can be used to generate the optimal answer based on the inquiry. For example, it can generate an answer such as, "The standard credit card limit is 500,000 yen."

[1275] Input: Preprocessed data to feed into a generative AI model

[1276] Output: Answer text from the generative AI model

[1277] Step 5:

[1278] The server sends the generated answer to the terminal. The server sends the generated answer text to the terminal in the form of an HTTP response.

[1279] Input: Answer text from the generative AI model

[1280] Output: The answer text sent as the HTTP response

[1281] Step 6:

[1282] The terminal displays the answer received from the server on the user interface. The terminal displays an answer such as "The standard credit card limit is 500,000 yen" on the application screen.

[1283] Input: The answer text received as an HTTP response

[1284] Output: What is displayed in the user interface

[1285] Step 7:

[1286] If the user checks the answer and the problem is not solved, the user inputs feedback such as "The problem was not solved." The user inputs the feedback content in text format, and the terminal transmits it to the server.

[1287] Input: Feedback text if issue persists

[1288] Output: Feedback data sent as an HTTP request

[1289] Step 8:

[1290] The server receives feedback from users and uses a congestion analysis system to obtain the congestion status of the counters. The server monitors the waiting time and number of receptions at specific counters in real time and generates appropriate congestion status information.

[1291] Input: Feedback data received as an HTTP request

[1292] Output: Congestion status information text

[1293] Step 9:

[1294] The server sends congestion information to the terminal, which then displays it on the user interface. For example, the app screen might say, "The current congestion status at the counter is a 10-minute wait."

[1295] Input: Congestion status information text

[1296] Output: What is displayed in the user interface

[1297] Step 10:

[1298] When a user desires to connect to the service desk, the user sends a request for connection to the service desk from the terminal to the server. For example, the request "I would like to connect to the service desk" is sent in text format.

[1299] Input: Request text for a connection

[1300] Output: Connection request data sent as an HTTP request

[1301] Step 11:

[1302] The server receives the connection request and attempts to connect the user to the appropriate agent using call center connection software, such as a PBX system, so that the user is connected to the appropriate agent at the optimal time.

[1303] Input: Connection request data received as an HTTP request

[1304] Output: Connection established with the window

[1305] (Application example 1)

[1306] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1307] Conventional inquiry support systems have the problem that users must contact a call center every time they have an inquiry, which takes time and effort. In particular, inquiries related to content distribution services require a quick response, which places a heavy burden on both users and the support desk. In addition, there are cases where users are kept waiting for a long time depending on the number of calls at the desk, which reduces user satisfaction.

[1308] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1309] In this invention, the server includes a means for a user to input an inquiry related to the content distribution service by voice or text, a means for generating an answer using a generative AI model, a means for a user terminal to display the generated answer, a means for obtaining feedback from the user and obtaining an appropriate congestion status of the support desk if the problem is not resolved, and a means for displaying the congestion status to the user and connecting to the support desk in real time if the user requests a connection to the support desk. This allows the user to quickly obtain the information they need and smoothly connect to the support desk even if the problem is not resolved.

[1310] "Inquiry Content" refers to the specific question or problem that a user enters for support.

[1311] "Voice or text input means" refers to a means by which a user inputs inquiry details into a terminal using voice recognition or text input.

[1312] A "generative AI model" is an artificial intelligence model that uses natural language processing to automatically generate appropriate answers to user inquiries.

[1313] "Means for generating an answer" refers to a means for creating an answer based on the user's inquiry using a generative AI model.

[1314] A "user terminal" is an electronic device used by a user, such as a smartphone, tablet, or computer.

[1315] The "means for displaying the generated answer" is a means for displaying the answer generated by the server on the user terminal in a form that the user can confirm.

[1316] A "means for obtaining feedback" is a means for users to gather ratings and additional opinions about the answers provided.

[1317] The "means for acquiring the appropriate congestion status of the counter" is a means for the server to acquire the current waiting time and congestion status of the call center or support counter in real time.

[1318] The "means for displaying the congestion status to the user" is a means for displaying the acquired information on the congestion status of the counter on the user terminal.

[1319] The "means for establishing a connection when a customer service connection is desired" refers to the means by which the server establishes a connection to an appropriate customer service point when a user desires to connect to a support center.

[1320] A "content distribution service" is a service that provides digital content such as video and audio to users via the Internet.

[1321] "Content distribution service-related inquiries" refer to questions or problems related to content distribution services, such as content playback, distribution schedules, and account management.

[1322] The "means for connecting to a support desk in real time" is a means for instantly establishing a connection with a support person when a user needs support.

[1323] The present invention is a system that provides solutions to inquiries related to content distribution services before users make them using an inquiry support app. The system uses a generative AI model to generate answers to user inquiries, aiming to reduce the burden on both users and support desks.

[1324] User query input

[1325] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, a user can input "When will the next episode be released?" or ask a similar question by voice.

[1326] Server-generated answers

[1327] The server passes the inquiry received from the user to the generative AI model, which generates an appropriate answer. At this time, the generative AI model is designed to learn from past inquiry data and related information to provide the optimal answer. The generative AI model uses natural language processing technology.

[1328] Displaying answers to users

[1329] The server sends the generated answer to the user's terminal, which displays it to the user. For example, if the server generates an answer such as "The next episode is scheduled to be released next Friday," the terminal displays this answer to the user. At this point, the user can decide whether the provided answer solved their problem.

[1330] If the issue persists, further support is available

[1331] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that "the problem was not solved," the server obtains more appropriate information about the congestion status of the counter. The server monitors the waiting time and congestion status of the counter in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[1332] Counter connection

[1333] If the user checks the congestion status and still wishes to connect to a hotline, the terminal sends a connection request to the server. The server receives this request and attempts to connect to the specified hotline. This ensures that the user is connected to the correct hotline at the optimal time and receives support to resolve the problem. This process reduces waiting times for users and allows them to receive support efficiently.

[1334] Specific examples

[1335] For example, if a user inquires about a problem such as "I can't play the currently streaming episode," the user enters this information into the app. The server uses a generative AI model to generate a response such as "If you can't play the episode, first check your internet connection. Then try restarting your device," and provides it to the user. If the response does not resolve the problem, the server obtains the congestion status of the streaming service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[1336] Prompt Sentence Examples

[1337] "How do I resolve episode playback errors?"

[1338] "I want to know when the next episode will be released."

[1339] In this way, the present invention allows users to quickly obtain the appropriate information and connect them to the correct point of contact as needed, thereby increasing the efficiency of the inquiry process and reducing the burden on support points.

[1340] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1341] Step 1: User enters inquiry

[1342] The user launches an inquiry support app on their smartphone or computer and inputs their inquiry by voice or text. The input data is sent to the server by the user's device. For example, if the user inputs "When will the next episode be released?", the text data is sent to the server.

[1343] Step 2: The server passes the query to the generative AI model

[1344] The server passes the inquiry received from the user to the generative AI model. The generative AI model receives the input text data and generates an appropriate answer based on past inquiry data and related information. For example, in response to the text data "When will the next episode be released?", the generative AI model generates an answer such as "The next episode is scheduled to be released next Friday."

[1345] Step 3: The server generates a response and sends it to the user's device.

[1346] The server sends the answer obtained from the generative AI model to the user's device, which receives the answer and displays it to the user. For example, the answer sent from the server, "The next episode is scheduled to be released next Friday," is displayed on the user's device screen.

[1347] Step 4: User reviews answers and provides feedback

[1348] The user checks the answer displayed on the terminal and determines whether the problem has been solved. If the problem has not been solved, the user inputs feedback such as "The problem has not been solved." This feedback data is again sent from the user terminal to the server.

[1349] Step 5: The server obtains the congestion status of the counter.

[1350] The server receives feedback from the user and initiates operations to obtain the appropriate congestion status of the counter. The server monitors the congestion status of the counter in real time and obtains information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[1351] Step 6: The server sends the congestion status to the user's device.

[1352] The server sends the obtained congestion status of the counter to the user's terminal. The user terminal receives this information and displays it to the user. For example, the information sent from the server, "The current congestion status of the counter is a waiting time of 10 minutes," is displayed on the user's terminal screen.

[1353] Step 7: If the user wants a direct connection

[1354] If the user checks the congestion status and still wishes to connect through the service window, the terminal sends a connection request to the server. This connection request data is sent from the user terminal to the server.

[1355] Step 8: The server attempts to connect to the gateway

[1356] The server receives a connection request from the user and attempts to connect to the specified support desk. Once the connection is established, the server sends a notification to the user's device that the connection is complete, allowing the user to contact the support desk. For example, the server may notify the user that "Connection has been established. An operator is waiting."

[1357] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1358] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[1359] User query input

[1360] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[1361] Emotion recognition by emotion engine

[1362] The device sends the voice or text inquiry received from the user to the emotion engine. The emotion engine recognizes the emotion from the user's voice or text and sends the result to the server. For example, if the user is angry, the emotion engine recognizes it as "anger."

[1363] Server-generated answers

[1364] The device sends the emotion recognition results along with the query content to the server. The server passes this query content to the generative AI model, which then generates an appropriate response. At this time, the generative AI model also takes the emotion recognition results into account. For example, if the user is angry, it will provide a polite response that takes their emotion into consideration.

[1365] Displaying answers to users

[1366] The server sends the generated answer to the user's terminal, and the terminal displays it to the user. For example, if the server generates an answer such as "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time," the terminal displays this answer to the user. The user checks the answer and determines whether the problem has been solved.

[1367] If the issue persists, further support is available

[1368] After the user confirms the answer, the terminal receives feedback from the user. If the user gives feedback that the problem was not solved, the server obtains the appropriate congestion status of the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes." This information is sent back to the user's terminal and displayed to the user.

[1369] Counter connection

[1370] If the user checks the congestion status and still wishes to connect to a service center, the terminal sends a connection request to the server. The server receives this request and attempts to connect the user to the specified service center. This allows the user to be connected to the correct service center at the optimal time and receive support to resolve the problem.

[1371] Specific examples

[1372] For example, if a user inquires about a problem such as "slow internet speed," the user inputs this information into the app. The emotion engine recognizes the emotion from the user's voice as "anger." The server uses a generative AI model to generate a response such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the response does not resolve the problem, the server obtains the congestion status of the internet service center and displays it to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[1373] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[1374] The processing flow will be explained below.

[1375] Step 1:

[1376] The user launches an inquiry support app installed on a device such as a smartphone or computer. When the app launches, a screen for entering the inquiry details is displayed.

[1377] Step 2:

[1378] The user inputs the inquiry by voice or text. For example, they might input something like "What is the credit card limit?" If they input the inquiry by voice, it is recorded as is, and if they input the inquiry by text, it is immediately entered into the text area.

[1379] Step 3:

[1380] The device sends the inquiry received from the user to the emotion engine, which analyzes the received voice or text data and recognizes the user's emotions, such as "anger," "sadness," and "joy."

[1381] Step 4:

[1382] The emotion engine sends information including the recognized user emotion to the server. The original query content is also transferred to the server together with the emotion recognition result.

[1383] Step 5:

[1384] The server passes the received inquiry and emotion recognition results to the generative AI model, which then generates the optimal answer based on past data and related information. For example, if the user expresses anger, the generative AI model will generate a polite answer that takes that emotion into consideration.

[1385] Step 6:

[1386] The server receives the answer generated by the generative AI model and sends it to the user's device. The answer may contain specific information, such as "The standard credit card limit is 500,000 yen. Please feel free to contact us with any questions."

[1387] Step 7:

[1388] The device displays the received answer to the user, who can then review the answer and determine whether it has resolved the problem. Specifically, the answer is displayed on the device's screen, and if it was entered via voice, it is played back aloud.

[1389] Step 8:

[1390] If the user checks the answer and the problem is not solved, the device receives feedback from the user. The feedback "The problem was not solved" is sent. This is done by the user providing feedback by pressing a button, etc.

[1391] Step 9:

[1392] The server receives feedback from users and obtains the appropriate congestion status for the counter. The server monitors the congestion status in real time and generates information such as "The current congestion status of the counter is a waiting time of 10 minutes."

[1393] Step 10:

[1394] The server sends the congestion status to the user's terminal. The terminal displays the received congestion status to the user and asks whether the user wishes to connect. Here, the congestion status is displayed visually in a format that is easy for the user to understand.

[1395] Step 11:

[1396] When a user requests a connection through the service window, the terminal sends a connection request to the server. The connection request includes information based on the user's inquiry and the congestion status.

[1397] Step 12:

[1398] The server receives the connection request and attempts to connect to the specified contact point. If the contact point is available, the user's device is connected to the contact point and support is available. The status is displayed to the user until the connection is complete.

[1399] Step 13:

[1400] After a user receives support at the customer service center, the terminal records the results and uses them as reference for future inquiries if necessary. This information is shared with the server and used to improve the efficiency of the entire system.

[1401] By specifying the specific processing steps as described above, it becomes clear how the system operates and what advantages it provides to both the user and the call center.

[1402] Example 2

[1403] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1404] Conventional inquiry support systems often found it difficult to respond appropriately, taking into account the user's emotions. Furthermore, when the response was insufficient and the problem was not resolved, it was difficult to quickly connect the user to the appropriate help desk, which led to a decrease in user satisfaction. Therefore, there was a need for a system that could automate appropriate responses that took the user's emotions into account and quickly connect the user to the appropriate help desk if the problem was not resolved.

[1405] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1406] In this invention, the server includes a means for transmitting the content of an inquiry entered by a user terminal to an emotion recognition engine and analyzing the user's emotions, a means for passing the recognition results from the emotion recognition engine and the content of the inquiry to a generative AI model and generating an answer that takes emotion recognition into consideration, and a means for transmitting the generated answer to the user terminal and for the user terminal to display the generated answer. This enables an appropriate answer that takes the user's emotions into consideration, thereby improving user satisfaction. In addition, by including a means for obtaining feedback from the user, obtaining an appropriate counter congestion status if the problem is not resolved, displaying the congestion status to the user, and connecting to a counter if the user requests it, a prompt and appropriate response can be made.

[1407] "User" means an individual or organization that makes an inquiry using the inquiry support app.

[1408] A "terminal" is an electronic device used by a user, such as a smartphone, computer, or tablet.

[1409] "Inquiry content" refers to questions or requests that a user inputs through voice or text.

[1410] "Speech or text" refers to textual or audio information that a user inputs into a terminal.

[1411] An "emotion recognition engine" is software or hardware that analyzes and recognizes a user's emotions from input voice or text.

[1412] The "server" is a computer system that processes the inquiry content and emotion recognition results, and generates and manages answers using a generative AI model.

[1413] A "generative AI model" is an artificial intelligence algorithm that generates appropriate answers based on input data.

[1414] A "prompt sentence" is an input sentence that prompts a generative AI model to generate an appropriate answer.

[1415] "Feedback" refers to the evaluation or opinion that a user submits after reviewing an answer.

[1416] The "contact point" refers to a department or service provider that handles inquiries from users.

[1417] "Congestion status" is information indicating the current waiting time at the counter and the status of resources available for processing.

[1418] A "connection" is the establishment of communication that allows a user to make a direct inquiry to a customer service center.

[1419] The present invention combines a system that uses an inquiry support app to provide solutions before a user makes an inquiry to a call center with an emotion recognition engine that recognizes the user's emotions. This enables a more appropriate response that matches the user's emotions, thereby improving user satisfaction.

[1420] Users can launch an inquiry support app installed on a device such as a smartphone or computer and input their inquiry by voice or text. For example, if they want to inquire about their credit card limit, they can input "What is the credit card limit?" or ask a similar question by voice.

[1421] The device sends the voice or text query received from the user to the emotion recognition engine. The emotion recognition engine uses speech recognition software (e.g., the SpeechRecognition library) and text analysis algorithms (e.g., the Python NLTK library) to recognize the user's emotion. For example, if the user types "What's my credit card limit?", the engine recognizes the user's emotion as "neutral."

[1422] After the emotion recognition engine obtains the recognition result, the device sends the query content and the emotion recognition result to the server. The server passes the query content and emotion recognition result to a generative AI model (e.g., OpenAI's GPT-3) and generates an appropriate response. At this time, the generative AI model takes the emotion recognition result into consideration. For example, if the user is angry, it will generate a kind and polite response that takes that emotion into account.

[1423] Example prompt sentence:

[1424] "The user has made the following query: 'What is the credit card limit?' The current sentiment is neutral. Please generate an appropriate answer."

[1425] The server sends the generated answer to the user's terminal, and the terminal displays this answer to the user. For example, if a response such as "The standard credit card limit is 500,000 yen. Please contact us anytime if you have any questions" is generated, this response will be displayed on the terminal. The user can check the response and determine whether the problem has been resolved.

[1426] If the user gives feedback that "the problem was not resolved," the device sends this feedback to the server. The server then obtains the appropriate counter congestion status using a real-time data collection API (for example, Firebase Realtime Database) and generates information such as "The current counter congestion status is a 10-minute wait time." This information is again sent to the user's device and displayed to the user. If the user wishes to connect to a counter, the device sends a connection request to the server, and the server attempts to connect to the specified counter. This allows the user to be connected to the correct counter at the optimal time and receive support to resolve the problem.

[1427] As a concrete example, let's consider the case of an inquiry about the problem of "slow internet speed." The user enters this information into the app, and the emotion recognition engine recognizes "anger" from the voice. The server uses a generative AI model to generate an answer such as "What to do if your internet speed is slow," and adds a message that takes the emotion into consideration, such as "We apologize for the inconvenience." If the answer does not solve the problem, the server obtains the congestion status of the internet service center and displays this to the user. If the user wishes to connect, the device sends a connection request to the server, and the device is connected to the appropriate center.

[1428] In this way, by recognizing the user's emotions, it becomes possible to respond more appropriately, thereby improving user satisfaction.

[1429] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1430] Step 1:

[1431] The user inputs the inquiry content by voice or text. The user starts the inquiry support app and inputs the inquiry content into a device such as a smartphone or computer using a text box or microphone. For example, the user inputs text or voice such as, "What is the credit card limit?"

[1432] Input: Enquiry (e.g. "What is the credit card limit?")

[1433] Output: Inquiry entered into the terminal

[1434] Step 2:

[1435] The device sends the query to the emotion recognition engine, which analyzes the user's emotions. The device sends the input voice or text to the emotion recognition engine, which uses voice recognition software and text analysis algorithms to analyze and recognize the user's emotions.

[1436] Input: User inquiry

[1437] Output: The emotion recognized by the emotion recognition engine (e.g. "Neutral")

[1438] Specific behavior: The terminal sends the text "What is the credit card limit?" to the emotion recognition engine, which recognizes it as "neutral."

[1439] Step 3:

[1440] The device sends the query and emotion recognition results to the server. The device generates an HTTP POST request to send the query and emotion recognition results to the server.

[1441] Input: User's inquiry and emotion recognition result (e.g., "neutral")

[1442] Output: Query content sent to the server and emotion recognition results

[1443] Specific operation: The device sends the query "What is the credit card limit?" and the emotion recognition result "Neutral" to the server as an HTTP POST request.

[1444] Step 4:

[1445] The server generates an answer using a generative AI model. The server inputs the query content and emotion recognition results into the generative AI model to obtain an appropriate answer. For example, OpenAI's GPT-3 is used as the generative AI model.

[1446] Input: Enquiry and emotion recognition result (e.g. "Neutral")

[1447] Output: The answer generated by the generative AI model

[1448] Specific operation: The server inputs the following prompt sentence into the generative AI model: "The user has made the following inquiry: 'What is the credit card limit?' The current emotion is neutral. Please generate an appropriate answer." The server receives the answer from the generative AI model: "The standard credit card limit is 500,000 yen."

[1449] Step 5:

[1450] The server transmits the generated response to the user terminal, and the server generates and transmits data for transmitting the generated response to the user terminal.

[1451] Input: The answer generated by the generative AI model

[1452] Output: The answer sent to the user's terminal

[1453] Specific operation: The server sends the HTTP response "The standard credit card limit is 500,000 yen" to the user's terminal.

[1454] Step 6:

[1455] The device displays the answer to the user. The device displays the answer it receives in the app so that the user can check it.

[1456] Input: Received Answer

[1457] Output: The answer displayed to the user

[1458] Specific operation: The device app displays to the user, "The standard credit card limit is 500,000 yen. If you have any questions, please contact us at any time."

[1459] Step 7:

[1460] The user sends feedback to the terminal, and the user gives feedback on whether the answer was helpful in solving the problem.

[1461] Input: User feedback

[1462] Output: Feedback sent to the terminal

[1463] Specific operation: The user types "The problem was not resolved" into the terminal and submits it.

[1464] Step 8:

[1465] The device sends the feedback to the server. The device generates an HTTP request to send the user's feedback to the server.

[1466] Input: User feedback

[1467] Output: Feedback sent to the server

[1468] Specific operation: The device sends the user's feedback "The problem was not resolved" to the server.

[1469] Step 9:

[1470] The server obtains the congestion status and sends it to the terminal. The server obtains the congestion status of the appropriate counter through the real-time data collection API and sends it to the terminal.

[1471] Input: Requests to the server and congestion status from the real-time data collection API

[1472] Output: Congestion status information sent to the user terminal

[1473] Specific operation: The server obtains the congestion status using the Firebase Realtime Database API and sends the message "The current congestion status at the counter is a waiting time of 10 minutes" to the user's terminal.

[1474] Step 10:

[1475] The user sends a request for connection to the terminal. The user checks the congestion status and sends a request for connection through the service window to the terminal.

[1476] Input: User connection request

[1477] Output: Connection request sent to the device

[1478] Specific operation: The user enters "I would like to connect through the counter" into the terminal and sends it.

[1479] Step 11:

[1480] The terminal sends a connection request to the server. The terminal generates an HTTP request to send the connection request to the server.

[1481] Input: A user's connection request

[1482] Output: Connection request sent to server

[1483] Specific operation: The terminal sends the user's connection request "I want to connect to the window" to the server.

[1484] Step 12:

[1485] The server connects to the specified gateway. The server receives the connection request and performs the operations to connect the user to the specified gateway.

[1486] Input: Connection request

[1487] Output: Connection established between user and contact

[1488] Specific operation: The server sends a connection request to the call center system and connects the user to the specified contact point.

[1489] The above is the specific flow of the program processing of the invented system and details of each processing step.

[1490] (Application example 2)

[1491] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1492] The present invention aims to improve user satisfaction by combining a system that provides solutions before the user makes an inquiry with an emotion engine that recognizes the user's emotions, enabling responses tailored to the user's emotions. However, current inquiry systems have the problem of difficulty in properly understanding the user's emotions and customizing responses. Furthermore, if a user's inquiry cannot be properly addressed, it is necessary to quickly connect them to an appropriate point of contact. An efficient and effective system is needed to solve this problem.

[1493] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1494] In this invention, the server includes means for the user to input the inquiry content by voice or text, means for the server to generate an answer using a generative AI model, means for the user terminal to display the generated answer, means for obtaining feedback from the user and obtaining an appropriate counter congestion status if the problem is not resolved, means for displaying the congestion status to the user and connecting to a counter if the user requests it, means for recognizing the user's emotion using an emotion engine and passing the recognition result to the generative AI model, and means for the server to generate an answer taking the emotion recognition result into consideration. This makes it possible to provide a system that enables more appropriate and customized responses according to the user's emotion and improves user satisfaction.

[1495] A "user" is an individual or legal entity that uses the system to make an inquiry.

[1496] The "content of inquiry" refers to the specific content of the question or problem that the user wants to solve.

[1497] "Voice or text" refers to the form of means by which a user inputs inquiry content, with voice meaning voice data and text meaning character information.

[1498] A "server" is a computer system that receives, processes, and stores data from user terminals over a network.

[1499] A "generative AI model" refers to an algorithm that uses artificial intelligence technology to generate appropriate answers to user inquiries.

[1500] A "user terminal" is a device that allows a user to access the system, input inquiries, and receive responses.

[1501] "Feedback" refers to the evaluations and comments that users make on the system's suggestions and answers.

[1502] The "contact point" refers to a department or person in charge who responds to user inquiries in person or by telephone or other means.

[1503] "Congestion status" refers to indicators such as the number of inquiries currently being handled at the counter and waiting times.

[1504] An "emotion engine" is an algorithm that recognizes a user's emotions from voice or text.

[1505] "Emotion recognition result" is data indicating the emotional state of the user analyzed by the emotion engine.

[1506] The present invention begins when a user uses an inquiry support app installed on a device such as a smartphone or computer to input their inquiry by voice or text. For example, the user may make an inquiry such as, "What is my credit card limit?" This input is sent to a server, and the emotion engine recognizes the user's emotion.

[1507] The server receives the user's inquiry and the emotion recognition results, inputs them into a generative AI model, and generates an appropriate answer. This generative AI model uses the prompt sentence to perform advanced natural language processing and provide a response that takes emotion into consideration. For example, if the user is angry, the model responds according to the user's emotion by adding expressions of gratitude or apology, such as "We apologize for the inconvenience."

[1508] The generated answer is sent to the user's terminal, where the user can review it. If the user is not satisfied with the answer, the terminal obtains feedback. If the feedback is "the problem was not resolved," the server further obtains the appropriate counter congestion status and provides the user with information such as "The current counter congestion status is a 10-minute wait time."

[1509] If a user checks the congestion status and wishes to connect to a hotline, the user's terminal sends a connection request to the server, and the server connects the user to the appropriate hotline. In this way, the user is connected to the appropriate hotline at the optimal time and can receive support to resolve their problem.

[1510] Specifically, by inputting the following prompt sentence into the generative AI model, an appropriate answer is generated:

[1511] "Please politely respond to angry users with the following question: My internet is slow. What should I do?"

[1512] This system uses the speech_recognition library for speech recognition and the transformers library for emotion recognition. It also uses the latest natural language processing technologies, such as GPT-3, for its generative AI model. It uses communication services such as the Twilio API for notification services. This enables highly efficient and appropriate customer support tailored to the user's emotional state. This system can significantly improve customer service in physical stores and increase user satisfaction.

[1513] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1514] Step 1:

[1515] The user inputs the inquiry by voice or text.

[1516] Input: User's voice or text query

[1517] How it works: A user uses an inquiry support app installed on a device such as a smartphone or computer and enters a specific question, such as "What is my credit card limit?"

[1518] Step 2:

[1519] The terminal sends the inquiry to the server.

[1520] Input: The audio or text data generated in step 1

[1521] Output: Query data arrives at the server

[1522] Operation: The terminal sends the user's inquiry to the server as voice data or text data.

[1523] Step 3:

[1524] The server converts the received voice data into text (in the case of voice input).

[1525] Input: Audio data

[1526] Output: Text data

[1527] How it works: The server uses the speech_recognition library to analyze the audio data and convert it to text data.

[1528] Step 4:

[1529] The server sends the received text data to the emotion engine to recognize the user's emotion.

[1530] Input: Text data of inquiry content

[1531] Output: Emotion recognition result data

[1532] How it works: The server uses the transformers library to parse the text data to determine the user's emotions, and then obtains the results. For example, it identifies emotions such as "anger," "joy," and "sadness."

[1533] Step 5:

[1534] The server inputs the query content and emotion recognition results into a generative AI model to generate an answer.

[1535] Input: Text data of inquiry content, emotion recognition result data

[1536] Output: Generated answer text

[1537] How it works: The server inputs a prompt into the generative AI model, which generates an answer based on the emotion recognition results. An example of a prompt is, "Please politely respond to the following question for an angry user: My internet speed is slow. What should I do?"

[1538] Step 6:

[1539] The server transmits the generated answer to the user terminal.

[1540] Input: Generated answer text

[1541] Output: The answer displayed on the user's terminal

[1542] Operation: The server sends the generated answer data to the user's terminal, which displays it.

[1543] Step 7:

[1544] The user reviews the answer and provides feedback on whether the problem was resolved.

[1545] Input: User feedback

[1546] Output: Feedback data

[1547] Action: The user reviews the answer they received and provides feedback via their device, such as "problem solved" or "not solved."

[1548] Step 8:

[1549] The server receives the feedback data and obtains the appropriate busy status of the counter if the problem is not resolved.

[1550] Input: Feedback data

[1551] Output: Counter congestion data

[1552] Operation: If the user's feedback is "not resolved", the server obtains the appropriate counter congestion status in real time.

[1553] Step 9:

[1554] The server transmits the congestion status of the counter to the user terminal, and when the user requests a counter connection, the connection is made.

[1555] Input: Counter congestion data, user connection requests

[1556] Output: Check connection to the counter

[1557] Operation: The server sends congestion status data to the user terminal, and if the user requests a gateway connection, processes the connection request and connects to the appropriate gateway.

[1558] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1559] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1560] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1561] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1562] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1563] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1564] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1565] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1566] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1567] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1568] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1569] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1570] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1571] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1572] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1573] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1574] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1575] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1576] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1577] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1578] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1579] The following is further disclosed regarding the above embodiment.

[1580] (Claim 1)

[1581] A means for a user to input a query by voice or text;

[1582] A means for the server to generate an answer using a generative AI model;

[1583] a means for the user terminal to display the generated answer;

[1584] A means for obtaining feedback from users and obtaining an appropriate congestion status of the counter if the problem is not resolved;

[1585] means for displaying the congestion status to a user and for connecting via a service window if the user desires;

[1586] A system including:

[1587] (Claim 2)

[1588] 2. The system according to claim 1, further comprising means for a user to input an inquiry in the form of an image or video.

[1589] (Claim 3)

[1590] 10. The system of claim 1, wherein the server includes means for automatically learning and optimizing answers with the generative AI model.

[1591] "Example 1"

[1592] (Claim 1)

[1593] A means for a user to input a query by voice or text;

[1594] A means for the terminal to transmit the contents of an inquiry from a user to a server;

[1595] A means for the server to generate an answer to the inquiry using the generative AI model;

[1596] means for transmitting the server-generated response to the terminal;

[1597] means for the terminal to display the generated answer;

[1598] A means for obtaining feedback from users and obtaining information on the busyness of the counter if the problem is not resolved;

[1599] means for transmitting the congestion status to a terminal and for the terminal to display the congestion status to a user;

[1600] a means for transmitting a connection request from the terminal to the server when the user desires a window connection, and for the server to attempt a connection to the designated window;

[1601] A system including:

[1602] (Claim 2)

[1603] 2. The system according to claim 1, further comprising means for a user to input an inquiry in the form of an image or video.

[1604] (Claim 3)

[1605] 2. The system of claim 1, wherein the server includes means for automatically learning and optimizing responses to queries using the generative AI model.

[1606] "Application Example 1"

[1607] (Claim 1)

[1608] A means for a user to input a query by voice or text;

[1609] A means for the server to generate an answer using a generative AI model;

[1610] a means for the user terminal to display the generated answer;

[1611] A means for obtaining feedback from users and obtaining an appropriate congestion status of the counter if the problem is not resolved;

[1612] means for displaying the congestion status to a user and for connecting via a service window if the user desires;

[1613] a means for a user to input an inquiry related to the content distribution service by voice or text;

[1614] A means to connect to the support desk of the content distribution service in real time,

[1615] A system including:

[1616] (Claim 2)

[1617] 2. The system according to claim 1, further comprising means for a user to input an inquiry in the form of an image or video.

[1618] (Claim 3)

[1619] 10. The system of claim 1, wherein the server includes means for automatically learning and optimizing answers with the generative AI model.

[1620] "Example 2: Combining Emotion Engines"

[1621] (Claim 1)

[1622] A means for a user to input a query by voice or text;

[1623] A means for transmitting the content of the inquiry input by the user terminal to an emotion recognition engine and analyzing the emotion of the user;

[1624] The server passes the recognition results from the emotion recognition engine and the query content to the generation AI model, and generates an answer that takes emotion recognition into consideration.

[1625] means for transmitting the generated answer from the server to the user terminal, and for the user terminal to display the generated answer;

[1626] A means for obtaining feedback from users and obtaining an appropriate congestion status of the counter if the problem is not resolved;

[1627] means for displaying the congestion status to a user and for connecting via a service window if the user desires;

[1628] A system including:

[1629] (Claim 2)

[1630] 2. The system according to claim 1, further comprising means for a user to input an inquiry in the form of an image or video.

[1631] (Claim 3)

[1632] 10. The system of claim 1, wherein the server includes means for automatically learning and optimizing answers with the generative AI model.

[1633] "Application example 2 when combining emotion engines"

[1634] (Claim 1)

[1635] A means for a user to input a query by voice or text;

[1636] A means for the server to generate an answer using a generative AI model;

[1637] a means for the user terminal to display the generated answer;

[1638] A means for obtaining feedback from users and obtaining an appropriate congestion status of the counter if the problem is not resolved;

[1639] means for displaying the congestion status to a user and for connecting via a service window if the user desires;

[1640] A means for recognizing the user's emotions using an emotion engine and passing the recognition results to a generative AI model;

[1641] A means for the server to generate an answer taking into account the emotion recognition result;

[1642] A system including:

[1643] (Claim 2)

[1644] 2. The system according to claim 1, further comprising means for a user to input an inquiry in the form of an image or video.

[1645] (Claim 3)

[1646] 10. The system of claim 1, wherein the server includes means for automatically learning and optimizing answers with the generative AI model. [Explanation of symbols]

[1647] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for a user to input a query by voice or text; A means for the server to generate an answer using a generative AI model; a means for the user terminal to display the generated answer; A means for obtaining feedback from users and obtaining an appropriate congestion status of the counter if the problem is not resolved; means for displaying the congestion status to a user and for connecting via a service window if the user desires; A system including:

2. 2. The system according to claim 1, further comprising means for a user to input an inquiry in the form of an image or video.

3. 10. The system of claim 1, wherein the server includes means for automatically learning and optimizing answers using a generative AI model.

Images