system

Abstract

We provide the system. [Solution] A means of receiving information entered by the user, Means for analyzing the input information, A means for searching for relevant information based on the aforementioned analysis results, Means for providing the retrieved information to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] will be described accordingly.

[0005] In modern online support systems, it may be difficult for users to obtain quick and appropriate information. Also, due to the limited multilingual support and video-based support, satisfactory services cannot be provided especially to foreign users and users who require technical support. Therefore, a system for efficiently solving user problems and improving customer satisfaction is required.

Means for Solving the Problems

[0006] The present invention solves the above problems by using the following means: a system including means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, and means for providing the searched information to the user. Furthermore, the system includes means for making the input information multilingual, means for generating a response in multiple languages ​​based on the analysis results, and means for providing information to the user, including a multilingual response. Furthermore, the system includes means for including a video link in the information provided to the user, and means for the user to view support information based on the video link. As a result, users can receive real-time, fast, and appropriate support, and companies can reduce operational costs while improving customer satisfaction.

[0007] A "user" is a person or group that operates the system and provides input information.

[0008] "Inputted information" refers to data including questions, problems, or requests that the user provides to the system.

[0009] "Means of receiving" refers to the function within the system that receives information provided by the user and holds it for processing.

[0010] "Means of analysis" refers to the function of extracting and understanding meaning from input information using natural language processing and document analysis techniques.

[0011] "Searching methods" refer to the function of finding relevant information from databases or external sources based on the analysis results.

[0012] "Means of providing" refers to functions for formalizing searched information and displaying or communicating it to the user.

[0013] "Means of supporting multiple languages" refers to the ability to process and provide input information and system responses in multiple languages.

[0014] The "generating means" refers to a function for newly creating a response based on the analysis result and presenting it in a form that can be presented to the user.

[0015] The "video link" refers to information including a URL or hyperlink to a video where the user can view the support information.

[0016] The "viewable means" refers to a function for the user to confirm specific support information in video using the provided video link.

Brief Description of the Drawings

[0017] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0018] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0019] First, the language used in the following description will be explained.

[0020] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), and the like.

[0021] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0022] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0023] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0025] [First Embodiment]

[0026] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0027] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0030] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0033] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0037] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0038] The system of the present invention includes an interface in which users can submit support requests through a website or application, a server for receiving and analyzing requests, and a database for providing relevant information based on the analysis results.

[0039] Explanation of program processing

[0040] 1. Interface

[0041] Users type, for example, "I don't know how to cancel my order," into a chat box on a website or application. This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[0042] 2. Server-side processing

[0043] The server receives information sent by the user. The received information is first analyzed using a natural language processing (NLP) engine. Through this analysis, keywords such as "cancel" and "order" are extracted. Based on this, the server searches the database for relevant information to address specific scenarios and FAQs.

[0044] 3. Database Search

[0045] The server searches for relevant FAQs and support documents based on the analysis results. For example, guides and procedures related to "order cancellation" are retrieved from the database. The retrieved information is then formatted as an appropriate response.

[0046] 4. Generating and providing responses

[0047] The server generates a response for the user based on the information it has obtained. The response may include video links or detailed instructions as needed. For example, it may include specific instructions and links such as, "To cancel your order, please follow these steps. For details, please refer to this video link."

[0048] 5. Sending and displaying responses

[0049] The server sends the generated response to the user's terminal. The terminal decodes the received response and displays it to the user in the chat box. This allows the user to obtain relevant information in real time.

[0050] As a concrete example, consider the following scenario.

[0051] Example 1: For Japanese users

[0052] 1. User: "I'm having trouble with my internet connection. What should I do?"

[0053] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[0054] 3. As a solution to the "Internet connection problem," generate a response that includes a video link and instructions.

[0055] 4. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link (example.com / internetissue)."

[0056] 5. The terminal displays a response to the user.

[0057] Example 2: For English users

[0058] 1. User: "How can I reset my password?"

[0059] 2. The server analyzes the keyword "reset password" and searches the database for relevant support information.

[0060] 3. As a solution for "password reset," generate a response that includes instructions.

[0061] 4. Response: "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0062] 5. The terminal displays a response to the user.

[0063] The system of this invention allows users to receive appropriate and detailed support in real time, and enables companies to reduce operational costs while improving customer satisfaction.

[0064] The following describes the processing flow.

[0065] Step 1:

[0066] Users enter questions or problems into a chat box on a website or application. For example, they might type, "How do I cancel my order?"

[0067] Step 2:

[0068] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[0069] Step 3:

[0070] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[0071] Step 4:

[0072] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" might be extracted.

[0073] Step 5:

[0074] The server uses the extracted keywords to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[0075] Step 6:

[0076] The server organizes the search results and generates a message to respond to the user. At this stage, if multilingual support is required, the message is translated into the appropriate language. For example, a response such as "To cancel your order, please select 'Cancel' from 'Order History' on My Page" is generated.

[0077] Step 7:

[0078] The server sends the generated response to the terminal. The generated response is encoded and sent as an HTTP response.

[0079] Step 8:

[0080] The device decodes the HTTP response received from the server and displays a response message to the user. The answer is displayed in the chat box, allowing the user to get solutions to their questions and problems in real time. For example, it might display, "To cancel your order, please select 'Cancel' from 'Order History' on My Page."

[0081] This series of steps allows users to receive quick and appropriate support, and enables companies to provide efficient customer support.

[0082] (Example 1)

[0083] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0084] Traditional support systems faced challenges in responding quickly and accurately to user inquiries. In particular, efficient answers were often lacking when multilingual support or detailed instructions were required. Furthermore, the lack of adequate systems for providing users with the information they needed in an appropriate format resulted in a poor user experience.

[0085] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0086] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, means for automatically formatting the input information, means for transmitting the formatted information to the server, means for extracting keywords using a natural language processing engine based on the analysis results, means for generating a response based on the acquired information, means for transmitting the generated response to the user's terminal, and means for displaying the transmitted response to the user. This enables the user to receive detailed multilingual support information in real time.

[0087] A "user" is an individual or organization that enters a support request through the system and obtains information.

[0088] "Means of receiving information" refers to the interface and protocol that allows a system to receive information entered by a user.

[0089] "Means of analyzing information" refers to natural language processing engines and other analytical tools used to process received information and understand its content.

[0090] "Means of searching for relevant information" refers to the process of finding the necessary information from appropriate databases or resources based on the results of the analysis.

[0091] "Means of providing information" refers to technologies for formatting retrieved information and displaying or transmitting it appropriately to the user.

[0092] "Means for automatically formatting information" refer to protocols and algorithms that convert user-inputted information into a format that the system can easily understand.

[0093] "Means for sending formatted information to a server" refers to communication means for transferring appropriately formatted data to a server.

[0094] A "natural language processing engine" is a program that analyzes human language, understands its meaning, and extracts keywords.

[0095] "Means for generating responses" refer to the logic and software used to create appropriate answers to user inquiries based on extracted keywords and search results.

[0096] "Means for sending a response to the user's terminal" refers to the communication protocol and system for sending the generated response to the user's device.

[0097] "Means for displaying responses" refers to interface technology that allows users to visually confirm the responses they have received on their devices.

[0098] This invention relates to a system in which a user submits a support request through a website or application, the system analyzes the request, and provides an appropriate response. The main components of this system include an interface for receiving user input, a server for analyzing the input information, a database for retrieving relevant information based on the analysis results, and means for providing a response to the user.

[0099] 1. Interface

[0100] Users use the chat box on the website or application to enter inquiries, such as "I don't know how to cancel my order." This interface has a feature that automatically detects the user's language settings and converts the entered information into the appropriate format.

[0101] 2. Sending data

[0102] The terminal sends information entered by the user to the server. This process uses protocols such as HTTP requests to transmit data securely and reliably. Data integrity and security are ensured during this process.

[0103] 3. Analysis of Information

[0104] The server analyzes the received information using a natural language processing (NLP) engine. The NLP engine tokenizes the text, performs part-of-speech analysis, and extracts important keywords. For example, keywords such as "cancel" and "order" are extracted.

[0105] 4. Searching for related information

[0106] The server searches the database for relevant information based on the extracted keywords. The database contains FAQs, guides, and instruction manuals. The search results are retrieved as structured data.

[0107] 5. Response generation

[0108] The server generates a response for the user based on the information it has obtained. This response may include detailed instructions or video links. For example, it might say, "To cancel your order, please follow these steps. For more details, please refer to this video link."

[0109] 6. Sending a response

[0110] The server sends the generated response to the terminal. This process also uses protocols such as HTTP responses to ensure secure and reliable data transmission.

[0111] 7. Display of response

[0112] The terminal decodes the received response and displays it to the user in a chat box. The display is formatted for user readability, and interactive elements such as hyperlinks are provided as needed.

[0113] Specific example

[0114] For example, consider a case where a Japanese-speaking user types, "I'm having trouble with my internet connection. What should I do?" This information is sent to the server and securely analyzed. The server extracts the keyword "internet connection" and searches its database for relevant support information. As a result, a response is generated that includes a video link and instructions, and the user is provided with a response such as, "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link."

[0115] For English-speaking users, typing "How can I reset my password?" will trigger a similar analysis process, extracting the keyword "reset password." The server will then search for relevant password reset information and generate a response containing instructions. The user will receive a response such as, "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword."

[0116] Examples of prompts for generative AI models

[0117] "I'm having trouble with my internet connection. What should I do?"

[0118] "I need help with resetting my password."

[0119] This system allows users to receive detailed, multilingual support information in real time, while enabling businesses to reduce operational costs and improve customer satisfaction.

[0120] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0121] Step 1: The user enters their question into the chat box of the website or application.

[0122] Input: The user enters the question in text format (e.g., "I don't know how to cancel my order").

[0123] Operation: The interface receives user input and automatically detects language settings.

[0124] Output: Formatted text information.

[0125] Step 2: The terminal sends user input to the server.

[0126] Input: Formatted text information generated by the interface.

[0127] Operation: The terminal uses an HTTP request to send formatted text information to the server.

[0128] Output: Text information received by the server.

[0129] Step 3: The server analyzes the received information using a natural language processing (NLP) engine.

[0130] Input: Text information received by the server.

[0131] Operation: The NLP engine performs the following processes:

[0132] Tokenization: Dividing text into words or phrases.

[0133] Part-of-speech analysis: Identifies the part of speech of each token.

[0134] Keyword extraction: Extraction of important keywords such as "cancel" and "order".

[0135] Output: Extracted keywords and analyzed text information.

[0136] Step 4: The server searches the database for relevant information based on the extracted keywords.

[0137] Input: Extracted keywords.

[0138] Operation: The process is carried out using the following steps:

[0139] Query generation: Generates search queries based on the extracted keywords.

[0140] Database query: Use the generated query to search the database for relevant FAQs and support documents.

[0141] Retrieving search results: Relevant information is retrieved from multiple databases.

[0142] Output: Relevant information retrieved from the database.

[0143] Step 5: The server generates a response based on the information it has obtained.

[0144] Input: Related information retrieved from the database.

[0145] Operation: Generates an answer that includes the following items:

[0146] Text generation: Creates appropriate response sentences based on user questions.

[0147] Additional information attached: Add video links and detailed instructions as needed.

[0148] Response format: Converts the generated response into a format that can be appropriately displayed to the user.

[0149] Output: Formatted answers and additional information.

[0150] Step 6: The server sends the generated response to the user's terminal.

[0151] Input: Formatted response.

[0152] Operation: The server uses the HTTP response to send the generated response to the user's terminal.

[0153] Output: Response data received by the terminal.

[0154] Step 7: The terminal displays the received response to the user.

[0155] Input: Response data received by the terminal.

[0156] Operation: The device decodes the received data and displays it in the chat box in a user-friendly format. Interactive elements such as hyperlinks and video links are also displayed as needed.

[0157] Output: The response displayed on the user's screen.

[0158] This system allows users to receive detailed, multilingual support information in real time. Businesses can reduce operational costs while improving customer satisfaction.

[0159] (Application Example 1)

[0160] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0161] Currently, many e-commerce sites dedicate significant resources to user support, and there is a demand for fast and efficient support. However, traditional systems have limitations in improving customer satisfaction and reducing operational costs due to language differences and the need for manual responses. In particular, there is a challenge in providing detailed information in real time when responding to requests such as order cancellations and product returns.

[0162] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0163] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, and means for utilizing a natural language processing engine to generate a response to the user. This enables the user to receive support through a multilingual chat box and to be provided with comprehensive and personalized information in real time.

[0164] "Means of receiving information entered by users" refers to methods of collecting questions and requests entered by users through websites and applications.

[0165] "Means for analyzing the input information" refers to means of analyzing the collected user questions and requests using a natural language processing engine to identify keywords and intents.

[0166] "Means for searching for relevant information based on the analysis results" refers to means for searching a database for relevant FAQs and support documents based on keywords and intents obtained through the analysis.

[0167] "Means of providing the searched information to the user" refers to means of formatting the support information obtained through the search and displaying it to the user through a chat box or other interface.

[0168] "Means for using a natural language processing engine to generate a response to the user" refers to means for analyzing input from the user and automatically generating a response in a natural conversational format based on the analysis results.

[0169] "Means of supporting multiple languages" refers to means that can translate and convert user input information, analysis results, and responses into multiple languages.

[0170] "Methods for providing information in real time" refer to means of providing information quickly by immediately performing analysis, information retrieval, and response generation in response to user requests.

[0171] "Means including video links" refers to means that include a URL link to a video related to the acquired support information, enabling the user to view specific instructions.

[0172] "A means of generating individual responses based on user requests using a generative AI model" refers to a method of generating personalized responses to different user requests by utilizing artificial intelligence.

[0173] This invention is a smartphone application system for e-commerce sites that analyzes user support requests in real time and provides appropriate information. This system consists of the following main elements:

[0174] 1. User Interface

[0175] Users can enter questions or requests into a chat box within the application. For example, they might type, "I don't know how to cancel my order." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[0176] 2. Server-side processing

[0177] The server first receives information sent by the user. The received information is parsed using a natural language processing engine (e.g., SpaCy, Google® Natural Language API). Through this parsing, keywords such as "cancel" and "order" are extracted. Based on this, the server searches for relevant information in a database (e.g., MySQL®, PostgreSQL).

[0178] 3. Database Search

[0179] Based on the analysis results, the server searches the database for relevant FAQs and support documents. For example, guides and procedures related to "order cancellation" are retrieved from the database.

[0180] 4. Generating a response

[0181] The server generates specific responses for the user based on the information it has obtained. These responses may include video links or detailed instructions as needed. For example, a response might be generated stating, "The procedure for canceling your order is as follows. Please refer to this link for details: example.com / cancelorder." This process also utilizes a generative AI model to generate personalized responses.

[0182] 5. Sending and displaying responses

[0183] The server sends the generated response to the user's device. The user's smartphone application decodes the received response and displays it in the chat box. This allows the user to obtain relevant information in real time.

[0184] Hardware and software configuration

[0185] hardware

[0186] Server (requires high-performance computing power)

[0187] User's smartphone

[0188] software

[0189] Natural language processing engines (e.g., SpaCy, Google Natural Language API)

[0190] Databases (e.g., MySQL, PostgreSQL)

[0191] Web application frameworks (e.g., Flask, Django)

[0192] Generative AI models (e.g., ChatGPT®)

[0193] Specific example

[0194] Example 1: For Japanese users

[0195] 1. User: "I'm having trouble with my internet connection. What should I do?"

[0196] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[0197] 3. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link: example.com / internetissue"

[0198] 4. The terminal displays a response to the user.

[0199] Example of a prompt

[0200] "I don't know how to return an item. What should I do?"

[0201] In this way, users can receive support through a multilingual chat box, and comprehensive and personalized information can be provided in real time. This system allows e-commerce sites to improve operational efficiency and enhance customer satisfaction.

[0202] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0203] Step 1: The user enters a question in the chat box.

[0204] Users enter questions or requests into a chat box within the e-commerce site's smartphone application. For example, they might type, "I don't know how to cancel my order."

[0205] Input: User's question or request

[0206] Output: Data sent to the server containing user input.

[0207] Step 2: The server receives user input.

[0208] The server receives information sent by the user. The received information is then analyzed using a natural language processing engine.

[0209] Input: User input

[0210] Output: Data ready for analysis

[0211] Step 3: Analyze the text using a natural language processing engine.

[0212] The server analyzes the received information using a natural language processing engine. During this process, keywords such as "cancel" and "order" are extracted.

[0213] Input: Received user input

[0214] Output: Extracted keyword and intent data

[0215] Step 4: Search for relevant information in the database

[0216] Based on the analysis results, the server searches the database for relevant FAQs and support documents. Specifically, it retrieves information related to "order cancellation."

[0217] Input: Extracted keywords and intent

[0218] Output: Relevant support information (FAQs and procedure manuals)

[0219] Step 5: Generate a response

[0220] The server generates specific responses for the user based on the information it has acquired. These responses may include video links or detailed instructions as needed.

[0221] Input: Related support information

[0222] Output: Final response data to be provided to the user

[0223] Step 6: Send the response to the user's device.

[0224] The server sends the generated response to the user's smartphone. The user's device decodes the received response and displays it in the chat box.

[0225] Input: Final response data

[0226] Output: Decoded data for display

[0227] Step 7: The user views the response.

[0228] Users can view the responses displayed in the chat box and find the necessary support information and procedures. This allows users to resolve issues in real time.

[0229] Input: Decoded data

[0230] Output: User understanding and behavior

[0231] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0232] The system of the present invention includes an interface in which users enter support requests through a website or application, a server for receiving and analyzing requests, a database for providing relevant information based on the analysis results, and an emotion engine for recognizing the user's emotions.

[0233] Explanation of program processing

[0234] 1. Interface

[0235] Users type messages into a chat box on a website or application, such as, "My order has been cancelled; please help me with this." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[0236] 2. Server-side processing

[0237] The server receives information sent by the user and first analyzes it using a natural language processing (NLP) engine. The NLP engine tokenizes the text and extracts important keywords and phrases such as "cancel" and "order." At the same time, an emotion engine detects emotions from the user's input and assigns emotion labels such as "anger" or "confusion."

[0238] 3. Database Search

[0239] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. For example, it retrieves guides and FAQs related to "order cancellation," and additional support information corresponding to the sentiment as needed.

[0240] 4. Generating and providing responses

[0241] The server generates a response to the user based on the information it has gathered. The tone and content of the response are adjusted based on the emotions recognized by the emotion engine. For example, a response might be generated such as, "We apologize for the inconvenience. Please refer to the following steps for instructions on how to cancel your order." Detailed instructions and video links may also be included.

[0242] 5. Sending and displaying responses

[0243] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[0244] As a concrete example, consider the following scenario.

[0245] Example 1: For Japanese users

[0246] 1. User: "My delivery is delayed and I'm having trouble with it. Is there anything you can do?"

[0247] 2. The server uses an NLP engine to extract keywords such as "delivery delay" and an emotion engine to detect the emotion of "confusion."

[0248] 3. Generate a response that includes a guide regarding "delivery delays" and an apology based on "confusion."

[0249] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[0250] 5. The terminal displays a response to the user.

[0251] Example 2: For English users

[0252] 1. User: "I can't log into my account and I'm getting frustrated."

[0253] 2. The server uses an NLP engine to extract the keyword "cannot log in" and an emotion engine to detect the emotion "frustration".

[0254] 3. Generate a response that includes a guide regarding "login issues" and encouragement based on "frustration."

[0255] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0256] 5. The terminal displays a response to the user.

[0257] The system of this invention allows users to receive appropriate and detailed support in real time, enabling companies to reduce operational costs while improving customer satisfaction. By utilizing an emotion engine, responses can be tailored to the user's emotions, resulting in more personalized support.

[0258] The following describes the processing flow.

[0259] Step 1:

[0260] Users enter questions or problems into the chat box of a website or application. For example, they might type, "My order has been cancelled, please help me with this."

[0261] Step 2:

[0262] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[0263] Step 3:

[0264] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[0265] Step 4:

[0266] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" are extracted. At the same time, an emotion engine detects the user's emotions from the text and assigns emotion labels such as "anger" or "confusion."

[0267] Step 5:

[0268] The server uses the extracted keywords and sentiment labels to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[0269] Step 6:

[0270] The server organizes search results and generates a message to respond to the user. Based on the emotions recognized by the sentiment engine, the tone and content of the response are adjusted. For example, a message such as, "We apologize for the inconvenience. Please refer to the following steps for how to cancel your order," might be generated. It may also include video links or detailed instructions.

[0271] Step 7:

[0272] The server sends the generated response to the terminal. The generated response is encoded as an HTTP response and sent.

[0273] Step 8:

[0274] The terminal decodes the HTTP response received from the server and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[0275] Specific example

[0276] Specific Example 1: For Japanese Users

[0277] 1. User: "I'm having trouble because the delivery is late. Can you do something about it?"

[0278] 2. The server extracts keywords such as "delivery delay" using the NLP engine and detects the emotion of "confusion" using the emotion engine.

[0279] 3. Generate a response that includes a guide regarding "delivery delay" and an apology based on "confusion".

[0280] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[0281] 5. The terminal displays the response to the user.

[0282] Specific Example 2: For English Users

[0283] 1. User: "I can't log into my account and I'm getting frustrated."

[0284] 2. The server extracts the keyword "cannot log in" using the NLP engine and detects the emotion of "frustration" using the emotion engine.

[0285] 3. Generate a response that includes a guide regarding "login problem" and encouragement based on "frustration".

[0286] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0287] 5. The terminal displays the response to the user.

[0288] The above is a specific embodiment of a system that combines an emotion engine for recognizing the user's emotions. With this system, the user can receive appropriate and detailed support in real time, and the enterprise can improve customer satisfaction while reducing operation costs. By using the emotion engine, responses according to the user's emotions become possible, and more personalized support can be realized.

[0289] (Example 2)

[0290] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as a "server", and the smart device 14 is referred to as a "terminal".

[0291] In the conventional support system, it is difficult to appropriately respond to the problems and emotions of the user, and an improvement in the user experience has been demanded. Also, the multilingual support is insufficient, and there is a high possibility that the response to users in different languages will be delayed. As a result, there is a problem that the operation cost of the enterprise increases and the customer satisfaction decreases. Furthermore, there is an issue that it is difficult for the user to obtain the necessary information quickly, and it is difficult to provide appropriate support.

[0292] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following respective means.

[0293] In this invention, the server includes means for analyzing information input by a user using language analysis means, means for detecting emotions and assigning emotion labels based on the analysis, and means for generating responses with tone and content appropriate to the emotions. This enables rapid and appropriate analysis of emotions based on user input and personalized responses based on the results. Furthermore, it enables the provision of information in multiple languages ​​and allows for rapid response to users who speak different languages, thereby reducing operational costs for companies and improving customer satisfaction.

[0294] "User-generated information" refers to text and data that users enter into a system to provide information such as support requests through a website or application.

[0295] "Language analysis means" refers to technologies and devices that analyze information input by users, understand its meaning, and extract key keywords. Specifically, natural language processing technology is included in this category.

[0296] "Means for detecting emotions and assigning emotion labels" refers to technologies and devices that analyze emotions from user input information and assign corresponding emotion tags. For example, this includes processes for classifying and tagging emotions such as anger, confusion, and joy.

[0297] "Means for searching for relevant information" refers to technologies and devices that search for and retrieve relevant information from databases and other information repositories based on the analyzed user input information.

[0298] "Means of generating responses" refers to technologies and devices that automatically generate appropriate responses and guidelines for users based on the searched relevant information and sentiment labels.

[0299] "Means of making information multilingual" refers to technologies and devices that translate user input information and generated responses into different languages, making them usable in a multilingual environment.

[0300] "Means including viewing links" refers to technologies and devices that include links to videos or tutorials in the information provided to users. This makes it easier for users to receive support in a visual way.

[0301] The system of this invention provides rapid and personalized support by analyzing support requests from users and generating appropriate responses. This system primarily consists of four parts: an interface, a server, a database, and an emotion engine. The following describes each component and its operation in detail.

[0302] interface

[0303] Users enter support requests into chat boxes on websites or applications. For example, a request might say, "My order has been cancelled; please help me with this." This interface receives the user's input, automatically detects their language settings, and sends the information to the server in the appropriate format.

[0304] server

[0305] The server receives information transmitted through the interface. This received information is first analyzed by a natural language processing (NLP) engine. This NLP engine uses services such as Google Cloud Natural Language API and Microsoft® Azure® Text Analytics to tokenize the text and extract important keywords and phrases. Next, an emotion engine is used to detect emotions from the user's input text and assign emotion labels such as "anger" or "confusion." IBM Watson® Tone Analyzer is used for this emotion analysis.

[0306] database

[0307] The server searches for relevant support information from the database based on the analysis results and sentiment labels. The database is managed by a database management system such as MySQL or MongoDB. For example, guides and FAQs related to "order cancellation" are obtained, and additional support information corresponding to the sentiment is retrieved as needed.

[0308] Generation and provision of responses

[0309] Based on the information obtained, the server generates a response to the user. The tone and content of the response are adjusted based on the sentiment recognized by the sentiment engine. For example, for a user's confusion, a response such as "We apologize for the inconvenience. For the method of canceling an order, please refer to the following procedure." is generated. Also, detailed procedures and video links may be included. This response can also be generated in multiple languages based on the user's language settings.

[0310] Transmission and display of responses

[0311] The generated response is sent from the server to the terminal. The terminal decodes the received response and displays it in an appropriate format in the user's chat box. As a result, the user can obtain appropriate information and support in real time.

[0312] Specific examples

[0313] Specific example 1: For Japanese users

[0314] User input: "I'm troubled because the delivery is delayed. Can you do something about it?"

[0315] Analysis by the NLP engine: Extract the keyword "delivery delay"

[0316] Analysis by the sentiment engine: Detect the sentiment of "confusion"

[0317] Database search: Obtain guides related to "delivery delay" and apology texts based on "confusion"

[0318] Generated response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[0319] Display to the user: The device displays the response in the chat box.

[0320] Example 2: For English users

[0321] User input: "I can't log into my account and I'm getting frustrated."

[0322] NLP engine analysis: Extracting the keyword "cannot log in".

[0323] Emotional Engine Analysis: Detecting the emotion of "frustration"

[0324] Database Search: Get guides and encouragement based on "login issues" and "frustration."

[0325] Generated response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0326] Display to the user: The device displays the response in the chat box.

[0327] This invention's system allows users to receive fast, personalized support in real time. In particular, its multilingual support and emotion recognition enable it to appropriately respond to users with different languages ​​and emotions. This allows companies to reduce operational costs and improve customer satisfaction.

[0328] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0329] Step 1:

[0330] Users enter support requests into a chat box on the website or application. For example, they might type, "My order was cancelled, please help me." This user input is the starting point for the system.

[0331] Input: Text of the support request entered by the user

[0332] Output: User input text is passed to the interface.

[0333] Step 2:

[0334] The interface receives user input text and automatically detects its language settings. This detection uses information such as web browser settings and account information. It then converts the input text to the appropriate format and sends it to the server.

[0335] Input: User input text

[0336] Data processing: Language setting detection and text formatting conversion

[0337] Output: Formatted text is sent to the server.

[0338] Step 3:

[0339] The server receives text data transmitted through the interface. The received text is sent to a natural language processing (NLP) engine for text analysis. The NLP engine tokenizes the text and extracts key keywords and phrases.

[0340] Input: Formatted text

[0341] Data processing: Text analysis and tokenization using a natural language processing engine.

[0342] Output: List of extracted keywords and phrases

[0343] Step 4:

[0344] The server sends the extracted keywords and phrases to the sentiment engine. The sentiment engine detects emotions from the text and assigns emotion labels such as "anger" or "confusion."

[0345] Input: List of extracted keywords or phrases

[0346] Data processing: Emotion analysis and emotion labeling using an emotion engine.

[0347] Output: Keywords and phrases with emotion labels

[0348] Step 5:

[0349] The server searches the database for relevant support information based on the analysis results and sentiment labels. The server constructs a search query and retrieves the relevant information using a database management system (e.g., MySQL or MongoDB).

[0350] Input: Keywords or phrases with emotion labels

[0351] Data Processing: Building database search queries and retrieving information.

[0352] Output: Related support information retrieved as search results

[0353] Step 6:

[0354] The server generates a response based on the acquired support information. It adjusts the response to the user with an appropriate tone and content, taking into account the emotion label detected by the emotion engine. For example, if the emotion "confused" is detected, it will include an apology such as, "We are sorry to trouble you."

[0355] Input: Related support information retrieved as search results

[0356] Data processing: Response text generation and tone adjustment

[0357] Output: Generated response

[0358] Step 7:

[0359] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[0360] Input: Generated response

[0361] Data processing: Decoding and formatting of response statements

[0362] Output: Response displayed in the user's chat box

[0363] The above outlines the specific processing flow of this system. Effective and personalized support is provided through a consistent process from user input to the display of responses.

[0364] (Application Example 2)

[0365] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0366] Conventional customer support systems have struggled to recognize user emotions and adjust the tone and content of responses accordingly, resulting in a decline in the quality of the user experience. Furthermore, simultaneously implementing multilingual support and emotion recognition capabilities is difficult, making improving customer satisfaction a particular challenge in global service provision. This invention aims to solve these problems and provide effective customer support that is attentive to the user's emotions.

[0367] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for recognizing the user's emotions based on the analyzed information, means for adjusting the tone and content of the response according to the emotion recognition results, means for providing the retrieved information to the user, means for supporting multiple languages, means for providing information to the user including multilingual responses, means for generating multilingual responses adjusted based on emotion recognition, means for including video links in the information provided to the user, means for the user to view support information based on the video links, and further means for selecting an appropriate video link based on the emotion recognition. This makes it possible to provide responses adapted to the user's emotions in real time and achieve high customer satisfaction even in a global context.

[0368] "Means of receiving information entered by users" refers to the function that allows the server to receive information entered by users through chat boxes or forms.

[0369] "Means for analyzing the input information" refers to a function that allows the server to analyze the user's input information received using natural language processing (NLP) or other analysis techniques to extract important keywords and phrases.

[0370] "Means for searching for relevant information based on the analysis results" refers to a function for searching for relevant information from a database using keywords or phrases obtained through the analysis.

[0371] "Means for providing the searched information to the user" refers to means for visually displaying the search results to the user, and includes websites, applications, chat boxes, etc.

[0372] "Means for recognizing the user's emotions based on the analyzed information" refers to a function that analyzes the user's emotions using an emotion engine based on the user's input information and assigns emotion labels such as "anger" or "confusion."

[0373] "Means for adjusting the tone and content of the response in accordance with the emotion recognition result" refers to a function for optimizing and adjusting the tone and content of the response presented to the user based on the emotion recognized by the emotion engine.

[0374] "Means of supporting multiple languages" refers to support functions that allow user input information and responses to be handled in multiple languages.

[0375] "Means of providing information to users, including multilingual responses" refers to a function that provides users with responses generated in multiple languages ​​based on analysis results, and displays them in an appropriate format according to the language the user is using.

[0376] "Means for generating multilingual responses based on the aforementioned emotion recognition" refers to a function for generating multilingual and emotion-appropriate response content based on the user's emotion recognition results.

[0377] "Means of including video links in the information provided to the user" refers to a function that provides the user with a link to watch a video as related support information.

[0378] "Means by which users can view support information based on the aforementioned video link" refers to a function that enables users to view support information through the provided video link.

[0379] "Means for selecting an appropriate video link based on the aforementioned emotion recognition" refers to a function for selecting and providing the most suitable video link for the user based on the emotion recognition results.

[0380] This invention relates to a system that analyzes support requests entered by users through websites and applications and provides relevant information. Specifically, it is characterized by analyzing the user's input information and emotions using a natural language processing (NLP) engine and an emotion engine, and generating an appropriate response based on that analysis.

[0381] System Configuration

[0382] This system consists of the following main components:

[0383] 1. User input receiving means:

[0384] This is a means of receiving information entered by users. Information is received through interfaces such as web forms, chat boxes, and mobile apps.

[0385] 2. Input information analysis means:

[0386] This is a method for analyzing the user's input information received. Specifically, it uses a natural language processing (NLP) engine (e.g., SpaCy, BERT) to extract important keywords and phrases from the input text.

[0387] 3. Emotion recognition means:

[0388] This is a method for recognizing user emotions based on NLP analysis results. It uses an emotion engine (e.g., IBM Watson Tone Analyzer) to detect emotions such as "anger" or "confusion" from user input.

[0389] 4. Related information retrieval methods:

[0390] This is a means of searching for relevant support information from a database based on the analysis results and sentiment recognition results. The support information is stored in a database such as Amazon RDS.

[0391] 5. Response generation means:

[0392] This is a means of generating appropriate responses for the user based on searched support information and sentiment recognition results. The generated responses are then adjusted in tone and content.

[0393] 6. Information provision method:

[0394] This is a means of providing the generated response to the user. The response is displayed through interfaces such as websites and mobile applications. The response may also include multilingual support.

[0395] 7. Method of providing video links:

[0396] This method includes including video links as support information, where necessary. This allows users to watch the videos and receive support to resolve their problems.

[0397] 8. Methods for selecting appropriate video links:

[0398] This is a means of selecting and providing appropriate video links to users based on emotion recognition results.

[0399] Hardware and software to be used

[0400] Server: Uses AWS® (Amazon Web Services) EC2 instances to receive, analyze, and process user input.

[0401] Database: Support information is stored using Amazon RDS (Relational Database Service).

[0402] NLP engine: Uses SpaCy or BERT for natural language processing.

[0403] Emotion Engine: Uses IBM Watson Tone Analyzer to recognize user emotions.

[0404] Frontend: Developing smartphone applications using React Native.

[0405] Specific example

[0406] The following is a specific example of how this system works.

[0407] Specific example 1: For Japanese users

[0408] The user types in the chat box, "My ordered item hasn't arrived yet, please check on it."

[0409] The server receives user input and uses an NLP engine to extract the keyword "product not delivered".

[0410] The emotion engine recognizes the emotion of "confusion."

[0411] The system searches the database for "information regarding delivery delays" and generates an apology message to address the user's feelings of distress.

[0412] The system provides the user with the most appropriate response, which includes a link such as "Check your current delivery status here."

[0413] Example of a prompt

[0414] User: "My ordered item hasn't arrived yet, please check on it."

[0415] server:

[0416] Keyword extraction: ["product", "not delivered"]

[0417] Emotional label: ["Confused"]

[0418] Response generated: "We apologize that your order has not yet arrived. Please check the current delivery status here. [link]."

[0419] As described above, this system can analyze user input information and emotions, and provide optimal support information in real time.

[0420] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0421] Step 1:

[0422] The user enters a support request into the chat box. Specifically, they might enter text such as, "My ordered item hasn't arrived yet, please check on it." The device receives this input and sends it to the server.

[0423] Input: User input text

[0424] Output: Input text sent to the server

[0425] Step 2:

[0426] The server analyzes the received input text using a natural language processing (NLP) engine. Specifically, it tokenizes the text and extracts important keywords and phrases. For example, "product" and "not delivered" might be extracted.

[0427] Input: Input text

[0428] Output: Extracted keywords and phrases

[0429] Step 3:

[0430] The server uses an emotion engine to recognize the user's emotions from the input text. Specifically, it analyzes the text and assigns emotion labels such as "confused" or "angry." In this step, the emotion "confused" is detected.

[0431] Input: Input text

[0432] Output: Emotional label (e.g., "Confused")

[0433] Step 4:

[0434] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. Specifically, it retrieves FAQs and guides related to the keyword "product not received." It also retrieves apology letters addressing the sentiment "confusion."

[0435] Input: Keywords, sentiment labels

[0436] Output: Related support information

[0437] Step 5:

[0438] The server generates an appropriate response based on the acquired support information and sentiment labels. Specifically, it combines relevant information to generate a response message for the user, adjusting the content and tone of the response. For example, it might generate a message like, "We apologize that your order has not yet arrived. You can check the current delivery status here."

[0439] Input: Relevant support information, sentiment labels

[0440] Output: Adjusted response

[0441] Step 6:

[0442] The server sends the generated response to the terminal. The terminal displays the received response to the user, for example, in a chat box.

[0443] Input: Adjusted response

[0444] Output: Response displayed to the user

[0445] With the above steps completed, the process of analyzing the user's input information and emotions, and providing optimal support information in real time, is finished.

[0446] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0447] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0448] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0449] [Second Embodiment]

[0450] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0451] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0452] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0453] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0454] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0455] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0456] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0457] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0458] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0459] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0460] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0461] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0462] The system of the present invention includes an interface in which users can submit support requests through a website or application, a server for receiving and analyzing requests, and a database for providing relevant information based on the analysis results.

[0463] Explanation of program processing

[0464] 1. Interface

[0465] Users type, for example, "I don't know how to cancel my order," into a chat box on a website or application. This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[0466] 2. Server-side processing

[0467] The server receives information sent by the user. The received information is first analyzed using a natural language processing (NLP) engine. Through this analysis, keywords such as "cancel" and "order" are extracted. Based on this, the server searches the database for relevant information to address specific scenarios and FAQs.

[0468] 3. Database Search

[0469] The server searches for relevant FAQs and support documents based on the analysis results. For example, guides and procedures related to "order cancellation" are retrieved from the database. The retrieved information is then formatted as an appropriate response.

[0470] 4. Generating and providing responses

[0471] The server generates a response for the user based on the information it has obtained. The response may include video links or detailed instructions as needed. For example, it may include specific instructions and links such as, "To cancel your order, please follow these steps. For details, please refer to this video link."

[0472] 5. Sending and displaying responses

[0473] The server sends the generated response to the user's terminal. The terminal decodes the received response and displays it to the user in the chat box. This allows the user to obtain relevant information in real time.

[0474] As a concrete example, consider the following scenario.

[0475] Example 1: For Japanese users

[0476] 1. User: "I'm having trouble with my internet connection. What should I do?"

[0477] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[0478] 3. As a solution to the "Internet connection problem," generate a response that includes a video link and instructions.

[0479] 4. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link (example.com / internetissue)."

[0480] 5. The terminal displays a response to the user.

[0481] Example 2: For English users

[0482] 1. User: "How can I reset my password?"

[0483] 2. The server analyzes the keyword "reset password" and searches the database for relevant support information.

[0484] 3. As a solution for "password reset," generate a response that includes instructions.

[0485] 4. Response: "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0486] 5. The terminal displays a response to the user.

[0487] The system of this invention allows users to receive appropriate and detailed support in real time, and enables companies to reduce operational costs while improving customer satisfaction.

[0488] The following describes the processing flow.

[0489] Step 1:

[0490] Users enter questions or problems into a chat box on a website or application. For example, they might type, "How do I cancel my order?"

[0491] Step 2:

[0492] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[0493] Step 3:

[0494] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[0495] Step 4:

[0496] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" might be extracted.

[0497] Step 5:

[0498] The server uses the extracted keywords to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[0499] Step 6:

[0500] The server organizes the search results and generates a message to respond to the user. At this stage, if multilingual support is required, the message is translated into the appropriate language. For example, a response such as "To cancel your order, please select 'Cancel' from 'Order History' on My Page" is generated.

[0501] Step 7:

[0502] The server sends the generated response to the terminal. The generated response is encoded and sent as an HTTP response.

[0503] Step 8:

[0504] The device decodes the HTTP response received from the server and displays a response message to the user. The answer is displayed in the chat box, allowing the user to get solutions to their questions and problems in real time. For example, it might display, "To cancel your order, please select 'Cancel' from 'Order History' on My Page."

[0505] This series of steps allows users to receive quick and appropriate support, and enables companies to provide efficient customer support.

[0506] (Example 1)

[0507] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0508] Traditional support systems faced challenges in responding quickly and accurately to user inquiries. In particular, efficient answers were often lacking when multilingual support or detailed instructions were required. Furthermore, the lack of adequate systems for providing users with the information they needed in an appropriate format resulted in a poor user experience.

[0509] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0510] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, means for automatically formatting the input information, means for transmitting the formatted information to the server, means for extracting keywords using a natural language processing engine based on the analysis results, means for generating a response based on the acquired information, means for transmitting the generated response to the user's terminal, and means for displaying the transmitted response to the user. This enables the user to receive detailed multilingual support information in real time.

[0511] A "user" is an individual or organization that enters a support request through the system and obtains information.

[0512] "Means of receiving information" refers to the interface and protocol that allows a system to receive information entered by a user.

[0513] "Means of analyzing information" refers to natural language processing engines and other analytical tools used to process received information and understand its content.

[0514] "Means of searching for relevant information" refers to the process of finding the necessary information from appropriate databases or resources based on the results of the analysis.

[0515] "Means of providing information" refers to technologies for formatting retrieved information and displaying or transmitting it appropriately to the user.

[0516] "Means for automatically formatting information" refer to protocols and algorithms that convert user-inputted information into a format that the system can easily understand.

[0517] "Means for sending formatted information to a server" refers to communication means for transferring appropriately formatted data to a server.

[0518] A "natural language processing engine" is a program that analyzes human language, understands its meaning, and extracts keywords.

[0519] "Means for generating responses" refer to the logic and software used to create appropriate answers to user inquiries based on extracted keywords and search results.

[0520] "Means for sending a response to the user's terminal" refers to the communication protocol and system for sending the generated response to the user's device.

[0521] "Means for displaying responses" refers to interface technology that allows users to visually confirm the responses they have received on their devices.

[0522] This invention relates to a system in which a user submits a support request through a website or application, the system analyzes the request, and provides an appropriate response. The main components of this system include an interface for receiving user input, a server for analyzing the input information, a database for retrieving relevant information based on the analysis results, and means for providing a response to the user.

[0523] 1. Interface

[0524] Users use the chat box on the website or application to enter inquiries, such as "I don't know how to cancel my order." This interface has a feature that automatically detects the user's language settings and converts the entered information into the appropriate format.

[0525] 2. Sending data

[0526] The terminal sends information entered by the user to the server. This process uses protocols such as HTTP requests to transmit data securely and reliably. Data integrity and security are ensured during this process.

[0527] 3. Analysis of Information

[0528] The server analyzes the received information using a natural language processing (NLP) engine. The NLP engine tokenizes the text, performs part-of-speech analysis, and extracts important keywords. For example, keywords such as "cancel" and "order" are extracted.

[0529] 4. Searching for related information

[0530] The server searches the database for relevant information based on the extracted keywords. The database contains FAQs, guides, and instruction manuals. The search results are retrieved as structured data.

[0531] 5. Response generation

[0532] The server generates a response for the user based on the information it has obtained. This response may include detailed instructions or video links. For example, it might say, "To cancel your order, please follow these steps. For more details, please refer to this video link."

[0533] 6. Sending a response

[0534] The server sends the generated response to the terminal. This process also uses protocols such as HTTP responses to ensure secure and reliable data transmission.

[0535] 7. Display of response

[0536] The terminal decodes the received response and displays it to the user in a chat box. The display is formatted for user readability, and interactive elements such as hyperlinks are provided as needed.

[0537] Specific example

[0538] For example, consider a case where a Japanese-speaking user types, "I'm having trouble with my internet connection. What should I do?" This information is sent to the server and securely analyzed. The server extracts the keyword "internet connection" and searches its database for relevant support information. As a result, a response is generated that includes a video link and instructions, and the user is provided with a response such as, "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link."

[0539] For English-speaking users, typing "How can I reset my password?" will trigger a similar analysis process, extracting the keyword "reset password." The server will then search for relevant password reset information and generate a response containing instructions. The user will receive a response such as, "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword."

[0540] Examples of prompts for generative AI models

[0541] "I'm having trouble with my internet connection. What should I do?"

[0542] "I need help with resetting my password."

[0543] This system allows users to receive detailed, multilingual support information in real time, while enabling businesses to reduce operational costs and improve customer satisfaction.

[0544] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0545] Step 1: The user enters their question into the chat box of the website or application.

[0546] Input: The user enters the question in text format (e.g., "I don't know how to cancel my order").

[0547] Operation: The interface receives user input and automatically detects language settings.

[0548] Output: Formatted text information.

[0549] Step 2: The terminal sends user input to the server.

[0550] Input: Formatted text information generated by the interface.

[0551] Operation: The terminal uses an HTTP request to send formatted text information to the server.

[0552] Output: Text information received by the server.

[0553] Step 3: The server analyzes the received information using a natural language processing (NLP) engine.

[0554] Input: Text information received by the server.

[0555] Operation: The NLP engine performs the following processes:

[0556] Tokenization: Dividing text into words or phrases.

[0557] Part-of-speech analysis: Identifies the part of speech of each token.

[0558] Keyword extraction: Extraction of important keywords such as "cancel" and "order".

[0559] Output: Extracted keywords and analyzed text information.

[0560] Step 4: The server searches the database for relevant information based on the extracted keywords.

[0561] Input: Extracted keywords.

[0562] Operation: The process is carried out using the following steps:

[0563] Query generation: Generates search queries based on the extracted keywords.

[0564] Database query: Use the generated query to search the database for relevant FAQs and support documents.

[0565] Retrieving search results: Relevant information is retrieved from multiple databases.

[0566] Output: Relevant information retrieved from the database.

[0567] Step 5: The server generates a response based on the information it has obtained.

[0568] Input: Related information retrieved from the database.

[0569] Operation: Generates an answer that includes the following items:

[0570] Text generation: Creates appropriate response sentences based on user questions.

[0571] Additional information attached: Add video links and detailed instructions as needed.

[0572] Response format: Converts the generated response into a format that can be appropriately displayed to the user.

[0573] Output: Formatted answers and additional information.

[0574] Step 6: The server sends the generated response to the user's terminal.

[0575] Input: Formatted response.

[0576] Operation: The server uses the HTTP response to send the generated response to the user's terminal.

[0577] Output: Response data received by the terminal.

[0578] Step 7: The terminal displays the received response to the user.

[0579] Input: Response data received by the terminal.

[0580] Operation: The device decodes the received data and displays it in the chat box in a user-friendly format. Interactive elements such as hyperlinks and video links are also displayed as needed.

[0581] Output: The response displayed on the user's screen.

[0582] This system allows users to receive detailed, multilingual support information in real time. Businesses can reduce operational costs while improving customer satisfaction.

[0583] (Application Example 1)

[0584] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0585] Currently, many e-commerce sites dedicate significant resources to user support, and there is a demand for fast and efficient support. However, traditional systems have limitations in improving customer satisfaction and reducing operational costs due to language differences and the need for manual responses. In particular, there is a challenge in providing detailed information in real time when responding to requests such as order cancellations and product returns.

[0586] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0587] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, and means for utilizing a natural language processing engine to generate a response to the user. This enables the user to receive support through a multilingual chat box and to be provided with comprehensive and personalized information in real time.

[0588] "Means of receiving information entered by users" refers to methods of collecting questions and requests entered by users through websites and applications.

[0589] "Means for analyzing the input information" refers to means of analyzing the collected user questions and requests using a natural language processing engine to identify keywords and intents.

[0590] "Means for searching for relevant information based on the analysis results" refers to means for searching a database for relevant FAQs and support documents based on keywords and intents obtained through the analysis.

[0591] "Means of providing the searched information to the user" refers to means of formatting the support information obtained through the search and displaying it to the user through a chat box or other interface.

[0592] "Means for using a natural language processing engine to generate a response to the user" refers to means for analyzing input from the user and automatically generating a response in a natural conversational format based on the analysis results.

[0593] "Means of supporting multiple languages" refers to means that can translate and convert user input information, analysis results, and responses into multiple languages.

[0594] "Methods for providing information in real time" refer to means of providing information quickly by immediately performing analysis, information retrieval, and response generation in response to user requests.

[0595] "Means including video links" refers to means that include a URL link to a video related to the acquired support information, enabling the user to view specific instructions.

[0596] "A means of generating individual responses based on user requests using a generative AI model" refers to a method of generating personalized responses to different user requests by utilizing artificial intelligence.

[0597] This invention is a smartphone application system for e-commerce sites that analyzes user support requests in real time and provides appropriate information. This system consists of the following main elements:

[0598] 1. User Interface

[0599] Users can enter questions or requests into a chat box within the application. For example, they might type, "I don't know how to cancel my order." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[0600] 2. Server-side processing

[0601] The server first receives information sent by the user. The received information is parsed using a natural language processing engine (e.g., SpaCy, Google Natural Language API). Through this parsing, keywords such as "cancel" and "order" are extracted. Based on this, the server searches for relevant information in a database (e.g., MySQL, PostgreSQL).

[0602] 3. Database Search

[0603] Based on the analysis results, the server searches the database for relevant FAQs and support documents. For example, guides and procedures related to "order cancellation" are retrieved from the database.

[0604] 4. Generating a response

[0605] The server generates specific responses for the user based on the information it has obtained. These responses may include video links or detailed instructions as needed. For example, a response might be generated stating, "The procedure for canceling your order is as follows. Please refer to this link for details: example.com / cancelorder." This process also utilizes a generative AI model to generate personalized responses.

[0606] 5. Sending and displaying responses

[0607] The server sends the generated response to the user's device. The user's smartphone application decodes the received response and displays it in the chat box. This allows the user to obtain relevant information in real time.

[0608] Hardware and software configuration

[0609] hardware

[0610] Server (requires high-performance computing power)

[0611] User's smartphone

[0612] software

[0613] Natural language processing engines (e.g., SpaCy, Google Natural Language API)

[0614] Databases (e.g., MySQL, PostgreSQL)

[0615] Web application frameworks (e.g., Flask, Django)

[0616] Generative AI models (e.g., ChatGPT)

[0617] Specific example

[0618] Example 1: For Japanese users

[0619] 1. User: "I'm having trouble with my internet connection. What should I do?"

[0620] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[0621] 3. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link: example.com / internetissue"

[0622] 4. The terminal displays a response to the user.

[0623] Example of a prompt

[0624] "I don't know how to return an item. What should I do?"

[0625] In this way, users can receive support through a multilingual chat box, and comprehensive and personalized information can be provided in real time. This system allows e-commerce sites to improve operational efficiency and enhance customer satisfaction.

[0626] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0627] Step 1: The user enters a question in the chat box.

[0628] Users enter questions or requests into a chat box within the e-commerce site's smartphone application. For example, they might type, "I don't know how to cancel my order."

[0629] Input: User's question or request

[0630] Output: Data sent to the server containing user input.

[0631] Step 2: The server receives user input.

[0632] The server receives information sent by the user. The received information is then analyzed using a natural language processing engine.

[0633] Input: User input

[0634] Output: Data ready for analysis

[0635] Step 3: Analyze the text using a natural language processing engine.

[0636] The server analyzes the received information using a natural language processing engine. During this process, keywords such as "cancel" and "order" are extracted.

[0637] Input: Received user input

[0638] Output: Extracted keyword and intent data

[0639] Step 4: Search for relevant information in the database

[0640] Based on the analysis results, the server searches the database for relevant FAQs and support documents. Specifically, it retrieves information related to "order cancellation."

[0641] Input: Extracted keywords and intent

[0642] Output: Relevant support information (FAQs and procedure manuals)

[0643] Step 5: Generate a response

[0644] The server generates specific responses for the user based on the information it has acquired. These responses may include video links or detailed instructions as needed.

[0645] Input: Related support information

[0646] Output: Final response data to be provided to the user

[0647] Step 6: Send the response to the user's device.

[0648] The server sends the generated response to the user's smartphone. The user's device decodes the received response and displays it in the chat box.

[0649] Input: Final response data

[0650] Output: Decoded data for display

[0651] Step 7: The user views the response.

[0652] Users can view the responses displayed in the chat box and find the necessary support information and procedures. This allows users to resolve issues in real time.

[0653] Input: Decoded data

[0654] Output: User understanding and behavior

[0655] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0656] The system of the present invention includes an interface in which users enter support requests through a website or application, a server for receiving and analyzing requests, a database for providing relevant information based on the analysis results, and an emotion engine for recognizing the user's emotions.

[0657] Explanation of program processing

[0658] 1. Interface

[0659] Users type messages into a chat box on a website or application, such as, "My order has been cancelled; please help me with this." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[0660] 2. Server-side processing

[0661] The server receives information sent by the user and first analyzes it using a natural language processing (NLP) engine. The NLP engine tokenizes the text and extracts important keywords and phrases such as "cancel" and "order." At the same time, an emotion engine detects emotions from the user's input and assigns emotion labels such as "anger" or "confusion."

[0662] 3. Database Search

[0663] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. For example, it retrieves guides and FAQs related to "order cancellation," and additional support information corresponding to the sentiment as needed.

[0664] 4. Generating and providing responses

[0665] The server generates a response to the user based on the information it has gathered. The tone and content of the response are adjusted based on the emotions recognized by the emotion engine. For example, a response might be generated such as, "We apologize for the inconvenience. Please refer to the following steps for instructions on how to cancel your order." Detailed instructions and video links may also be included.

[0666] 5. Sending and displaying responses

[0667] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[0668] As a concrete example, consider the following scenario.

[0669] Example 1: For Japanese users

[0670] 1. User: "My delivery is delayed and I'm having trouble with it. Is there anything you can do?"

[0671] 2. The server uses an NLP engine to extract keywords such as "delivery delay" and an emotion engine to detect the emotion of "confusion."

[0672] 3. Generate a response that includes a guide regarding "delivery delays" and an apology based on "confusion."

[0673] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[0674] 5. The terminal displays a response to the user.

[0675] Example 2: For English users

[0676] 1. User: "I can't log into my account and I'm getting frustrated."

[0677] 2. The server uses an NLP engine to extract the keyword "cannot log in" and an emotion engine to detect the emotion "frustration".

[0678] 3. Generate a response that includes a guide regarding "login issues" and encouragement based on "frustration."

[0679] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0680] 5. The terminal displays a response to the user.

[0681] The system of this invention allows users to receive appropriate and detailed support in real time, enabling companies to reduce operational costs while improving customer satisfaction. By utilizing an emotion engine, responses can be tailored to the user's emotions, resulting in more personalized support.

[0682] The following describes the processing flow.

[0683] Step 1:

[0684] Users enter questions or problems into the chat box of a website or application. For example, they might type, "My order has been cancelled, please help me with this."

[0685] Step 2:

[0686] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[0687] Step 3:

[0688] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[0689] Step 4:

[0690] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" are extracted. At the same time, an emotion engine detects the user's emotions from the text and assigns emotion labels such as "anger" or "confusion."

[0691] Step 5:

[0692] The server uses the extracted keywords and sentiment labels to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[0693] Step 6:

[0694] The server organizes search results and generates a message to respond to the user. Based on the emotions recognized by the sentiment engine, the tone and content of the response are adjusted. For example, a message such as, "We apologize for the inconvenience. Please refer to the following steps for how to cancel your order," might be generated. It may also include video links or detailed instructions.

[0695] Step 7:

[0696] The server sends the generated response to the terminal. The generated response is encoded as an HTTP response and sent.

[0697] Step 8:

[0698] The terminal decodes the HTTP response received from the server and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[0699] Specific example

[0700] Example 1: For Japanese users

[0701] 1. User: "My delivery is delayed and I'm having trouble with it. Is there anything you can do?"

[0702] 2. The server uses an NLP engine to extract keywords such as "delivery delay" and an emotion engine to detect the emotion of "confusion."

[0703] 3. Generate a response that includes a guide regarding "delivery delays" and an apology based on "confusion."

[0704] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[0705] 5. The terminal displays a response to the user.

[0706] Example 2: For English users

[0707] 1. User: "I can't log into my account and I'm getting frustrated."

[0708] 2. The server uses an NLP engine to extract the keyword "cannot log in" and an emotion engine to detect the emotion "frustration".

[0709] 3. Generate a response that includes a guide regarding "login issues" and encouragement based on "frustration."

[0710] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0711] 5. The terminal displays a response to the user.

[0712] The above is a specific embodiment of a system that combines an emotion engine to recognize user emotions. This system allows users to receive appropriate and detailed support in real time, enabling companies to reduce operational costs while improving customer satisfaction. The use of the emotion engine allows for responses tailored to the user's emotions, resulting in more personalized support.

[0713] (Example 2)

[0714] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0715] Traditional support systems struggled to adequately address users' problems and emotions, highlighting the need for improved user experience. Furthermore, insufficient multilingual support often led to delays in assisting users who spoke different languages. This resulted in increased operational costs and decreased customer satisfaction. Additionally, users had difficulty obtaining necessary information quickly, hindering the provision of appropriate support.

[0716] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0717] In this invention, the server includes means for analyzing information input by a user using language analysis means, means for detecting emotions and assigning emotion labels based on the analysis, and means for generating responses with tone and content appropriate to the emotions. This enables rapid and appropriate analysis of emotions based on user input and personalized responses based on the results. Furthermore, it enables the provision of information in multiple languages ​​and allows for rapid response to users who speak different languages, thereby reducing operational costs for companies and improving customer satisfaction.

[0718] "User-generated information" refers to text and data that users enter into a system to provide information such as support requests through a website or application.

[0719] "Language analysis means" refers to technologies and devices that analyze information input by users, understand its meaning, and extract key keywords. Specifically, natural language processing technology is included in this category.

[0720] "Means for detecting emotions and assigning emotion labels" refers to technologies and devices that analyze emotions from user input information and assign corresponding emotion tags. For example, this includes processes for classifying and tagging emotions such as anger, confusion, and joy.

[0721] "Means for searching for relevant information" refers to technologies and devices that search for and retrieve relevant information from databases and other information repositories based on the analyzed user input information.

[0722] "Means of generating responses" refers to technologies and devices that automatically generate appropriate responses and guidelines for users based on the searched relevant information and sentiment labels.

[0723] "Means of making information multilingual" refers to technologies and devices that translate user input information and generated responses into different languages, making them usable in a multilingual environment.

[0724] "Means including viewing links" refers to technologies and devices that include links to videos or tutorials in the information provided to users. This makes it easier for users to receive support in a visual way.

[0725] The system of this invention provides rapid and personalized support by analyzing support requests from users and generating appropriate responses. This system primarily consists of four parts: an interface, a server, a database, and an emotion engine. The following describes each component and its operation in detail.

[0726] interface

[0727] Users enter support requests into chat boxes on websites or applications. For example, a request might say, "My order has been cancelled; please help me with this." This interface receives the user's input, automatically detects their language settings, and sends the information to the server in the appropriate format.

[0728] server

[0729] The server receives information transmitted through the interface. This received information is first analyzed by a natural language processing (NLP) engine. This NLP engine uses services such as Google Cloud Natural Language API and Microsoft Azure Text Analytics to tokenize the text and extract important keywords and phrases. Next, a sentiment engine is used to detect emotions from the user's input text and assign sentiment labels such as "anger" or "confusion." IBM Watson Tone Analyzer is used for this sentiment analysis.

[0730] database

[0731] The server searches the database for relevant support information based on the analysis results and sentiment labels. The database is managed by a database management system such as MySQL or MongoDB. For example, guides and FAQs related to "order cancellation" are retrieved, as well as additional support information corresponding to the sentiment as needed.

[0732] Response generation and provision

[0733] The server generates a response to the user based on the information it has acquired. The tone and content of the response are adjusted based on the emotions recognized by the emotion engine. For example, in response to user confusion, a response such as, "We apologize for the inconvenience. Please refer to the following steps for instructions on how to cancel your order," might be generated. Detailed instructions and video links may also be included. This response can also be generated in multiple languages ​​based on the user's language settings.

[0734] Sending and displaying responses

[0735] The generated response is sent from the server to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[0736] Specific example

[0737] Example 1: For Japanese users

[0738] User input: "My delivery is delayed and I'm having trouble. Can you do something about it?"

[0739] NLP engine analysis: Extraction of the keyword "delivery delay"

[0740] Emotional Engine Analysis: Detecting the emotion of "confusion"

[0741] Database Search: Retrieve guides and apology letters regarding "delivery delays" and "confusion."

[0742] Generated response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[0743] Display to the user: The device displays the response in the chat box.

[0744] Example 2: For English users

[0745] User input: "I can't log into my account and I'm getting frustrated."

[0746] NLP engine analysis: Extracting the keyword "cannot log in".

[0747] Emotional Engine Analysis: Detecting the emotion of "frustration"

[0748] Database Search: Get guides and encouragement based on "login issues" and "frustration."

[0749] Generated response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0750] Display to the user: The device displays the response in the chat box.

[0751] This invention's system allows users to receive fast, personalized support in real time. In particular, its multilingual support and emotion recognition enable it to appropriately respond to users with different languages ​​and emotions. This allows companies to reduce operational costs and improve customer satisfaction.

[0752] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0753] Step 1:

[0754] Users enter support requests into a chat box on the website or application. For example, they might type, "My order was cancelled, please help me." This user input is the starting point for the system.

[0755] Input: Text of the support request entered by the user

[0756] Output: User input text is passed to the interface.

[0757] Step 2:

[0758] The interface receives user input text and automatically detects its language settings. This detection uses information such as web browser settings and account information. It then converts the input text to the appropriate format and sends it to the server.

[0759] Input: User input text

[0760] Data processing: Language setting detection and text formatting conversion

[0761] Output: Formatted text is sent to the server.

[0762] Step 3:

[0763] The server receives text data transmitted through the interface. The received text is sent to a natural language processing (NLP) engine for text analysis. The NLP engine tokenizes the text and extracts key keywords and phrases.

[0764] Input: Formatted text

[0765] Data processing: Text analysis and tokenization using a natural language processing engine.

[0766] Output: List of extracted keywords and phrases

[0767] Step 4:

[0768] The server sends the extracted keywords and phrases to the sentiment engine. The sentiment engine detects emotions from the text and assigns emotion labels such as "anger" or "confusion."

[0769] Input: List of extracted keywords or phrases

[0770] Data processing: Emotion analysis and emotion labeling using an emotion engine.

[0771] Output: Keywords and phrases with emotion labels

[0772] Step 5:

[0773] The server searches the database for relevant support information based on the analysis results and sentiment labels. The server constructs a search query and retrieves the relevant information using a database management system (e.g., MySQL or MongoDB).

[0774] Input: Keywords or phrases with emotion labels

[0775] Data Processing: Building database search queries and retrieving information.

[0776] Output: Related support information retrieved as search results

[0777] Step 6:

[0778] The server generates a response based on the acquired support information. It adjusts the response to the user with an appropriate tone and content, taking into account the emotion label detected by the emotion engine. For example, if the emotion "confused" is detected, it will include an apology such as, "We are sorry to trouble you."

[0779] Input: Related support information retrieved as search results

[0780] Data processing: Response text generation and tone adjustment

[0781] Output: Generated response

[0782] Step 7:

[0783] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[0784] Input: Generated response

[0785] Data processing: Decoding and formatting of response statements

[0786] Output: Response displayed in the user's chat box

[0787] The above outlines the specific processing flow of this system. Effective and personalized support is provided through a consistent process from user input to the display of responses.

[0788] (Application Example 2)

[0789] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0790] Conventional customer support systems have struggled to recognize user emotions and adjust the tone and content of responses accordingly, resulting in a decline in the quality of the user experience. Furthermore, simultaneously implementing multilingual support and emotion recognition capabilities is difficult, making improving customer satisfaction a particular challenge in global service provision. This invention aims to solve these problems and provide effective customer support that is attentive to the user's emotions.

[0791] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for recognizing the user's emotions based on the analyzed information, means for adjusting the tone and content of the response according to the emotion recognition results, means for providing the retrieved information to the user, means for supporting multiple languages, means for providing information to the user including multilingual responses, means for generating multilingual responses adjusted based on emotion recognition, means for including video links in the information provided to the user, means for the user to view support information based on the video links, and further means for selecting an appropriate video link based on the emotion recognition. This makes it possible to provide responses adapted to the user's emotions in real time and achieve high customer satisfaction even in a global context.

[0792] "Means of receiving information entered by users" refers to the function that allows the server to receive information entered by users through chat boxes or forms.

[0793] "Means for analyzing the input information" refers to a function that allows the server to analyze the user's input information received using natural language processing (NLP) or other analysis techniques to extract important keywords and phrases.

[0794] "Means for searching for relevant information based on the analysis results" refers to a function for searching for relevant information from a database using keywords or phrases obtained through the analysis.

[0795] "Means for providing the searched information to the user" refers to means for visually displaying the search results to the user, and includes websites, applications, chat boxes, etc.

[0796] "Means for recognizing the user's emotions based on the analyzed information" refers to a function that analyzes the user's emotions using an emotion engine based on the user's input information and assigns emotion labels such as "anger" or "confusion."

[0797] "Means for adjusting the tone and content of the response in accordance with the emotion recognition result" refers to a function for optimizing and adjusting the tone and content of the response presented to the user based on the emotion recognized by the emotion engine.

[0798] "Means of supporting multiple languages" refers to support functions that allow user input information and responses to be handled in multiple languages.

[0799] "Means of providing information to users, including multilingual responses" refers to a function that provides users with responses generated in multiple languages ​​based on analysis results, and displays them in an appropriate format according to the language the user is using.

[0800] "Means for generating multilingual responses based on the aforementioned emotion recognition" refers to a function for generating multilingual and emotion-appropriate response content based on the user's emotion recognition results.

[0801] "Means of including video links in the information provided to the user" refers to a function that provides the user with a link to watch a video as related support information.

[0802] "Means by which users can view support information based on the aforementioned video link" refers to a function that enables users to view support information through the provided video link.

[0803] "Means for selecting an appropriate video link based on the aforementioned emotion recognition" refers to a function for selecting and providing the most suitable video link for the user based on the emotion recognition results.

[0804] This invention relates to a system that analyzes support requests entered by users through websites and applications and provides relevant information. Specifically, it is characterized by analyzing the user's input information and emotions using a natural language processing (NLP) engine and an emotion engine, and generating an appropriate response based on that analysis.

[0805] System Configuration

[0806] This system consists of the following main components:

[0807] 1. User input receiving means:

[0808] This is a means of receiving information entered by users. Information is received through interfaces such as web forms, chat boxes, and mobile apps.

[0809] 2. Input information analysis means:

[0810] This is a method for analyzing the user's input information received. Specifically, it uses a natural language processing (NLP) engine (e.g., SpaCy, BERT) to extract important keywords and phrases from the input text.

[0811] 3. Emotion recognition means:

[0812] This is a method for recognizing user emotions based on NLP analysis results. It uses an emotion engine (e.g., IBM Watson Tone Analyzer) to detect emotions such as "anger" or "confusion" from user input.

[0813] 4. Related information retrieval methods:

[0814] This is a means of searching for relevant support information from a database based on the analysis results and sentiment recognition results. The support information is stored in a database such as Amazon RDS.

[0815] 5. Response generation means:

[0816] This is a means of generating appropriate responses for the user based on searched support information and sentiment recognition results. The generated responses are then adjusted in tone and content.

[0817] 6. Information provision method:

[0818] This is a means of providing the generated response to the user. The response is displayed through interfaces such as websites and mobile applications. The response may also include multilingual support.

[0819] 7. Method of providing video links:

[0820] This method includes including video links as support information, where necessary. This allows users to watch the videos and receive support to resolve their problems.

[0821] 8. Methods for selecting appropriate video links:

[0822] This is a means of selecting and providing appropriate video links to users based on emotion recognition results.

[0823] Hardware and software to be used

[0824] Server: An AWS (Amazon Web Services) EC2 instance is used to receive, analyze, and process user input.

[0825] Database: Support information is stored using Amazon RDS (Relational Database Service).

[0826] NLP engine: Uses SpaCy or BERT for natural language processing.

[0827] Emotion Engine: Uses IBM Watson Tone Analyzer to recognize user emotions.

[0828] Frontend: Developing smartphone applications using React Native.

[0829] Specific example

[0830] The following is a specific example of how this system works.

[0831] Specific example 1: For Japanese users

[0832] The user types in the chat box, "My ordered item hasn't arrived yet, please check on it."

[0833] The server receives user input and uses an NLP engine to extract the keyword "product not delivered".

[0834] The emotion engine recognizes the emotion of "confusion."

[0835] The system searches the database for "information regarding delivery delays" and generates an apology message to address the user's feelings of distress.

[0836] The system provides the user with the most appropriate response, which includes a link such as "Check your current delivery status here."

[0837] Example of a prompt

[0838] User: "My ordered item hasn't arrived yet, please check on it."

[0839] server:

[0840] Keyword extraction: ["product", "not delivered"]

[0841] Emotional label: ["Confused"]

[0842] Response generated: "We apologize that your order has not yet arrived. Please check the current delivery status here. [link]."

[0843] As described above, this system can analyze user input information and emotions, and provide optimal support information in real time.

[0844] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0845] Step 1:

[0846] The user enters a support request into the chat box. Specifically, they might enter text such as, "My ordered item hasn't arrived yet, please check on it." The device receives this input and sends it to the server.

[0847] Input: User input text

[0848] Output: Input text sent to the server

[0849] Step 2:

[0850] The server analyzes the received input text using a natural language processing (NLP) engine. Specifically, it tokenizes the text and extracts important keywords and phrases. For example, "product" and "not delivered" might be extracted.

[0851] Input: Input text

[0852] Output: Extracted keywords and phrases

[0853] Step 3:

[0854] The server uses an emotion engine to recognize the user's emotions from the input text. Specifically, it analyzes the text and assigns emotion labels such as "confused" or "angry." In this step, the emotion "confused" is detected.

[0855] Input: Input text

[0856] Output: Emotional label (e.g., "Confused")

[0857] Step 4:

[0858] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. Specifically, it retrieves FAQs and guides related to the keyword "product not received." It also retrieves apology letters addressing the sentiment "confusion."

[0859] Input: Keywords, sentiment labels

[0860] Output: Related support information

[0861] Step 5:

[0862] The server generates an appropriate response based on the acquired support information and sentiment labels. Specifically, it combines relevant information to generate a response message for the user, adjusting the content and tone of the response. For example, it might generate a message like, "We apologize that your order has not yet arrived. You can check the current delivery status here."

[0863] Input: Relevant support information, sentiment labels

[0864] Output: Adjusted response

[0865] Step 6:

[0866] The server sends the generated response to the terminal. The terminal displays the received response to the user, for example, in a chat box.

[0867] Input: Adjusted response

[0868] Output: Response displayed to the user

[0869] With the above steps completed, the process of analyzing the user's input information and emotions, and providing optimal support information in real time, is finished.

[0870] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0871] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0872] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0873] [Third Embodiment]

[0874] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0875] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0876] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0877] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0878] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0879] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0880] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0881] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0882] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0883] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0884] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0885] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0886] The system of the present invention includes an interface in which users can submit support requests through a website or application, a server for receiving and analyzing requests, and a database for providing relevant information based on the analysis results.

[0887] Explanation of program processing

[0888] 1. Interface

[0889] Users type, for example, "I don't know how to cancel my order," into a chat box on a website or application. This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[0890] 2. Server-side processing

[0891] The server receives information sent by the user. The received information is first analyzed using a natural language processing (NLP) engine. Through this analysis, keywords such as "cancel" and "order" are extracted. Based on this, the server searches the database for relevant information to address specific scenarios and FAQs.

[0892] 3. Database Search

[0893] The server searches for relevant FAQs and support documents based on the analysis results. For example, guides and procedures related to "order cancellation" are retrieved from the database. The retrieved information is then formatted as an appropriate response.

[0894] 4. Generating and providing responses

[0895] The server generates a response for the user based on the information it has obtained. The response may include video links or detailed instructions as needed. For example, it may include specific instructions and links such as, "To cancel your order, please follow these steps. For details, please refer to this video link."

[0896] 5. Sending and displaying responses

[0897] The server sends the generated response to the user's terminal. The terminal decodes the received response and displays it to the user in the chat box. This allows the user to obtain relevant information in real time.

[0898] As a concrete example, consider the following scenario.

[0899] Example 1: For Japanese users

[0900] 1. User: "I'm having trouble with my internet connection. What should I do?"

[0901] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[0902] 3. As a solution to the "Internet connection problem," generate a response that includes a video link and instructions.

[0903] 4. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link (example.com / internetissue)."

[0904] 5. The terminal displays a response to the user.

[0905] Example 2: For English users

[0906] 1. User: "How can I reset my password?"

[0907] 2. The server analyzes the keyword "reset password" and searches the database for relevant support information.

[0908] 3. As a solution for "password reset," generate a response that includes instructions.

[0909] 4. Response: "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[0910] 5. The terminal displays a response to the user.

[0911] The system of this invention allows users to receive appropriate and detailed support in real time, and enables companies to reduce operational costs while improving customer satisfaction.

[0912] The following describes the processing flow.

[0913] Step 1:

[0914] Users enter questions or problems into a chat box on a website or application. For example, they might type, "How do I cancel my order?"

[0915] Step 2:

[0916] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[0917] Step 3:

[0918] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[0919] Step 4:

[0920] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" might be extracted.

[0921] Step 5:

[0922] The server uses the extracted keywords to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[0923] Step 6:

[0924] The server organizes the search results and generates a message to respond to the user. At this stage, if multilingual support is required, the message is translated into the appropriate language. For example, a response such as "To cancel your order, please select 'Cancel' from 'Order History' on My Page" is generated.

[0925] Step 7:

[0926] The server sends the generated response to the terminal. The generated response is encoded and sent as an HTTP response.

[0927] Step 8:

[0928] The device decodes the HTTP response received from the server and displays a response message to the user. The answer is displayed in the chat box, allowing the user to get solutions to their questions and problems in real time. For example, it might display, "To cancel your order, please select 'Cancel' from 'Order History' on My Page."

[0929] This series of steps allows users to receive quick and appropriate support, and enables companies to provide efficient customer support.

[0930] (Example 1)

[0931] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0932] Traditional support systems faced challenges in responding quickly and accurately to user inquiries. In particular, efficient answers were often lacking when multilingual support or detailed instructions were required. Furthermore, the lack of adequate systems for providing users with the information they needed in an appropriate format resulted in a poor user experience.

[0933] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0934] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, means for automatically formatting the input information, means for transmitting the formatted information to the server, means for extracting keywords using a natural language processing engine based on the analysis results, means for generating a response based on the acquired information, means for transmitting the generated response to the user's terminal, and means for displaying the transmitted response to the user. This enables the user to receive detailed multilingual support information in real time.

[0935] A "user" is an individual or organization that enters a support request through the system and obtains information.

[0936] "Means of receiving information" refers to the interface and protocol that allows a system to receive information entered by a user.

[0937] "Means of analyzing information" refers to natural language processing engines and other analytical tools used to process received information and understand its content.

[0938] "Means of searching for relevant information" refers to the process of finding the necessary information from appropriate databases or resources based on the results of the analysis.

[0939] "Means of providing information" refers to technologies for formatting retrieved information and displaying or transmitting it appropriately to the user.

[0940] "Means for automatically formatting information" refer to protocols and algorithms that convert user-inputted information into a format that the system can easily understand.

[0941] "Means for sending formatted information to a server" refers to communication means for transferring appropriately formatted data to a server.

[0942] A "natural language processing engine" is a program that analyzes human language, understands its meaning, and extracts keywords.

[0943] "Means for generating responses" refer to the logic and software used to create appropriate answers to user inquiries based on extracted keywords and search results.

[0944] "Means for sending a response to the user's terminal" refers to the communication protocol and system for sending the generated response to the user's device.

[0945] "Means for displaying responses" refers to interface technology that allows users to visually confirm the responses they have received on their devices.

[0946] This invention relates to a system in which a user submits a support request through a website or application, the system analyzes the request, and provides an appropriate response. The main components of this system include an interface for receiving user input, a server for analyzing the input information, a database for retrieving relevant information based on the analysis results, and means for providing a response to the user.

[0947] 1. Interface

[0948] Users use the chat box on the website or application to enter inquiries, such as "I don't know how to cancel my order." This interface has a feature that automatically detects the user's language settings and converts the entered information into the appropriate format.

[0949] 2. Sending data

[0950] The terminal sends information entered by the user to the server. This process uses protocols such as HTTP requests to transmit data securely and reliably. Data integrity and security are ensured during this process.

[0951] 3. Analysis of Information

[0952] The server analyzes the received information using a natural language processing (NLP) engine. The NLP engine tokenizes the text, performs part-of-speech analysis, and extracts important keywords. For example, keywords such as "cancel" and "order" are extracted.

[0953] 4. Searching for related information

[0954] The server searches the database for relevant information based on the extracted keywords. The database contains FAQs, guides, and instruction manuals. The search results are retrieved as structured data.

[0955] 5. Response generation

[0956] The server generates a response for the user based on the information it has obtained. This response may include detailed instructions or video links. For example, it might say, "To cancel your order, please follow these steps. For more details, please refer to this video link."

[0957] 6. Sending a response

[0958] The server sends the generated response to the terminal. This process also uses protocols such as HTTP responses to ensure secure and reliable data transmission.

[0959] 7. Display of response

[0960] The terminal decodes the received response and displays it to the user in a chat box. The display is formatted for user readability, and interactive elements such as hyperlinks are provided as needed.

[0961] Specific example

[0962] For example, consider a case where a Japanese-speaking user types, "I'm having trouble with my internet connection. What should I do?" This information is sent to the server and securely analyzed. The server extracts the keyword "internet connection" and searches its database for relevant support information. As a result, a response is generated that includes a video link and instructions, and the user is provided with a response such as, "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link."

[0963] For English-speaking users, typing "How can I reset my password?" will trigger a similar analysis process, extracting the keyword "reset password." The server will then search for relevant password reset information and generate a response containing instructions. The user will receive a response such as, "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword."

[0964] Examples of prompts for generative AI models

[0965] "I'm having trouble with my internet connection. What should I do?"

[0966] "I need help with resetting my password."

[0967] This system allows users to receive detailed, multilingual support information in real time, while enabling businesses to reduce operational costs and improve customer satisfaction.

[0968] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0969] Step 1: The user enters their question into the chat box of the website or application.

[0970] Input: The user enters the question in text format (e.g., "I don't know how to cancel my order").

[0971] Operation: The interface receives user input and automatically detects language settings.

[0972] Output: Formatted text information.

[0973] Step 2: The terminal sends user input to the server.

[0974] Input: Formatted text information generated by the interface.

[0975] Operation: The terminal uses an HTTP request to send formatted text information to the server.

[0976] Output: Text information received by the server.

[0977] Step 3: The server analyzes the received information using a natural language processing (NLP) engine.

[0978] Input: Text information received by the server.

[0979] Operation: The NLP engine performs the following processes:

[0980] Tokenization: Dividing text into words or phrases.

[0981] Part-of-speech analysis: Identifies the part of speech of each token.

[0982] Keyword extraction: Extraction of important keywords such as "cancel" and "order".

[0983] Output: Extracted keywords and analyzed text information.

[0984] Step 4: The server searches the database for relevant information based on the extracted keywords.

[0985] Input: Extracted keywords.

[0986] Operation: The process is carried out using the following steps:

[0987] Query generation: Generates search queries based on the extracted keywords.

[0988] Database query: Use the generated query to search the database for relevant FAQs and support documents.

[0989] Retrieving search results: Relevant information is retrieved from multiple databases.

[0990] Output: Relevant information retrieved from the database.

[0991] Step 5: The server generates a response based on the information it has obtained.

[0992] Input: Related information retrieved from the database.

[0993] Operation: Generates an answer that includes the following items:

[0994] Text generation: Creates appropriate response sentences based on user questions.

[0995] Additional information attached: Add video links and detailed instructions as needed.

[0996] Response format: Converts the generated response into a format that can be appropriately displayed to the user.

[0997] Output: Formatted answers and additional information.

[0998] Step 6: The server sends the generated response to the user's terminal.

[0999] Input: Formatted response.

[1000] Operation: The server uses the HTTP response to send the generated response to the user's terminal.

[1001] Output: Response data received by the terminal.

[1002] Step 7: The terminal displays the received response to the user.

[1003] Input: Response data received by the terminal.

[1004] Operation: The device decodes the received data and displays it in the chat box in a user-friendly format. Interactive elements such as hyperlinks and video links are also displayed as needed.

[1005] Output: The response displayed on the user's screen.

[1006] This system allows users to receive detailed, multilingual support information in real time. Businesses can reduce operational costs while improving customer satisfaction.

[1007] (Application Example 1)

[1008] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1009] Currently, many e-commerce sites dedicate significant resources to user support, and there is a demand for fast and efficient support. However, traditional systems have limitations in improving customer satisfaction and reducing operational costs due to language differences and the need for manual responses. In particular, there is a challenge in providing detailed information in real time when responding to requests such as order cancellations and product returns.

[1010] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1011] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, and means for utilizing a natural language processing engine to generate a response to the user. This enables the user to receive support through a multilingual chat box and to be provided with comprehensive and personalized information in real time.

[1012] "Means of receiving information entered by users" refers to methods of collecting questions and requests entered by users through websites and applications.

[1013] "Means for analyzing the input information" refers to means of analyzing the collected user questions and requests using a natural language processing engine to identify keywords and intents.

[1014] "Means for searching for relevant information based on the analysis results" refers to means for searching a database for relevant FAQs and support documents based on keywords and intents obtained through the analysis.

[1015] "Means of providing the searched information to the user" refers to means of formatting the support information obtained through the search and displaying it to the user through a chat box or other interface.

[1016] "Means for using a natural language processing engine to generate a response to the user" refers to means for analyzing input from the user and automatically generating a response in a natural conversational format based on the analysis results.

[1017] "Means of supporting multiple languages" refers to means that can translate and convert user input information, analysis results, and responses into multiple languages.

[1018] "Methods for providing information in real time" refer to means of providing information quickly by immediately performing analysis, information retrieval, and response generation in response to user requests.

[1019] "Means including video links" refers to means that include a URL link to a video related to the acquired support information, enabling the user to view specific instructions.

[1020] "A means of generating individual responses based on user requests using a generative AI model" refers to a method of generating personalized responses to different user requests by utilizing artificial intelligence.

[1021] This invention is a smartphone application system for e-commerce sites that analyzes user support requests in real time and provides appropriate information. This system consists of the following main elements:

[1022] 1. User Interface

[1023] Users can enter questions or requests into a chat box within the application. For example, they might type, "I don't know how to cancel my order." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[1024] 2. Server-side processing

[1025] The server first receives information sent by the user. The received information is parsed using a natural language processing engine (e.g., SpaCy, Google Natural Language API). Through this parsing, keywords such as "cancel" and "order" are extracted. Based on this, the server searches for relevant information in a database (e.g., MySQL, PostgreSQL).

[1026] 3. Database Search

[1027] Based on the analysis results, the server searches the database for relevant FAQs and support documents. For example, guides and procedures related to "order cancellation" are retrieved from the database.

[1028] 4. Generating a response

[1029] The server generates specific responses for the user based on the information it has obtained. These responses may include video links or detailed instructions as needed. For example, a response might be generated stating, "The procedure for canceling your order is as follows. Please refer to this link for details: example.com / cancelorder." This process also utilizes a generative AI model to generate personalized responses.

[1030] 5. Sending and displaying responses

[1031] The server sends the generated response to the user's device. The user's smartphone application decodes the received response and displays it in the chat box. This allows the user to obtain relevant information in real time.

[1032] Hardware and software configuration

[1033] hardware

[1034] Server (requires high-performance computing power)

[1035] User's smartphone

[1036] software

[1037] Natural language processing engines (e.g., SpaCy, Google Natural Language API)

[1038] Databases (e.g., MySQL, PostgreSQL)

[1039] Web application frameworks (e.g., Flask, Django)

[1040] Generative AI models (e.g., ChatGPT)

[1041] Specific example

[1042] Example 1: For Japanese users

[1043] 1. User: "I'm having trouble with my internet connection. What should I do?"

[1044] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[1045] 3. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link: example.com / internetissue"

[1046] 4. The terminal displays a response to the user.

[1047] Example of a prompt

[1048] "I don't know how to return an item. What should I do?"

[1049] In this way, users can receive support through a multilingual chat box, and comprehensive and personalized information can be provided in real time. This system allows e-commerce sites to improve operational efficiency and enhance customer satisfaction.

[1050] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1051] Step 1: The user enters a question in the chat box.

[1052] Users enter questions or requests into a chat box within the e-commerce site's smartphone application. For example, they might type, "I don't know how to cancel my order."

[1053] Input: User's question or request

[1054] Output: Data sent to the server containing user input.

[1055] Step 2: The server receives user input.

[1056] The server receives information sent by the user. The received information is then analyzed using a natural language processing engine.

[1057] Input: User input

[1058] Output: Data ready for analysis

[1059] Step 3: Analyze the text using a natural language processing engine.

[1060] The server analyzes the received information using a natural language processing engine. During this process, keywords such as "cancel" and "order" are extracted.

[1061] Input: Received user input

[1062] Output: Extracted keyword and intent data

[1063] Step 4: Search for relevant information in the database

[1064] Based on the analysis results, the server searches the database for relevant FAQs and support documents. Specifically, it retrieves information related to "order cancellation."

[1065] Input: Extracted keywords and intent

[1066] Output: Relevant support information (FAQs and procedure manuals)

[1067] Step 5: Generate a response

[1068] The server generates specific responses for the user based on the information it has acquired. These responses may include video links or detailed instructions as needed.

[1069] Input: Related support information

[1070] Output: Final response data to be provided to the user

[1071] Step 6: Send the response to the user's device.

[1072] The server sends the generated response to the user's smartphone. The user's device decodes the received response and displays it in the chat box.

[1073] Input: Final response data

[1074] Output: Decoded data for display

[1075] Step 7: The user views the response.

[1076] Users can view the responses displayed in the chat box and find the necessary support information and procedures. This allows users to resolve issues in real time.

[1077] Input: Decoded data

[1078] Output: User understanding and behavior

[1079] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1080] The system of the present invention includes an interface in which users enter support requests through a website or application, a server for receiving and analyzing requests, a database for providing relevant information based on the analysis results, and an emotion engine for recognizing the user's emotions.

[1081] Explanation of program processing

[1082] 1. Interface

[1083] Users type messages into a chat box on a website or application, such as, "My order has been cancelled; please help me with this." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[1084] 2. Server-side processing

[1085] The server receives information sent by the user and first analyzes it using a natural language processing (NLP) engine. The NLP engine tokenizes the text and extracts important keywords and phrases such as "cancel" and "order." At the same time, an emotion engine detects emotions from the user's input and assigns emotion labels such as "anger" or "confusion."

[1086] 3. Database Search

[1087] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. For example, it retrieves guides and FAQs related to "order cancellation," and additional support information corresponding to the sentiment as needed.

[1088] 4. Generating and providing responses

[1089] The server generates a response to the user based on the information it has gathered. The tone and content of the response are adjusted based on the emotions recognized by the emotion engine. For example, a response might be generated such as, "We apologize for the inconvenience. Please refer to the following steps for instructions on how to cancel your order." Detailed instructions and video links may also be included.

[1090] 5. Sending and displaying responses

[1091] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1092] As a concrete example, consider the following scenario.

[1093] Example 1: For Japanese users

[1094] 1. User: "My delivery is delayed and I'm having trouble with it. Is there anything you can do?"

[1095] 2. The server uses an NLP engine to extract keywords such as "delivery delay" and an emotion engine to detect the emotion of "confusion."

[1096] 3. Generate a response that includes a guide regarding "delivery delays" and an apology based on "confusion."

[1097] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[1098] 5. The terminal displays a response to the user.

[1099] Example 2: For English users

[1100] 1. User: "I can't log into my account and I'm getting frustrated."

[1101] 2. The server uses an NLP engine to extract the keyword "cannot log in" and an emotion engine to detect the emotion "frustration".

[1102] 3. Generate a response that includes a guide regarding "login issues" and encouragement based on "frustration."

[1103] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[1104] 5. The terminal displays a response to the user.

[1105] The system of this invention allows users to receive appropriate and detailed support in real time, enabling companies to reduce operational costs while improving customer satisfaction. By utilizing an emotion engine, responses can be tailored to the user's emotions, resulting in more personalized support.

[1106] The following describes the processing flow.

[1107] Step 1:

[1108] Users enter questions or problems into the chat box of a website or application. For example, they might type, "My order has been cancelled, please help me with this."

[1109] Step 2:

[1110] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[1111] Step 3:

[1112] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[1113] Step 4:

[1114] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" are extracted. At the same time, an emotion engine detects the user's emotions from the text and assigns emotion labels such as "anger" or "confusion."

[1115] Step 5:

[1116] The server uses the extracted keywords and sentiment labels to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[1117] Step 6:

[1118] The server organizes search results and generates a message to respond to the user. Based on the emotions recognized by the sentiment engine, the tone and content of the response are adjusted. For example, a message such as, "We apologize for the inconvenience. Please refer to the following steps for how to cancel your order," might be generated. It may also include video links or detailed instructions.

[1119] Step 7:

[1120] The server sends the generated response to the terminal. The generated response is encoded as an HTTP response and sent.

[1121] Step 8:

[1122] The terminal decodes the HTTP response received from the server and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1123] Specific example

[1124] Example 1: For Japanese users

[1125] 1. User: "My delivery is delayed and I'm having trouble with it. Is there anything you can do?"

[1126] 2. The server uses an NLP engine to extract keywords such as "delivery delay" and an emotion engine to detect the emotion of "confusion."

[1127] 3. Generate a response that includes a guide regarding "delivery delays" and an apology based on "confusion."

[1128] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[1129] 5. The terminal displays a response to the user.

[1130] Example 2: For English users

[1131] 1. User: "I can't log into my account and I'm getting frustrated."

[1132] 2. The server uses an NLP engine to extract the keyword "cannot log in" and an emotion engine to detect the emotion "frustration".

[1133] 3. Generate a response that includes a guide regarding "login issues" and encouragement based on "frustration."

[1134] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[1135] 5. The terminal displays a response to the user.

[1136] The above is a specific embodiment of a system that combines an emotion engine to recognize user emotions. This system allows users to receive appropriate and detailed support in real time, enabling companies to reduce operational costs while improving customer satisfaction. The use of the emotion engine allows for responses tailored to the user's emotions, resulting in more personalized support.

[1137] (Example 2)

[1138] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1139] Traditional support systems struggled to adequately address users' problems and emotions, highlighting the need for improved user experience. Furthermore, insufficient multilingual support often led to delays in assisting users who spoke different languages. This resulted in increased operational costs and decreased customer satisfaction. Additionally, users had difficulty obtaining necessary information quickly, hindering the provision of appropriate support.

[1140] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1141] In this invention, the server includes means for analyzing information input by a user using language analysis means, means for detecting emotions and assigning emotion labels based on the analysis, and means for generating responses with tone and content appropriate to the emotions. This enables rapid and appropriate analysis of emotions based on user input and personalized responses based on the results. Furthermore, it enables the provision of information in multiple languages ​​and allows for rapid response to users who speak different languages, thereby reducing operational costs for companies and improving customer satisfaction.

[1142] "User-generated information" refers to text and data that users enter into a system to provide information such as support requests through a website or application.

[1143] "Language analysis means" refers to technologies and devices that analyze information input by users, understand its meaning, and extract key keywords. Specifically, natural language processing technology is included in this category.

[1144] "Means for detecting emotions and assigning emotion labels" refers to technologies and devices that analyze emotions from user input information and assign corresponding emotion tags. For example, this includes processes for classifying and tagging emotions such as anger, confusion, and joy.

[1145] "Means for searching for relevant information" refers to technologies and devices that search for and retrieve relevant information from databases and other information repositories based on the analyzed user input information.

[1146] "Means of generating responses" refers to technologies and devices that automatically generate appropriate responses and guidelines for users based on the searched relevant information and sentiment labels.

[1147] "Means of making information multilingual" refers to technologies and devices that translate user input information and generated responses into different languages, making them usable in a multilingual environment.

[1148] "Means including viewing links" refers to technologies and devices that include links to videos or tutorials in the information provided to users. This makes it easier for users to receive support in a visual way.

[1149] The system of this invention provides rapid and personalized support by analyzing support requests from users and generating appropriate responses. This system primarily consists of four parts: an interface, a server, a database, and an emotion engine. The following describes each component and its operation in detail.

[1150] interface

[1151] Users enter support requests into chat boxes on websites or applications. For example, a request might say, "My order has been cancelled; please help me with this." This interface receives the user's input, automatically detects their language settings, and sends the information to the server in the appropriate format.

[1152] server

[1153] The server receives information transmitted through the interface. This received information is first analyzed by a natural language processing (NLP) engine. This NLP engine uses services such as Google Cloud Natural Language API and Microsoft Azure Text Analytics to tokenize the text and extract important keywords and phrases. Next, a sentiment engine is used to detect emotions from the user's input text and assign sentiment labels such as "anger" or "confusion." IBM Watson Tone Analyzer is used for this sentiment analysis.

[1154] database

[1155] The server searches the database for relevant support information based on the analysis results and sentiment labels. The database is managed by a database management system such as MySQL or MongoDB. For example, guides and FAQs related to "order cancellation" are retrieved, as well as additional support information corresponding to the sentiment as needed.

[1156] Response generation and provision

[1157] The server generates a response to the user based on the information it has acquired. The tone and content of the response are adjusted based on the emotions recognized by the emotion engine. For example, in response to user confusion, a response such as, "We apologize for the inconvenience. Please refer to the following steps for instructions on how to cancel your order," might be generated. Detailed instructions and video links may also be included. This response can also be generated in multiple languages ​​based on the user's language settings.

[1158] Sending and displaying responses

[1159] The generated response is sent from the server to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1160] Specific example

[1161] Example 1: For Japanese users

[1162] User input: "My delivery is delayed and I'm having trouble. Can you do something about it?"

[1163] NLP engine analysis: Extraction of the keyword "delivery delay"

[1164] Emotional Engine Analysis: Detecting the emotion of "confusion"

[1165] Database Search: Retrieve guides and apology letters regarding "delivery delays" and "confusion."

[1166] Generated response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[1167] Display to the user: The device displays the response in the chat box.

[1168] Example 2: For English users

[1169] User input: "I can't log into my account and I'm getting frustrated."

[1170] NLP engine analysis: Extracting the keyword "cannot log in".

[1171] Emotional Engine Analysis: Detecting the emotion of "frustration"

[1172] Database Search: Get guides and encouragement based on "login issues" and "frustration."

[1173] Generated response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[1174] Display to the user: The device displays the response in the chat box.

[1175] This invention's system allows users to receive fast, personalized support in real time. In particular, its multilingual support and emotion recognition enable it to appropriately respond to users with different languages ​​and emotions. This allows companies to reduce operational costs and improve customer satisfaction.

[1176] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1177] Step 1:

[1178] Users enter support requests into a chat box on the website or application. For example, they might type, "My order was cancelled, please help me." This user input is the starting point for the system.

[1179] Input: Text of the support request entered by the user

[1180] Output: User input text is passed to the interface.

[1181] Step 2:

[1182] The interface receives user input text and automatically detects its language settings. This detection uses information such as web browser settings and account information. It then converts the input text to the appropriate format and sends it to the server.

[1183] Input: User input text

[1184] Data processing: Language setting detection and text formatting conversion

[1185] Output: Formatted text is sent to the server.

[1186] Step 3:

[1187] The server receives text data transmitted through the interface. The received text is sent to a natural language processing (NLP) engine for text analysis. The NLP engine tokenizes the text and extracts key keywords and phrases.

[1188] Input: Formatted text

[1189] Data processing: Text analysis and tokenization using a natural language processing engine.

[1190] Output: List of extracted keywords and phrases

[1191] Step 4:

[1192] The server sends the extracted keywords and phrases to the sentiment engine. The sentiment engine detects emotions from the text and assigns emotion labels such as "anger" or "confusion."

[1193] Input: List of extracted keywords or phrases

[1194] Data processing: Emotion analysis and emotion labeling using an emotion engine.

[1195] Output: Keywords and phrases with emotion labels

[1196] Step 5:

[1197] The server searches the database for relevant support information based on the analysis results and sentiment labels. The server constructs a search query and retrieves the relevant information using a database management system (e.g., MySQL or MongoDB).

[1198] Input: Keywords or phrases with emotion labels

[1199] Data Processing: Building database search queries and retrieving information.

[1200] Output: Related support information retrieved as search results

[1201] Step 6:

[1202] The server generates a response based on the acquired support information. It adjusts the response to the user with an appropriate tone and content, taking into account the emotion label detected by the emotion engine. For example, if the emotion "confused" is detected, it will include an apology such as, "We are sorry to trouble you."

[1203] Input: Related support information retrieved as search results

[1204] Data processing: Response text generation and tone adjustment

[1205] Output: Generated response

[1206] Step 7:

[1207] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1208] Input: Generated response

[1209] Data processing: Decoding and formatting of response statements

[1210] Output: Response displayed in the user's chat box

[1211] The above outlines the specific processing flow of this system. Effective and personalized support is provided through a consistent process from user input to the display of responses.

[1212] (Application Example 2)

[1213] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1214] Conventional customer support systems have struggled to recognize user emotions and adjust the tone and content of responses accordingly, resulting in a decline in the quality of the user experience. Furthermore, simultaneously implementing multilingual support and emotion recognition capabilities is difficult, making improving customer satisfaction a particular challenge in global service provision. This invention aims to solve these problems and provide effective customer support that is attentive to the user's emotions.

[1215] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for recognizing the user's emotions based on the analyzed information, means for adjusting the tone and content of the response according to the emotion recognition results, means for providing the retrieved information to the user, means for supporting multiple languages, means for providing information to the user including multilingual responses, means for generating multilingual responses adjusted based on emotion recognition, means for including video links in the information provided to the user, means for the user to view support information based on the video links, and further means for selecting an appropriate video link based on the emotion recognition. This makes it possible to provide responses adapted to the user's emotions in real time and achieve high customer satisfaction even in a global context.

[1216] "Means of receiving information entered by users" refers to the function that allows the server to receive information entered by users through chat boxes or forms.

[1217] "Means for analyzing the input information" refers to a function that allows the server to analyze the user's input information received using natural language processing (NLP) or other analysis techniques to extract important keywords and phrases.

[1218] "Means for searching for relevant information based on the analysis results" refers to a function for searching for relevant information from a database using keywords or phrases obtained through the analysis.

[1219] "Means for providing the searched information to the user" refers to means for visually displaying the search results to the user, and includes websites, applications, chat boxes, etc.

[1220] "Means for recognizing the user's emotions based on the analyzed information" refers to a function that analyzes the user's emotions using an emotion engine based on the user's input information and assigns emotion labels such as "anger" or "confusion."

[1221] "Means for adjusting the tone and content of the response in accordance with the emotion recognition result" refers to a function for optimizing and adjusting the tone and content of the response presented to the user based on the emotion recognized by the emotion engine.

[1222] "Means of supporting multiple languages" refers to support functions that allow user input information and responses to be handled in multiple languages.

[1223] "Means of providing information to users, including multilingual responses" refers to a function that provides users with responses generated in multiple languages ​​based on analysis results, and displays them in an appropriate format according to the language the user is using.

[1224] "Means for generating multilingual responses based on the aforementioned emotion recognition" refers to a function for generating multilingual and emotion-appropriate response content based on the user's emotion recognition results.

[1225] "Means of including video links in the information provided to the user" refers to a function that provides the user with a link to watch a video as related support information.

[1226] "Means by which users can view support information based on the aforementioned video link" refers to a function that enables users to view support information through the provided video link.

[1227] "Means for selecting an appropriate video link based on the aforementioned emotion recognition" refers to a function for selecting and providing the most suitable video link for the user based on the emotion recognition results.

[1228] This invention relates to a system that analyzes support requests entered by users through websites and applications and provides relevant information. Specifically, it is characterized by analyzing the user's input information and emotions using a natural language processing (NLP) engine and an emotion engine, and generating an appropriate response based on that analysis.

[1229] System Configuration

[1230] This system consists of the following main components:

[1231] 1. User input receiving means:

[1232] This is a means of receiving information entered by users. Information is received through interfaces such as web forms, chat boxes, and mobile apps.

[1233] 2. Input information analysis means:

[1234] This is a method for analyzing the user's input information received. Specifically, it uses a natural language processing (NLP) engine (e.g., SpaCy, BERT) to extract important keywords and phrases from the input text.

[1235] 3. Emotion recognition means:

[1236] This is a method for recognizing user emotions based on NLP analysis results. It uses an emotion engine (e.g., IBM Watson Tone Analyzer) to detect emotions such as "anger" or "confusion" from user input.

[1237] 4. Related information retrieval methods:

[1238] This is a means of searching for relevant support information from a database based on the analysis results and sentiment recognition results. The support information is stored in a database such as Amazon RDS.

[1239] 5. Response generation means:

[1240] This is a means of generating appropriate responses for the user based on searched support information and sentiment recognition results. The generated responses are then adjusted in tone and content.

[1241] 6. Information provision method:

[1242] This is a means of providing the generated response to the user. The response is displayed through interfaces such as websites and mobile applications. The response may also include multilingual support.

[1243] 7. Method of providing video links:

[1244] This method includes including video links as support information, where necessary. This allows users to watch the videos and receive support to resolve their problems.

[1245] 8. Methods for selecting appropriate video links:

[1246] This is a means of selecting and providing appropriate video links to users based on emotion recognition results.

[1247] Hardware and software to be used

[1248] Server: An AWS (Amazon Web Services) EC2 instance is used to receive, analyze, and process user input.

[1249] Database: Support information is stored using Amazon RDS (Relational Database Service).

[1250] NLP engine: Uses SpaCy or BERT for natural language processing.

[1251] Emotion Engine: Uses IBM Watson Tone Analyzer to recognize user emotions.

[1252] Frontend: Developing smartphone applications using React Native.

[1253] Specific example

[1254] The following is a specific example of how this system works.

[1255] Specific example 1: For Japanese users

[1256] The user types in the chat box, "My ordered item hasn't arrived yet, please check on it."

[1257] The server receives user input and uses an NLP engine to extract the keyword "product not delivered".

[1258] The emotion engine recognizes the emotion of "confusion."

[1259] The system searches the database for "information regarding delivery delays" and generates an apology message to address the user's feelings of distress.

[1260] The system provides the user with the most appropriate response, which includes a link such as "Check your current delivery status here."

[1261] Example of a prompt

[1262] User: "My ordered item hasn't arrived yet, please check on it."

[1263] server:

[1264] Keyword extraction: ["product", "not delivered"]

[1265] Emotional label: ["Confused"]

[1266] Response generated: "We apologize that your order has not yet arrived. Please check the current delivery status here. [link]."

[1267] As described above, this system can analyze user input information and emotions, and provide optimal support information in real time.

[1268] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1269] Step 1:

[1270] The user enters a support request into the chat box. Specifically, they might enter text such as, "My ordered item hasn't arrived yet, please check on it." The device receives this input and sends it to the server.

[1271] Input: User input text

[1272] Output: Input text sent to the server

[1273] Step 2:

[1274] The server analyzes the received input text using a natural language processing (NLP) engine. Specifically, it tokenizes the text and extracts important keywords and phrases. For example, "product" and "not delivered" might be extracted.

[1275] Input: Input text

[1276] Output: Extracted keywords and phrases

[1277] Step 3:

[1278] The server uses an emotion engine to recognize the user's emotions from the input text. Specifically, it analyzes the text and assigns emotion labels such as "confused" or "angry." In this step, the emotion "confused" is detected.

[1279] Input: Input text

[1280] Output: Emotional label (e.g., "Confused")

[1281] Step 4:

[1282] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. Specifically, it retrieves FAQs and guides related to the keyword "product not received." It also retrieves apology letters addressing the sentiment "confusion."

[1283] Input: Keywords, sentiment labels

[1284] Output: Related support information

[1285] Step 5:

[1286] The server generates an appropriate response based on the acquired support information and sentiment labels. Specifically, it combines relevant information to generate a response message for the user, adjusting the content and tone of the response. For example, it might generate a message like, "We apologize that your order has not yet arrived. You can check the current delivery status here."

[1287] Input: Relevant support information, sentiment labels

[1288] Output: Adjusted response

[1289] Step 6:

[1290] The server sends the generated response to the terminal. The terminal displays the received response to the user, for example, in a chat box.

[1291] Input: Adjusted response

[1292] Output: Response displayed to the user

[1293] With the above steps completed, the process of analyzing the user's input information and emotions, and providing optimal support information in real time, is finished.

[1294] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1295] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1296] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1297] [Fourth Embodiment]

[1298] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1299] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1300] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1301] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1302] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1303] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1304] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1305] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1306] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1307] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1308] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1309] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1310] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1311] The system of the present invention includes an interface in which users can submit support requests through a website or application, a server for receiving and analyzing requests, and a database for providing relevant information based on the analysis results.

[1312] Explanation of program processing

[1313] 1. Interface

[1314] Users type, for example, "I don't know how to cancel my order," into a chat box on a website or application. This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[1315] 2. Server-side processing

[1316] The server receives information sent by the user. The received information is first analyzed using a natural language processing (NLP) engine. Through this analysis, keywords such as "cancel" and "order" are extracted. Based on this, the server searches the database for relevant information to address specific scenarios and FAQs.

[1317] 3. Database Search

[1318] The server searches for relevant FAQs and support documents based on the analysis results. For example, guides and procedures related to "order cancellation" are retrieved from the database. The retrieved information is then formatted as an appropriate response.

[1319] 4. Generating and providing responses

[1320] The server generates a response for the user based on the information it has obtained. The response may include video links or detailed instructions as needed. For example, it may include specific instructions and links such as, "To cancel your order, please follow these steps. For details, please refer to this video link."

[1321] 5. Sending and displaying responses

[1322] The server sends the generated response to the user's terminal. The terminal decodes the received response and displays it to the user in the chat box. This allows the user to obtain relevant information in real time.

[1323] As a concrete example, consider the following scenario.

[1324] Example 1: For Japanese users

[1325] 1. User: "I'm having trouble with my internet connection. What should I do?"

[1326] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[1327] 3. As a solution to the "Internet connection problem," generate a response that includes a video link and instructions.

[1328] 4. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link (example.com / internetissue)."

[1329] 5. The terminal displays a response to the user.

[1330] Example 2: For English users

[1331] 1. User: "How can I reset my password?"

[1332] 2. The server analyzes the keyword "reset password" and searches the database for relevant support information.

[1333] 3. As a solution for "password reset," generate a response that includes instructions.

[1334] 4. Response: "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[1335] 5. The terminal displays a response to the user.

[1336] The system of this invention allows users to receive appropriate and detailed support in real time, and enables companies to reduce operational costs while improving customer satisfaction.

[1337] The following describes the processing flow.

[1338] Step 1:

[1339] Users enter questions or problems into a chat box on a website or application. For example, they might type, "How do I cancel my order?"

[1340] Step 2:

[1341] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[1342] Step 3:

[1343] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[1344] Step 4:

[1345] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" might be extracted.

[1346] Step 5:

[1347] The server uses the extracted keywords to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[1348] Step 6:

[1349] The server organizes the search results and generates a message to respond to the user. At this stage, if multilingual support is required, the message is translated into the appropriate language. For example, a response such as "To cancel your order, please select 'Cancel' from 'Order History' on My Page" is generated.

[1350] Step 7:

[1351] The server sends the generated response to the terminal. The generated response is encoded and sent as an HTTP response.

[1352] Step 8:

[1353] The device decodes the HTTP response received from the server and displays a response message to the user. The answer is displayed in the chat box, allowing the user to get solutions to their questions and problems in real time. For example, it might display, "To cancel your order, please select 'Cancel' from 'Order History' on My Page."

[1354] This series of steps allows users to receive quick and appropriate support, and enables companies to provide efficient customer support.

[1355] (Example 1)

[1356] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1357] Traditional support systems faced challenges in responding quickly and accurately to user inquiries. In particular, efficient answers were often lacking when multilingual support or detailed instructions were required. Furthermore, the lack of adequate systems for providing users with the information they needed in an appropriate format resulted in a poor user experience.

[1358] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1359] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, means for automatically formatting the input information, means for transmitting the formatted information to the server, means for extracting keywords using a natural language processing engine based on the analysis results, means for generating a response based on the acquired information, means for transmitting the generated response to the user's terminal, and means for displaying the transmitted response to the user. This enables the user to receive detailed multilingual support information in real time.

[1360] A "user" is an individual or organization that enters a support request through the system and obtains information.

[1361] "Means of receiving information" refers to the interface and protocol that allows a system to receive information entered by a user.

[1362] "Means of analyzing information" refers to natural language processing engines and other analytical tools used to process received information and understand its content.

[1363] "Means of searching for relevant information" refers to the process of finding the necessary information from appropriate databases or resources based on the results of the analysis.

[1364] "Means of providing information" refers to technologies for formatting retrieved information and displaying or transmitting it appropriately to the user.

[1365] "Means for automatically formatting information" refer to protocols and algorithms that convert user-inputted information into a format that the system can easily understand.

[1366] "Means for sending formatted information to a server" refers to communication means for transferring appropriately formatted data to a server.

[1367] A "natural language processing engine" is a program that analyzes human language, understands its meaning, and extracts keywords.

[1368] "Means of generating responses" refer to the logic and software used to create appropriate answers to user inquiries based on extracted keywords and search results.

[1369] "Means for sending a response to the user's terminal" refers to the communication protocol and system for sending the generated response to the user's device.

[1370] "Means for displaying responses" refers to interface technology that allows users to visually confirm the responses they have received on their devices.

[1371] This invention relates to a system in which a user submits a support request through a website or application, the system analyzes the request, and provides an appropriate response. The main components of this system include an interface for receiving user input, a server for analyzing the input information, a database for retrieving relevant information based on the analysis results, and means for providing a response to the user.

[1372] 1. Interface

[1373] Users use the chat box on the website or application to enter inquiries, such as "I don't know how to cancel my order." This interface has a feature that automatically detects the user's language settings and converts the entered information into the appropriate format.

[1374] 2. Sending data

[1375] The terminal sends information entered by the user to the server. This process uses protocols such as HTTP requests to transmit data securely and reliably. Data integrity and security are ensured during this process.

[1376] 3. Analysis of Information

[1377] The server analyzes the received information using a natural language processing (NLP) engine. The NLP engine tokenizes the text, performs part-of-speech analysis, and extracts important keywords. For example, keywords such as "cancel" and "order" are extracted.

[1378] 4. Searching for related information

[1379] The server searches the database for relevant information based on the extracted keywords. The database contains FAQs, guides, and instruction manuals. The search results are retrieved as structured data.

[1380] 5. Response generation

[1381] The server generates a response for the user based on the information it has obtained. This response may include detailed instructions or video links. For example, it might say, "To cancel your order, please follow these steps. For more details, please refer to this video link."

[1382] 6. Sending a response

[1383] The server sends the generated response to the terminal. This process also uses protocols such as HTTP responses to ensure secure and reliable data transmission.

[1384] 7. Display of response

[1385] The terminal decodes the received response and displays it to the user in a chat box. The display is formatted for user readability, and interactive elements such as hyperlinks are provided as needed.

[1386] Specific example

[1387] For example, consider a case where a Japanese-speaking user types, "I'm having trouble with my internet connection. What should I do?" This information is sent to the server and securely analyzed. The server extracts the keyword "internet connection" and searches its database for relevant support information. As a result, a response is generated that includes a video link and instructions, and the user is provided with a response such as, "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link."

[1388] For English-speaking users, typing "How can I reset my password?" will trigger a similar analysis process, extracting the keyword "reset password." The server will then search for relevant password reset information and generate a response containing instructions. The user will receive a response such as, "Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword."

[1389] Examples of prompts for generative AI models

[1390] "I'm having trouble with my internet connection. What should I do?"

[1391] "I need help with resetting my password."

[1392] This system allows users to receive detailed, multilingual support information in real time, while enabling businesses to reduce operational costs and improve customer satisfaction.

[1393] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1394] Step 1: The user enters their question into the chat box of the website or application.

[1395] Input: The user enters the question in text format (e.g., "I don't know how to cancel my order").

[1396] Operation: The interface receives user input and automatically detects language settings.

[1397] Output: Formatted text information.

[1398] Step 2: The terminal sends user input to the server.

[1399] Input: Formatted text information generated by the interface.

[1400] Operation: The terminal uses an HTTP request to send formatted text information to the server.

[1401] Output: Text information received by the server.

[1402] Step 3: The server analyzes the received information using a natural language processing (NLP) engine.

[1403] Input: Text information received by the server.

[1404] Operation: The NLP engine performs the following processes:

[1405] Tokenization: Dividing text into words or phrases.

[1406] Part-of-speech analysis: Identifies the part of speech of each token.

[1407] Keyword extraction: Extraction of important keywords such as "cancel" and "order".

[1408] Output: Extracted keywords and analyzed text information.

[1409] Step 4: The server searches the database for relevant information based on the extracted keywords.

[1410] Input: Extracted keywords.

[1411] Operation: The process is carried out using the following steps:

[1412] Query generation: Generates search queries based on the extracted keywords.

[1413] Database query: Use the generated query to search the database for relevant FAQs and support documents.

[1414] Retrieving search results: Relevant information is retrieved from multiple databases.

[1415] Output: Relevant information retrieved from the database.

[1416] Step 5: The server generates a response based on the information it has obtained.

[1417] Input: Related information retrieved from the database.

[1418] Operation: Generates an answer that includes the following items:

[1419] Text generation: Creates appropriate response sentences based on user questions.

[1420] Additional information attached: Add video links and detailed instructions as needed.

[1421] Response format: Converts the generated response into a format that can be appropriately displayed to the user.

[1422] Output: Formatted answers and additional information.

[1423] Step 6: The server sends the generated response to the user's terminal.

[1424] Input: Formatted response.

[1425] Operation: The server uses the HTTP response to send the generated response to the user's terminal.

[1426] Output: Response data received by the terminal.

[1427] Step 7: The terminal displays the received response to the user.

[1428] Input: Response data received by the terminal.

[1429] Operation: The device decodes the received data and displays it in the chat box in a user-friendly format. Interactive elements such as hyperlinks and video links are also displayed as needed.

[1430] Output: The response displayed on the user's screen.

[1431] This system allows users to receive detailed, multilingual support information in real time. Businesses can reduce operational costs while improving customer satisfaction.

[1432] (Application Example 1)

[1433] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1434] Currently, many e-commerce sites dedicate significant resources to user support, and there is a demand for fast and efficient support. However, traditional systems have limitations in improving customer satisfaction and reducing operational costs due to language differences and the need for manual responses. In particular, there is a challenge in providing detailed information in real time when responding to requests such as order cancellations and product returns.

[1435] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1436] In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for providing the searched information to the user, and means for utilizing a natural language processing engine to generate a response to the user. This enables the user to receive support through a multilingual chat box and to be provided with comprehensive and personalized information in real time.

[1437] "Means of receiving information entered by users" refers to methods of collecting questions and requests entered by users through websites and applications.

[1438] "Means for analyzing the input information" refers to means of analyzing the collected user questions and requests using a natural language processing engine to identify keywords and intents.

[1439] "Means for searching for relevant information based on the analysis results" refers to means for searching a database for relevant FAQs and support documents based on keywords and intents obtained through the analysis.

[1440] "Means of providing the searched information to the user" refers to means of formatting the support information obtained through the search and displaying it to the user through a chat box or other interface.

[1441] "Means for using a natural language processing engine to generate a response to the user" refers to means for analyzing input from the user and automatically generating a response in a natural conversational format based on the analysis results.

[1442] "Means of supporting multiple languages" refers to means that can translate and convert user input information, analysis results, and responses into multiple languages.

[1443] "Methods for providing information in real time" refer to means of providing information quickly by immediately performing analysis, information retrieval, and response generation in response to user requests.

[1444] "Means including video links" refers to means that include a URL link to a video related to the acquired support information, enabling the user to view specific instructions.

[1445] "A means of generating individual responses based on user requests using a generative AI model" refers to a method of generating personalized responses to different user requests by utilizing artificial intelligence.

[1446] This invention is a smartphone application system for e-commerce sites that analyzes user support requests in real time and provides appropriate information. This system consists of the following main elements:

[1447] 1. User Interface

[1448] Users can enter questions or requests into a chat box within the application. For example, they might type, "I don't know how to cancel my order." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[1449] 2. Server-side processing

[1450] The server first receives information sent by the user. The received information is parsed using a natural language processing engine (e.g., SpaCy, Google Natural Language API). Through this parsing, keywords such as "cancel" and "order" are extracted. Based on this, the server searches for relevant information in a database (e.g., MySQL, PostgreSQL).

[1451] 3. Database Search

[1452] Based on the analysis results, the server searches the database for relevant FAQs and support documents. For example, guides and procedures related to "order cancellation" are retrieved from the database.

[1453] 4. Generating a response

[1454] The server generates specific responses for the user based on the information it has obtained. These responses may include video links or detailed instructions as needed. For example, a response might be generated stating, "The procedure for canceling your order is as follows. Please refer to this link for details: example.com / cancelorder." This process also utilizes a generative AI model to generate personalized responses.

[1455] 5. Sending and displaying responses

[1456] The server sends the generated response to the user's device. The user's smartphone application decodes the received response and displays it in the chat box. This allows the user to obtain relevant information in real time.

[1457] Hardware and software configuration

[1458] hardware

[1459] Server (requires high-performance computing power)

[1460] User's smartphone

[1461] software

[1462] Natural language processing engines (e.g., SpaCy, Google Natural Language API)

[1463] Databases (e.g., MySQL, PostgreSQL)

[1464] Web application frameworks (e.g., Flask, Django)

[1465] Generative AI models (e.g., ChatGPT)

[1466] Specific example

[1467] Example 1: For Japanese users

[1468] 1. User: "I'm having trouble with my internet connection. What should I do?"

[1469] 2. The server analyzes the keyword "Internet connection" and searches the database for relevant support information.

[1470] 3. Response: "If you are having trouble with your internet connection, please follow these steps. For detailed instructions, please refer to this video link: example.com / internetissue"

[1471] 4. The terminal displays a response to the user.

[1472] Example of a prompt

[1473] "I don't know how to return an item. What should I do?"

[1474] In this way, users can receive support through a multilingual chat box, and comprehensive and personalized information can be provided in real time. This system allows e-commerce sites to improve operational efficiency and enhance customer satisfaction.

[1475] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1476] Step 1: The user enters a question in the chat box.

[1477] Users enter questions or requests into a chat box within the e-commerce site's smartphone application. For example, they might type, "I don't know how to cancel my order."

[1478] Input: User's question or request

[1479] Output: Data sent to the server containing user input.

[1480] Step 2: The server receives user input.

[1481] The server receives information sent by the user. The received information is then analyzed using a natural language processing engine.

[1482] Input: User input

[1483] Output: Data ready for analysis

[1484] Step 3: Analyze the text using a natural language processing engine.

[1485] The server analyzes the received information using a natural language processing engine. During this process, keywords such as "cancel" and "order" are extracted.

[1486] Input: Received user input

[1487] Output: Extracted keyword and intent data

[1488] Step 4: Search for relevant information in the database

[1489] Based on the analysis results, the server searches the database for relevant FAQs and support documents. Specifically, it retrieves information related to "order cancellation."

[1490] Input: Extracted keywords and intent

[1491] Output: Relevant support information (FAQs and procedure manuals)

[1492] Step 5: Generate a response

[1493] The server generates specific responses for the user based on the information it has acquired. These responses may include video links or detailed instructions as needed.

[1494] Input: Related support information

[1495] Output: Final response data to be provided to the user

[1496] Step 6: Send the response to the user's device.

[1497] The server sends the generated response to the user's smartphone. The user's device decodes the received response and displays it in the chat box.

[1498] Input: Final response data

[1499] Output: Decoded data for display

[1500] Step 7: The user views the response.

[1501] Users can view the responses displayed in the chat box and find the necessary support information and procedures. This allows users to resolve issues in real time.

[1502] Input: Decoded data

[1503] Output: User understanding and behavior

[1504] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1505] The system of the present invention includes an interface in which users enter support requests through a website or application, a server for receiving and analyzing requests, a database for providing relevant information based on the analysis results, and an emotion engine for recognizing the user's emotions.

[1506] Explanation of program processing

[1507] 1. Interface

[1508] Users type messages into a chat box on a website or application, such as, "My order has been cancelled; please help me with this." This interface automatically detects the user's language settings and sends the information to the server in the appropriate format.

[1509] 2. Server-side processing

[1510] The server receives information sent by the user and first analyzes it using a natural language processing (NLP) engine. The NLP engine tokenizes the text and extracts important keywords and phrases such as "cancel" and "order." At the same time, an emotion engine detects emotions from the user's input and assigns emotion labels such as "anger" or "confusion."

[1511] 3. Database Search

[1512] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. For example, it retrieves guides and FAQs related to "order cancellation," and additional support information corresponding to the sentiment as needed.

[1513] 4. Generating and providing responses

[1514] The server generates a response to the user based on the information it has gathered. The tone and content of the response are adjusted based on the emotions recognized by the emotion engine. For example, a response might be generated such as, "We apologize for the inconvenience. Please refer to the following steps for instructions on how to cancel your order." Detailed instructions and video links may also be included.

[1515] 5. Sending and displaying responses

[1516] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1517] As a concrete example, consider the following scenario.

[1518] Example 1: For Japanese users

[1519] 1. User: "My delivery is delayed and I'm having trouble with it. Is there anything you can do?"

[1520] 2. The server uses an NLP engine to extract keywords such as "delivery delay" and an emotion engine to detect the emotion of "confusion."

[1521] 3. Generate a response that includes a guide regarding "delivery delays" and an apology based on "confusion."

[1522] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[1523] 5. The terminal displays a response to the user.

[1524] Example 2: For English users

[1525] 1. User: "I can't log into my account and I'm getting frustrated."

[1526] 2. The server uses an NLP engine to extract the keyword "cannot log in" and an emotion engine to detect the emotion "frustration".

[1527] 3. Generate a response that includes a guide regarding "login issues" and encouragement based on "frustration."

[1528] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[1529] 5. The terminal displays a response to the user.

[1530] The system of this invention allows users to receive appropriate and detailed support in real time, enabling companies to reduce operational costs while improving customer satisfaction. By utilizing an emotion engine, responses can be tailored to the user's emotions, resulting in more personalized support.

[1531] The following describes the processing flow.

[1532] Step 1:

[1533] Users enter questions or problems into the chat box of a website or application. For example, they might type, "My order has been cancelled, please help me with this."

[1534] Step 2:

[1535] The terminal receives the entered information and sends it to the server. An HTTP request is used for transmission, and it includes necessary user identification and language information.

[1536] Step 3:

[1537] The server decodes the received HTTP request and extracts the text data entered by the user. This text data is then preprocessed to remove noise.

[1538] Step 4:

[1539] The server passes the text data to a natural language processing (NLP) engine for analysis. The NLP engine tokenizes the text and extracts important keywords and phrases. For example, keywords such as "cancel" and "order" are extracted. At the same time, an emotion engine detects the user's emotions from the text and assigns emotion labels such as "anger" or "confusion."

[1540] Step 5:

[1541] The server uses the extracted keywords and sentiment labels to search the database for relevant support information. This search retrieves appropriate information from sources such as FAQs, manuals, and support documents.

[1542] Step 6:

[1543] The server organizes search results and generates a message to respond to the user. Based on the emotions recognized by the sentiment engine, the tone and content of the response are adjusted. For example, a message such as, "We apologize for the inconvenience. Please refer to the following steps for how to cancel your order," might be generated. It may also include video links or detailed instructions.

[1544] Step 7:

[1545] The server sends the generated response to the terminal. The generated response is encoded as an HTTP response and sent.

[1546] Step 8:

[1547] The terminal decodes the HTTP response received from the server and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1548] Specific example

[1549] Example 1: For Japanese users

[1550] 1. User: "My delivery is delayed and I'm having trouble with it. Is there anything you can do?"

[1551] 2. The server uses an NLP engine to extract keywords such as "delivery delay" and an emotion engine to detect the emotion of "confusion."

[1552] 3. Generate a response that includes a guide regarding "delivery delays" and an apology based on "confusion."

[1553] 4. Response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[1554] 5. The terminal displays a response to the user.

[1555] Example 2: For English users

[1556] 1. User: "I can't log into my account and I'm getting frustrated."

[1557] 2. The server uses an NLP engine to extract the keyword "cannot log in" and an emotion engine to detect the emotion "frustration".

[1558] 3. Generate a response that includes a guide regarding "login issues" and encouragement based on "frustration."

[1559] 4. Response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[1560] 5. The terminal displays a response to the user.

[1561] The above is a specific embodiment of a system that combines an emotion engine to recognize user emotions. This system allows users to receive appropriate and detailed support in real time, enabling companies to reduce operational costs while improving customer satisfaction. The use of the emotion engine allows for responses tailored to the user's emotions, resulting in more personalized support.

[1562] (Example 2)

[1563] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1564] Traditional support systems struggled to adequately address users' problems and emotions, highlighting the need for improved user experience. Furthermore, insufficient multilingual support often led to delays in assisting users who spoke different languages. This resulted in increased operational costs and decreased customer satisfaction. Additionally, users had difficulty obtaining necessary information quickly, hindering the provision of appropriate support.

[1565] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1566] In this invention, the server includes means for analyzing information input by a user using language analysis means, means for detecting emotions and assigning emotion labels based on the analysis, and means for generating responses with tone and content appropriate to the emotions. This enables rapid and appropriate analysis of emotions based on user input and personalized responses based on the results. Furthermore, it enables the provision of information in multiple languages ​​and allows for rapid response to users who speak different languages, thereby reducing operational costs for companies and improving customer satisfaction.

[1567] "User-generated information" refers to text and data that users enter into a system to provide information such as support requests through a website or application.

[1568] "Language analysis means" refers to technologies and devices that analyze information input by users, understand its meaning, and extract key keywords. Specifically, natural language processing technology is included in this category.

[1569] "Means for detecting emotions and assigning emotion labels" refers to technologies and devices that analyze emotions from user input information and assign corresponding emotion tags. For example, this includes processes for classifying and tagging emotions such as anger, confusion, and joy.

[1570] "Means for searching for relevant information" refers to technologies and devices that search for and retrieve relevant information from databases and other information repositories based on the analyzed user input information.

[1571] "Means of generating responses" refers to technologies and devices that automatically generate appropriate responses and guidelines for users based on the searched relevant information and sentiment labels.

[1572] "Means of making information multilingual" refers to technologies and devices that translate user input information and generated responses into different languages, making them usable in a multilingual environment.

[1573] "Means including viewing links" refers to technologies and devices that include links to videos or tutorials in the information provided to users. This makes it easier for users to receive support in a visual way.

[1574] The system of this invention provides rapid and personalized support by analyzing support requests from users and generating appropriate responses. This system primarily consists of four parts: an interface, a server, a database, and an emotion engine. The following describes each component and its operation in detail.

[1575] interface

[1576] Users enter support requests into chat boxes on websites or applications. For example, a request might say, "My order has been cancelled; please help me with this." This interface receives the user's input, automatically detects their language settings, and sends the information to the server in the appropriate format.

[1577] server

[1578] The server receives information transmitted through the interface. This received information is first analyzed by a natural language processing (NLP) engine. This NLP engine uses services such as Google Cloud Natural Language API and Microsoft Azure Text Analytics to tokenize the text and extract important keywords and phrases. Next, a sentiment engine is used to detect emotions from the user's input text and assign sentiment labels such as "anger" or "confusion." IBM Watson Tone Analyzer is used for this sentiment analysis.

[1579] database

[1580] The server searches the database for relevant support information based on the analysis results and sentiment labels. The database is managed by a database management system such as MySQL or MongoDB. For example, guides and FAQs related to "order cancellation" are retrieved, as well as additional support information corresponding to the sentiment as needed.

[1581] Response generation and provision

[1582] The server generates a response to the user based on the information it has acquired. The tone and content of the response are adjusted based on the emotions recognized by the emotion engine. For example, in response to user confusion, a response such as, "We apologize for the inconvenience. Please refer to the following steps for instructions on how to cancel your order," might be generated. Detailed instructions and video links may also be included. This response can also be generated in multiple languages ​​based on the user's language settings.

[1583] Sending and displaying responses

[1584] The generated response is sent from the server to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1585] Specific example

[1586] Example 1: For Japanese users

[1587] User input: "My delivery is delayed and I'm having trouble. Can you do something about it?"

[1588] NLP engine analysis: Extraction of the keyword "delivery delay"

[1589] Emotional Engine Analysis: Detecting the emotion of "confusion"

[1590] Database Search: Retrieve guides and apology letters regarding "delivery delays" and "confusion."

[1591] Generated response: "We apologize for the delivery delay. Please refer to this link to check the current delivery status."

[1592] Display to the user: The device displays the response in the chat box.

[1593] Example 2: For English users

[1594] User input: "I can't log into my account and I'm getting frustrated."

[1595] NLP engine analysis: Extracting the keyword "cannot log in".

[1596] Emotional Engine Analysis: Detecting the emotion of "frustration"

[1597] Database Search: Get guides and encouragement based on "login issues" and "frustration."

[1598] Generated response: "We understand that you are frustrated. Please follow these steps to reset your password. For detailed instructions, refer to this link: example.com / resetpassword"

[1599] Display to the user: The device displays the response in the chat box.

[1600] This invention's system allows users to receive fast, personalized support in real time. In particular, its multilingual support and emotion recognition enable it to appropriately respond to users with different languages ​​and emotions. This allows companies to reduce operational costs and improve customer satisfaction.

[1601] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1602] Step 1:

[1603] Users enter support requests into a chat box on the website or application. For example, they might type, "My order was cancelled, please help me." This user input is the starting point for the system.

[1604] Input: Text of the support request entered by the user

[1605] Output: User input text is passed to the interface.

[1606] Step 2:

[1607] The interface receives user input text and automatically detects its language settings. This detection uses information such as web browser settings and account information. It then converts the input text to the appropriate format and sends it to the server.

[1608] Input: User input text

[1609] Data processing: Language setting detection and text formatting conversion

[1610] Output: Formatted text is sent to the server.

[1611] Step 3:

[1612] The server receives text data transmitted through the interface. The received text is sent to a natural language processing (NLP) engine for text analysis. The NLP engine tokenizes the text and extracts key keywords and phrases.

[1613] Input: Formatted text

[1614] Data processing: Text analysis and tokenization using a natural language processing engine.

[1615] Output: List of extracted keywords and phrases

[1616] Step 4:

[1617] The server sends the extracted keywords and phrases to the sentiment engine. The sentiment engine detects emotions from the text and assigns emotion labels such as "anger" or "confusion."

[1618] Input: List of extracted keywords or phrases

[1619] Data processing: Emotion analysis and emotion labeling using an emotion engine.

[1620] Output: Keywords and phrases with emotion labels

[1621] Step 5:

[1622] The server searches the database for relevant support information based on the analysis results and sentiment labels. The server constructs a search query and retrieves the relevant information using a database management system (e.g., MySQL or MongoDB).

[1623] Input: Keywords or phrases with emotion labels

[1624] Data Processing: Building database search queries and retrieving information.

[1625] Output: Related support information retrieved as search results

[1626] Step 6:

[1627] The server generates a response based on the acquired support information. It adjusts the response to the user with an appropriate tone and content, taking into account the emotion label detected by the emotion engine. For example, if the emotion "confused" is detected, it will include an apology such as, "We are sorry to trouble you."

[1628] Input: Related support information retrieved as search results

[1629] Data processing: Response text generation and tone adjustment

[1630] Output: Generated response

[1631] Step 7:

[1632] The server sends the generated response to the terminal. The terminal decodes the received response and displays it in the user's chat box in the appropriate format. This allows the user to receive timely and relevant information and support.

[1633] Input: Generated response

[1634] Data processing: Decoding and formatting of response statements

[1635] Output: Response displayed in the user's chat box

[1636] The above outlines the specific processing flow of this system. Effective and personalized support is provided through a consistent process from user input to the display of responses.

[1637] (Application Example 2)

[1638] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1639] Conventional customer support systems have struggled to recognize user emotions and adjust the tone and content of responses accordingly, resulting in a decline in the quality of the user experience. Furthermore, simultaneously implementing multilingual support and emotion recognition capabilities is difficult, making improving customer satisfaction a particular challenge in global service provision. This invention aims to solve these problems and provide effective customer support that is attentive to the user's emotions.

[1640] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for receiving information input from a user, means for analyzing the input information, means for searching for relevant information based on the analysis results, means for recognizing the user's emotions based on the analyzed information, means for adjusting the tone and content of the response according to the emotion recognition results, means for providing the retrieved information to the user, means for supporting multiple languages, means for providing information to the user including multilingual responses, means for generating multilingual responses adjusted based on emotion recognition, means for including video links in the information provided to the user, means for the user to view support information based on the video links, and further means for selecting an appropriate video link based on the emotion recognition. This makes it possible to provide responses adapted to the user's emotions in real time and achieve high customer satisfaction even in a global context.

[1641] "Means of receiving information entered by users" refers to the function that allows the server to receive information entered by users through chat boxes or forms.

[1642] "Means for analyzing the input information" refers to a function that allows the server to analyze the user's input information received using natural language processing (NLP) or other analysis techniques to extract important keywords and phrases.

[1643] "Means for searching for relevant information based on the analysis results" refers to a function for searching for relevant information from a database using keywords or phrases obtained through the analysis.

[1644] "Means for providing the searched information to the user" refers to means for visually displaying the search results to the user, and includes websites, applications, chat boxes, etc.

[1645] "Means for recognizing the user's emotions based on the analyzed information" refers to a function that analyzes the user's emotions using an emotion engine based on the user's input information and assigns emotion labels such as "anger" or "confusion."

[1646] "Means for adjusting the tone and content of the response in accordance with the emotion recognition result" refers to a function for optimizing and adjusting the tone and content of the response presented to the user based on the emotion recognized by the emotion engine.

[1647] "Means of supporting multiple languages" refers to support functions that allow user input information and responses to be handled in multiple languages.

[1648] "Means of providing information to users, including multilingual responses" refers to a function that provides users with responses generated in multiple languages ​​based on analysis results, and displays them in an appropriate format according to the language the user is using.

[1649] "Means for generating multilingual responses based on the aforementioned emotion recognition" refers to a function for generating multilingual and emotion-appropriate response content based on the user's emotion recognition results.

[1650] "Means of including video links in the information provided to the user" refers to a function that provides the user with a link to watch a video as related support information.

[1651] "Means by which users can view support information based on the aforementioned video link" refers to a function that enables users to view support information through the provided video link.

[1652] "Means for selecting an appropriate video link based on the aforementioned emotion recognition" refers to a function for selecting and providing the most suitable video link for the user based on the emotion recognition results.

[1653] This invention relates to a system that analyzes support requests entered by users through websites and applications and provides relevant information. Specifically, it is characterized by analyzing the user's input information and emotions using a natural language processing (NLP) engine and an emotion engine, and generating an appropriate response based on that analysis.

[1654] System Configuration

[1655] This system consists of the following main components:

[1656] 1. User input receiving means:

[1657] This is a means of receiving information entered by users. Information is received through interfaces such as web forms, chat boxes, and mobile apps.

[1658] 2. Input information analysis means:

[1659] This is a method for analyzing the user's input information received. Specifically, it uses a natural language processing (NLP) engine (e.g., SpaCy, BERT) to extract important keywords and phrases from the input text.

[1660] 3. Emotion recognition means:

[1661] This is a method for recognizing user emotions based on NLP analysis results. It uses an emotion engine (e.g., IBM Watson Tone Analyzer) to detect emotions such as "anger" or "confusion" from user input.

[1662] 4. Related information retrieval methods:

[1663] This is a means of searching for relevant support information from a database based on the analysis results and sentiment recognition results. The support information is stored in a database such as Amazon RDS.

[1664] 5. Response generation means:

[1665] This is a means of generating appropriate responses for the user based on searched support information and sentiment recognition results. The generated responses are then adjusted in tone and content.

[1666] 6. Information provision method:

[1667] This is a means of providing the generated response to the user. The response is displayed through interfaces such as websites and mobile applications. The response may also include multilingual support.

[1668] 7. Method of providing video links:

[1669] This method includes including video links as support information, where necessary. This allows users to watch the videos and receive support to resolve their problems.

[1670] 8. Methods for selecting appropriate video links:

[1671] This is a means of selecting and providing appropriate video links to users based on emotion recognition results.

[1672] Hardware and software to be used

[1673] Server: An AWS (Amazon Web Services) EC2 instance is used to receive, analyze, and process user input.

[1674] Database: Support information is stored using Amazon RDS (Relational Database Service).

[1675] NLP engine: Uses SpaCy or BERT for natural language processing.

[1676] Emotion Engine: Uses IBM Watson Tone Analyzer to recognize user emotions.

[1677] Frontend: Developing smartphone applications using React Native.

[1678] Specific example

[1679] The following is a specific example of how this system works.

[1680] Specific example 1: For Japanese users

[1681] The user types in the chat box, "My ordered item hasn't arrived yet, please check on it."

[1682] The server receives user input and uses an NLP engine to extract the keyword "product not delivered".

[1683] The emotion engine recognizes the emotion of "confusion."

[1684] The system searches the database for "information regarding delivery delays" and generates an apology message to address the user's feelings of distress.

[1685] The system provides the user with the most appropriate response, which includes a link such as "Check your current delivery status here."

[1686] Example of a prompt

[1687] User: "My ordered item hasn't arrived yet, please check on it."

[1688] server:

[1689] Keyword extraction: ["product", "not delivered"]

[1690] Emotional label: ["Confused"]

[1691] Response generated: "We apologize that your order has not yet arrived. Please check the current delivery status here. [link]."

[1692] As described above, this system can analyze user input information and emotions, and provide optimal support information in real time.

[1693] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1694] Step 1:

[1695] The user enters a support request into the chat box. Specifically, they might enter text such as, "My ordered item hasn't arrived yet, please check on it." The device receives this input and sends it to the server.

[1696] Input: User input text

[1697] Output: Input text sent to the server

[1698] Step 2:

[1699] The server analyzes the received input text using a natural language processing (NLP) engine. Specifically, it tokenizes the text and extracts important keywords and phrases. For example, "product" and "not delivered" might be extracted.

[1700] Input: Input text

[1701] Output: Extracted keywords and phrases

[1702] Step 3:

[1703] The server uses an emotion engine to recognize the user's emotions from the input text. Specifically, it analyzes the text and assigns emotion labels such as "confused" or "angry." In this step, the emotion "confused" is detected.

[1704] Input: Input text

[1705] Output: Emotional label (e.g., "Confused")

[1706] Step 4:

[1707] Based on the analysis results and sentiment labels, the server searches the database for relevant support information. Specifically, it retrieves FAQs and guides related to the keyword "product not received." It also retrieves apology letters addressing the sentiment "confusion."

[1708] Input: Keywords, sentiment labels

[1709] Output: Related support information

[1710] Step 5:

[1711] The server generates an appropriate response based on the acquired support information and sentiment labels. Specifically, it combines relevant information to generate a response message for the user, adjusting the content and tone of the response. For example, it might generate a message like, "We apologize that your order has not yet arrived. You can check the current delivery status here."

[1712] Input: Relevant support information, sentiment labels

[1713] Output: Adjusted response

[1714] Step 6:

[1715] The server sends the generated response to the terminal. The terminal displays the received response to the user, for example, in a chat box.

[1716] Input: Adjusted response

[1717] Output: Response displayed to the user

[1718] With the above steps completed, the process of analyzing the user's input information and emotions, and providing optimal support information in real time, is finished.

[1719] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1720] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1721] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1722] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1723] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1724] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1725] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1726] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1727] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1728] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1729] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1730] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1731] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1732] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1733] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1734] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1735] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1736] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1737] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1738] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1739] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1740] The following is further disclosed regarding the embodiments described above.

[1741] (Claim 1)

[1742] A means of receiving information entered by the user,

[1743] Means for analyzing the input information,

[1744] A means for searching for relevant information based on the aforementioned analysis results,

[1745] Means for providing the retrieved information to the user,

[1746] A system that includes this.

[1747] (Claim 2)

[1748] The input information is provided as a means for supporting multiple languages,

[1749] A means for generating a response in multiple languages ​​based on the aforementioned analysis results,

[1750] It includes means for providing information to users, including multilingual responses.

[1751] The system according to claim 1.

[1752] (Claim 3)

[1753] The means of providing information to the aforementioned user includes a video link,

[1754] The system provides a means for users to view support information based on the aforementioned video link.

[1755] The system according to claim 1.

[1756] "Example 1"

[1757] (Claim 1)

[1758] A means of receiving information entered by the user,

[1759] Means for analyzing the input information,

[1760] A means for searching for relevant information based on the aforementioned analysis results,

[1761] Means for providing the retrieved information to the user,

[1762] A means for automatically formatting the input information,

[1763] Means for sending the formatted information to the server,

[1764] A means for extracting keywords using a natural language processing engine based on the aforementioned analysis results,

[1765] A means for generating a response based on the acquired information,

[1766] Means for sending the generated response to the user's terminal,

[1767] Means for displaying the transmitted response to the user,

[1768] A system that includes this.

[1769] (Claim 2)

[1770] The input information is provided as a means for supporting multiple languages,

[1771] A means for generating a response in multiple languages ​​based on the aforementioned analysis results,

[1772] Means of providing information to users, including multilingual responses,

[1773] The system according to claim 1, comprising the natural language processing engine.

[1774] (Claim 3)

[1775] The means of providing information to the aforementioned user includes a video link,

[1776] A means by which users can view support information based on the aforementioned video link,

[1777] Means of including interactive elements such as hyperlinks in the displayed response,

[1778] The system according to claim 1.

[1779] "Application Example 1"

[1780] (Claim 1)

[1781] A means of receiving information entered by the user,

[1782] Means for analyzing the input information,

[1783] A means for searching for relevant information based on the aforementioned analysis results,

[1784] Means for providing the retrieved information to the user,

[1785] Means for utilizing a natural language processing engine to generate a response to the user,

[1786] A system that includes this.

[1787] (Claim 2)

[1788] The input information is provided as a means for supporting multiple languages,

[1789] A means for generating a response in multiple languages ​​based on the aforementioned analysis results,

[1790] Means of providing information to users, including multilingual responses,

[1791] It includes a means of providing the generated response to the user's terminal in real time.

[1792] The system according to claim 1.

[1793] (Claim 3)

[1794] The means of providing information to the aforementioned user includes a video link,

[1795] A means by which users can view support information based on the aforementioned video link,

[1796] It includes a means of generating individual responses based on user requests using a generative AI model.

[1797] The system according to claim 1.

[1798] "Example 2 of combining an emotion engine"

[1799] (Claim 1)

[1800] A means of receiving information entered by the user,

[1801] A means for analyzing the input information using language analysis means,

[1802] A means for detecting emotions and assigning emotion labels based on the aforementioned analysis,

[1803] A means for searching for relevant information based on the aforementioned analysis results and emotion labels,

[1804] A means for generating a response in tone and content corresponding to the aforementioned emotion,

[1805] Means for providing the generated response to the user,

[1806] A system that includes this.

[1807] (Claim 2)

[1808] The input information is provided as a means for supporting multiple languages,

[1809] A means for generating responses in multiple languages ​​based on the aforementioned analysis results and emotion labels,

[1810] It includes means for providing information to users, including multilingual responses.

[1811] The system according to claim 1.

[1812] (Claim 3)

[1813] The information provided to the user includes means such as a viewing link,

[1814] The system provides a means for users to view support information based on the aforementioned viewing link.

[1815] The system according to claim 1.

[1816] "Application example 2 when combining with an emotional engine"

[1817] (Claim 1)

[1818] A means of receiving information entered by the user,

[1819] Means for analyzing the input information,

[1820] A means for searching for relevant information based on the aforementioned analysis results,

[1821] Means for providing the retrieved information to the user,

[1822] A means for recognizing the user's emotions based on the analyzed information,

[1823] Means for adjusting the tone and content of the response according to the emotion recognition result,

[1824] A system that includes this.

[1825] (Claim 2)

[1826] The input information is provided as a means for supporting multiple languages,

[1827] A means for generating a response in multiple languages ​​based on the aforementioned analysis results,

[1828] Means of providing information to users, including multilingual responses,

[1829] It includes means for generating multilingual, tailored responses based on emotion recognition.

[1830] The system according to claim 1.

[1831] (Claim 3)

[1832] The means of providing information to the aforementioned user includes a video link,

[1833] The system provides a means for users to view support information based on the aforementioned video link,

[1834] Furthermore, it includes means for selecting an appropriate video link based on the aforementioned emotion recognition.

[1835] The system according to claim 1. [Explanation of symbols]

[1836] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of receiving information entered by the user, Means for analyzing the input information, A means for searching for relevant information based on the aforementioned analysis results, Means for providing the retrieved information to the user, A system that includes this.

2. The input information is provided as a means for supporting multiple languages, A means for generating a response in multiple languages ​​based on the aforementioned analysis results, It includes means for providing information to users, including multilingual responses. The system according to claim 1.

3. The means of providing information to the aforementioned user includes a video link, The system provides a means for users to view support information based on the aforementioned video link. The system according to claim 1.

Images