System

Abstract

A system is provided.SOLUTION: This system is provided with a means for acquiring a message inputted by a user, a means for analyzing the acquired message by natural language processing, and for extracting a keyword and a topic, a means for retrieving a database based on the extracted keyword and topic, and for specifying relevant information and a person in charge, and a means for providing the specified relevant information and person in charge to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The inability to quickly find people with specific information or expertise within a company can delay the progress of important projects. Furthermore, the time required to search for and share information can lead to problems that reduce work efficiency. This can lead to a decline in overall company performance and make it difficult to make quick decisions and give work instructions. This invention aims to solve such problems. [Means for solving the problem]

[0005] The present invention provides a system that acquires messages entered by users, analyzes them using natural language processing to extract keywords and topics, searches a database based on the extracted keywords, and identifies relevant information and personnel. Furthermore, the system includes a means for searching the database again and providing updated information if the user enters an additional question. The system also provides contact information for the identified personnel, allowing the user to contact them directly.

[0006] "User" means an individual or entity that utilizes the System to search for information or make inquiries.

[0007] "Typed Message" means a textual inquiry or question that a User submits through the System.

[0008] "Natural language processing" refers to the technology that allows computers to understand and analyze human language, and is used to extract keywords and topics.

[0009] "Keywords" refer to key words or phrases extracted from a user's message that form the basis of a database search.

[0010] A "topic" refers to a theme or subject that roughly classifies the content of a user's message.

[0011] A "database" is a system that systematically collects and stores information, and is a place where it can be searched based on keywords or topics.

[0012] "Search" refers to the act of locating information related to a specified keyword or topic within a database.

[0013] "Relevant information" refers to documents and data in databases found through a search and the associated expertise.

[0014] "Person in Charge" refers to an individual or department responsible for work related to specific information or expertise.

[0015] "Contact information" is information that provides a means of directly contacting a person in charge, and includes email addresses, telephone numbers, etc. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[0038] 1. Getting the input message

[0039] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[0040] 2. Message Analysis

[0041] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[0042] 3. Database Search

[0043] The device generates a search query based on the keywords extracted by the generative AI module and sends it to the server. The server uses this query to search a database and identify relevant information and personnel. For example, guidelines and past project data related to "database design" may be returned as search results.

[0044] 4. Organizing and providing information

[0045] The server receives the search results, organizes them, and generates information packets to provide to the user. The organized information is sent to the user via their device. Specific examples include links to related documents and contact information for the person in charge.

[0046] 5. Processing of follow-up questions

[0047] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query, and the server again searches the database and provides updated information.

[0048] 6. Direct Contact Options

[0049] Additionally, if the user wants more information about a particular piece of information, the device will provide contact information for the identified person so the user can contact them directly, such as the database engineer's email address and phone number.

[0050] Specific examples

[0051] Example 1: Retrieving Database Design

[0052] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[0053] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[0054] 3. Database search: The server searches the database based on keywords to identify relevant documents and people.

[0055] 4. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[0056] 5. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[0057] 6. Direct Contact Option: If necessary, users will be provided with contact information for a database engineer.

[0058] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates information sharing within the company and improves business efficiency.

[0059] The processing flow will be explained below.

[0060] Step 1:

[0061] A user types a question through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[0062] Step 2:

[0063] The terminal receives the user's input message and forwards it to the generative AI module for analysis.

[0064] Step 3:

[0065] The generative AI module analyzes the user's message using natural language processing technology to extract important keywords and topics, such as "database design," "new project," and "optimal method."

[0066] Step 4:

[0067] The generative AI module uses the extracted keywords to generate a query for database search, which is then sent by the server to the database server.

[0068] Step 5:

[0069] The server searches the database server based on the generated query for relevant documents and contact information.

[0070] Step 6:

[0071] The server receives search results from the database server, which include a list of relevant documents and contacts.

[0072] Step 7:

[0073] The server organizes the retrieved information and generates an information packet to provide to the user, which includes a link to the document of the search result and contact information of the person in charge.

[0074] Step 8:

[0075] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[0076] Step 9:

[0077] If the user enters an additional question, the device again sends the new question to the generative AI module, and the next cycle begins. Similarly, the generative AI module analyzes the new question and performs a search again.

[0078] Step 10:

[0079] If the user wishes to contact a specific person directly, the device provides the contact information for the identified person, and the user can contact the person directly using the provided contact information.

[0080] Example 1

[0081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0082] Conventional information search and inquiry systems have had difficulty efficiently analyzing user input messages and quickly providing relevant information. In particular, they often struggled to accurately extract keywords and provide appropriate information for complex questions in natural language. Furthermore, when users asked follow-up questions, the lack of a seamless system to respond often led to a decline in operational efficiency.

[0083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0084] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and topics and sending it to the server, means for the server to search a database based on the search query and identify related information and personnel, and means for organizing the identified related information and personnel and generating an information packet to provide to the user. This makes it possible to quickly and accurately provide related information in response to user inquiries in complex natural language, thereby improving business efficiency.

[0085] An "input message" is text about a question or piece of information that a user sends through a chat app or other communication tool.

[0086] "Natural language processing" is a technology that allows computers to understand, interpret, and generate natural human language, and is used to analyze user input messages and extract keywords and topics.

[0087] "Keywords" are important words or phrases extracted from a user's input message and used to generate search queries.

[0088] "Topics" are major themes or subjects extracted from a user's input message and used to generate search queries.

[0089] A "search query" is a query generated based on the extracted keywords and topics, and is used by the server to search the database.

[0090] A "server" is a computer system that maintains a database and is responsible for retrieving relevant information based on a search query from a user.

[0091] A "database" is a collection of information that stores related information and data on personnel.

[0092] An "information packet" is a data structure containing relevant information that is generated by the server to organize and present search results to the user.

[0093] "Related Information" refers to necessary documentation, guidelines, and other reference materials identified based on an analysis of the user's input message and search query.

[0094] A "contact person" is someone with expertise or responsibilities related to the user's question or information, and whose contact information will be provided to the user as needed.

[0095] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[0096] First, a user inputs information or a question using a communication tool such as a chat app. For example, they might input, "I would like to know the best way to design a database for a new project." This message is received by the device and recorded as internal data.

[0097] The device then analyzes the received message using natural language processing (NLP) technology, using a generative AI model (e.g., OpenAI's GPT-4) installed on the device. The generative AI model analyzes the message and extracts important keywords and topics. For example, keywords such as "database design," "new project," and "optimal method" are extracted from the input message.

[0098] Based on the extracted keywords, the device generates a search query, which is then sent to the server. The query includes phrases such as "database design," "new project," and "best method."

[0099] The server searches its database based on the received search query. The database contains relevant information and personnel data. The server identifies relevant information and compiles it as search results. For example, it extracts relevant guidelines or past project data.

[0100] The server organizes the search results and generates an information packet to provide to the user. This information packet includes links to related documents and contact information for the person in charge. The server then sends the information packet to the device, which then provides it to the user. For example, the device may display links to related documents in a chat app and send the contact information for the person in charge.

[0101] If the user enters an additional question, the device analyzes the message again and generates a new search query. This query is also sent to the server, which searches the database again. For example, if the user enters "I'd like to know about specific design examples," the information will be provided again.

[0102] Furthermore, if the user needs more information, the device will provide the contact information of the identified person so that the user can contact them directly, for example, providing the database engineer's email address and phone number.

[0103] Specific examples

[0104] Example prompt sentence:

[0105] A user types "Please tell me the best way to design a database for a new project" into a chat app. The device receives the message, and the generative AI model extracts the keywords "database design," "new project," and "best method." The device generates a search query including keywords such as "database design," "new project," and "best method," and sends it to the server. The server searches the database based on the search query and identifies relevant documents and personnel. The server organizes links to relevant documents and contact information for personnel, generates an information packet, and sends it to the device. The device provides the user with the information packet received from the server. If the user additionally types "I would also like to know specific design examples," the device analyzes it again and sends a new search query to the server. If necessary, the device provides the user with contact information for database engineers.

[0106] This system allows users to quickly and efficiently obtain the information they need and connect them with the appropriate experts, facilitating smooth information sharing within the company and improving work efficiency.

[0107] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0108] Step 1:

[0109] A user uses a chat app to enter information or a question.

[0110] What happens: A user types into a chat app, "I'd like to know the best way to design a database for a new project."

[0111] Input: User message: "I'd like to know the best way to design a database for a new project."

[0112] Output: Message data sent to the chat app.

[0113] Step 2:

[0114] The message received by the terminal is analyzed using natural language processing.

[0115] How it works: A generative AI model (e.g., OpenAI's GPT-4) installed on the device analyzes the message and extracts keywords such as "database design," "new project," and "optimal method."

[0116] Input: The user's message data.

[0117] Output: Extracted keywords: "database design", "new project", "best practice".

[0118] Step 3:

[0119] The terminal generates a search query based on the extracted keywords and transmits it to the server.

[0120] Specific operation: The device uses keywords to create a search query in the format "database design new project best method" and sends it to the server.

[0121] Input: extracted keywords "database design", "new project", "best practice".

[0122] Output: Generated search query "database design new project best way".

[0123] Step 4:

[0124] The server searches the database based on the received search query.

[0125] What happens: The server searches its internal database for this search query to identify relevant information (e.g., guidelines, past project data) and responsible parties.

[0126] Input: Received search query "database design new project best way".

[0127] Output: Search results with relevant information and contact data.

[0128] Step 5:

[0129] The server organizes the search results, generates information packets, and sends them to the terminal.

[0130] Specific operation: The server organizes the search results, compiles relevant document links and contact information for the person in charge into an information packet, and sends it to the terminal.

[0131] Input: relevant information and contact data.

[0132] Output: Information packet (document links and contact information).

[0133] Step 6:

[0134] The terminal provides the information packets received from the server to the user.

[0135] What happens: The device interprets the information packet, displays a link to the relevant document in the chat app, and sends the contact information of the person in charge.

[0136] Input: Information packet sent by the server.

[0137] Output: Relevant document links and contact information of the person in charge displayed within the chat app.

[0138] Step 7:

[0139] If the user enters an additional question, the device analyzes the message again and generates a new search query.

[0140] Specific operation: When a user types "I would also like to know about specific design examples," the device analyzes the message, extracts new keywords, generates a search query, and sends it to the server.

[0141] Input: User's additional message: "I would also like to know specific design examples."

[0142] Output: The newly generated search query.

[0143] Step 8:

[0144] The server will then query the database again and provide updated information.

[0145] Specific operation: The server searches the database based on the newly received search query, collects relevant information, and sends it to the terminal.

[0146] Input: A new search query.

[0147] Output: Updated search results (relevant information).

[0148] Step 9:

[0149] The terminal again provides the received information to the user.

[0150] What happens: The device will provide the updated information to the user again, specifically by displaying the new information in the chat app.

[0151] Input: Updated search results (related information).

[0152] Output: Updates displayed within the chat app.

[0153] Step 10:

[0154] If the user wants to know more about a particular piece of information, the terminal will provide contact information for the person in charge.

[0155] Specific operation: Provide the user with contact information (e.g., email address, phone number) of the person whose device was identified.

[0156] Input: Contact information for the person.

[0157] Output: The contact information of the person provided to the user.

[0158] (Application example 1)

[0159] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0160] In today's commercial environment, speed and efficiency of customer service in brick-and-mortar stores is a key challenge. In particular, it is necessary for store associates to provide appropriate information in response to customer questions in real time. However, with current systems, it is difficult for store associates to grasp all product information, and they are often unable to respond immediately when customers request more detailed information. This can lead to lower customer satisfaction and missed sales opportunities.

[0161] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0162] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and acquiring product information based on the created query, and means for providing the acquired product information to the user in an easy-to-understand format. This enables store clerks to provide optimal information in response to customer questions in real time, thereby improving customer satisfaction and maximizing sales opportunities.

[0163] A "user" is a person or customer who inputs information.

[0164] A "message" is text information containing a question or request entered by a user.

[0165] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0166] "Keywords" are important words extracted from a message.

[0167] A "topic" is an element that indicates the subject or topic of a message.

[0168] A "search query" is a form of question that is generated to search a database.

[0169] "Product information" refers to all product-related information retrieved from a database based on retrieved keywords.

[0170] "Information format" or "easy-to-understand format" refers to a way of presenting information in an organized manner that is easy for users to understand.

[0171] "Contact Person" refers to the individual or department responsible for providing relevant information or support.

[0172] This invention provides a "smart customer service assistant" system for improving customer service in brick-and-mortar stores. Specifically, this system allows store clerks to use smartphones or smart glasses to provide optimal information in response to customer questions in real time. The program processing and implementation method of this system are described in detail below.

[0173] First, the user (store clerk) inputs or speaks a customer's question through a smartphone or smart glasses. The captured message is then received by the device. The device's natural language processing (NLP) module then analyzes the message and extracts important keywords and topics. This can be done using Google Cloud's NLP API.

[0174] Next, a search query is generated based on the extracted keywords. This search query is sent to the store's database and used to search for related product information and contact information. The database contains detailed product information (such as ingredient lists, usage instructions, and warnings) and returns the relevant information.

[0175] The product information obtained as search results is organized in an easy-to-understand format by the terminal and provided to the user. This allows store staff to quickly respond to customer questions. Furthermore, if the user enters additional questions, the same process can be repeated to obtain and provide new information.

[0176] If necessary, users can make further inquiries based on the identified information. For example, if product expertise is required, the app will provide contact information for the customer support center or product development team, allowing users to quickly connect with an expert.

[0177] For example, if a customer asks, "What are the ingredients in this product?" the system will act as follows:

[0178] 1. Obtaining input message: The clerk enters, "Please tell me about the ingredients of this product."

[0179] 2. Message analysis: The device receives the message and the NLP module extracts keywords related to the "product ingredients."

[0180] 3. Database search: Retrieve product ingredient information from the database based on the search query.

[0181] 4. Providing information: The acquired information is provided to the salesperson in an easy-to-understand format. The display reads, "This product contains ingredients A, B, and C."

[0182] An example prompt is:

[0183] A customer asks, "What are the ingredients in this product?" How can I provide the necessary information for this question?

[0184] This invention will improve customer service in brick-and-mortar stores, allowing store associates to quickly and efficiently respond to customer questions, increasing customer satisfaction and maximizing sales opportunities.

[0185] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0186] Step 1:

[0187] The user receives the customer's question through a smartphone or smart glasses. The user inputs or speaks the customer's question into the terminal, which then receives the message. The input message is stored on the terminal as raw text data.

[0188] Step 2:

[0189] The device analyzes the received message. This analysis uses Google Cloud's NLP API to extract important keywords and topics from the message. Specifically, the NLP API parses the input message and identifies notable words such as nouns and verbs. The output is a list of extracted keywords.

[0190] Step 3:

[0191] A search query is generated based on the extracted keywords. The terminal combines the extracted keywords to create a search query for the database. This search query is structured in an appropriate format for retrieving product information. The output is the generated query.

[0192] Step 4:

[0193] The terminal sends the generated search query to the server and searches the database. The server searches the database based on the received query and retrieves the relevant product information. The database used here contains detailed product information (such as ingredient list, usage instructions, and precautions). The output is the product information as a search result.

[0194] Step 5:

[0195] The server then organizes the product information into an easy-to-understand format, including the product name, ingredient list, and usage instructions, and outputs the information as a data packet ready to be delivered to the user.

[0196] Step 6:

[0197] The server sends the organized product information to the terminal. The terminal displays the received information and provides it to the user. The display format can be a visual interface including text and images. This allows the user to quickly respond to the customer. Specifically, it will be displayed in the form of "The ingredients of this product include components A, B, and C."

[0198] Step 7:

[0199] If the user enters a follow-up question, the process from step 1 to step 6 is repeated. The follow-up question is also parsed using natural language processing, a database search is performed, and updated information is provided.

[0200] Step 8:

[0201] If necessary, the server will provide contact information for specific personnel for further inquiries or specialized support, for example, "If you need more information, please contact our support center here," allowing the user to directly contact an expert.

[0202] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0203] The system according to the present invention is realized by a program that allows users to quickly and efficiently search for information and respond to inquiries. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to provide more appropriate search results and personnel. The program for this system is explained below, along with the process flow and specific examples.

[0204] 1. Getting the input message

[0205] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[0206] 2. Message Analysis

[0207] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[0208] 3. Emotion recognition

[0209] As the device analyzes the message, the emotion engine recognizes emotions from the user's input. Emotions include joy, sadness, anger, surprise, fear, etc. For example, if the user inputs a message expressing impatience, that emotion will be recognized.

[0210] 4. Database Search

[0211] Based on the keywords extracted by the generative AI module, a query for database search is generated. This query is sent to the database server by the server. The search results may be adjusted taking into account the results of the emotion engine.

[0212] 5. Tailoring search results

[0213] The server searches the database server based on the generated query, and the search results are adjusted based on the emotions recognized by the emotion engine. For example, if the user is feeling anxious, documents with more detailed explanations will be prioritized.

[0214] 6. Organizing and providing information

[0215] The server organizes the information it has acquired and generates an information packet to provide to the user. This packet includes document links for search results and contact information for the agent. The agent recommendations may be adjusted based on the results of the emotion engine.

[0216] 7. Processing of follow-up questions

[0217] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query. The server again searches the database and provides updated information. The emotion engine also continues to analyze the input message and make adjustments based on emotion.

[0218] 8. Direct Contact Option

[0219] Additionally, if the user wants to know more about a specific piece of information, the device will provide the contact information of the identified person, allowing the user to contact them directly. For example, the device may provide the email address and phone number of a database engineer. Based on the results of the emotion engine, the device may also recommend a person the user feels comfortable contacting.

[0220] Specific examples

[0221] Example 1: Retrieving Database Design

[0222] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[0223] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[0224] 3. Emotion Recognition: The emotion engine recognizes anxiety from the user's message.

[0225] 4. Database search: The server searches the database based on keywords to identify relevant documents and people.

[0226] 5. Tailoring search results: Based on the results of the sentiment engine, documents with detailed descriptions are prioritized.

[0227] 6. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[0228] 7. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[0229] 8. Direct Contact Option: If needed, users will be provided with contact information for a database engineer. Based on the results of the sentiment engine, a reassuring contact will be recommended.

[0230] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates smooth information sharing within the company and improves work efficiency. In addition, the introduction of an emotion engine enables detailed responses tailored to the user's emotional state.

[0231] The processing flow will be explained below.

[0232] Step 1:

[0233] A user types a question or request for information through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[0234] Step 2:

[0235] The terminal receives the user's input message and forwards it to the emotion engine and generative AI module.

[0236] Step 3:

[0237] The emotion engine analyzes the user's input message and recognizes emotions, for example, "anxiety" from the user's message.

[0238] Step 4:

[0239] The generative AI module analyzes the content of the user's message using natural language processing technology and extracts important keywords and topics, such as "database design," "new project," and "optimal method."

[0240] Step 5:

[0241] The generative AI module generates search queries based on the extracted keywords. These queries also incorporate the results of the emotion engine and are adjusted to prioritize information that takes the user's emotions into consideration.

[0242] Step 6:

[0243] The server uses the generated search query to search the database server for related documents and contact information.

[0244] Step 7:

[0245] The server receives the search results from the database and organizes them, prioritizing documents with detailed descriptions that reduce the user's anxiety based on an emotion engine.

[0246] Step 8:

[0247] The server then uses the organized information to generate an information packet to present to the user, which includes links to related documentation and contact information for the appropriate person.

[0248] Step 9:

[0249] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[0250] Step 10:

[0251] If the user inputs an additional question, the device again sends the new question to the emotion engine and generative AI module, which continues to analyze the input message and make emotion-based adjustments.

[0252] Step 11:

[0253] The generative AI module analyzes the additional questions and generates new search queries, and the server again searches the database and provides updated information.

[0254] Step 12:

[0255] If the user wants to contact a specific agent directly, the device will provide the contact information of the identified agent, and based on the results of the emotion engine, a reassuring agent will be recommended.

[0256] Step 13:

[0257] The user can use the contact information provided to directly contact the appropriate person, for example, by providing an email address or phone number for the person.

[0258] Example 2

[0259] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0260] Conventional information retrieval systems only extract keywords from messages entered by users and perform database searches, but they have the problem of being unable to respond to the user's emotional state. Therefore, there is a need for a system that can respond in a detailed manner according to the user's emotional state.

[0261] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0262] In this invention, the server includes means for acquiring a message input by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for recognizing emotions from the user's input message, means for searching a database based on the extracted keywords and topics and the recognized emotions to identify related information and a person in charge, and means for providing the identified related information and a person in charge to the user, thereby enabling detailed information provision that takes into account the user's emotional state.

[0263] "User" refers to the end user who inputs information and uses the system.

[0264] "Message" refers to text data that a user sends to the system to display information or ask a question.

[0265] "Natural language processing" refers to the technology that enables computers to understand, analyze, and process human language.

[0266] "Keywords" refer to words that have a major meaning in message analysis.

[0267] "Topic" refers to the concept of the main topic or theme of a message.

[0268] "Emotion" refers to the psychological state perceived from a user's message.

[0269] "Database" refers to a data storage system for systematically storing and retrieving information.

[0270] "Contact Person" refers to a person designated to respond to user inquiries about specific information.

[0271] "Search" refers to the process of finding specific information from stored data.

[0272] "Contact Information" refers to information such as telephone numbers and email addresses used to contact a person in charge.

[0273] The system according to the present invention acquires a message input by a user and realizes efficient and appropriate information retrieval by combining natural language processing and emotion recognition. Specific embodiments of this system will be described below.

[0274] First, a user enters information or a question using a chat app or other communication tool. This sends a message to the device and receives it. For example, the message might be something like, "What's the best way to design a database for a new project?" The device then prepares the message to be saved in the database.

[0275] The device then analyzes the received message using a natural language processing (NLP) module. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords from the message, such as "database design," "new project," and "optimal method." This generative AI model has advanced natural language understanding capabilities and can extract keywords for a variety of topics.

[0276] Furthermore, the device uses an emotion engine (e.g., Affectiva's emotion recognition API) to analyze emotions from the user's input messages. The emotion engine detects emotions such as impatience or anxiety from the user's messages and records the results in a log. This information plays an important role in subsequent data retrieval and result adjustment.

[0277] Next, the server generates a database query based on the analysis results and keywords sent from the device. The generated query is sent to the database server, which searches for relevant information. At this stage, the emotion recognition results are taken into account and information that matches the user's emotions is searched for with priority. This process makes it possible to provide appropriate information according to the user's emotional state.

[0278] The server analyzes the search results retrieved from the database and adjusts them based on the emotion engine's recognition results. For example, if the user expresses anxiety, the server adjusts the results to prioritize detailed explanatory documents and support information.

[0279] The server then generates an information packet based on the adjusted search results, which includes links to related documents and contact information for the relevant person. The server also recommends the most appropriate contact information for the user based on the results of the emotion engine.

[0280] If the user has further questions, for example, by entering "I would also like to know about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server again searches the database. In the same way, it provides the user with search results according to their emotions.

[0281] Finally, if the user wants to know more about a particular piece of information, the device will provide contact information for the identified person, such as the email address and phone number of a database engineer. Based on the results of the emotion engine, the device will recommend the appropriate person for the user to contact.

[0282] As a concrete example, consider the prompt, "What is the best way to design a database for a new project?" Based on this prompt, a generative AI model extracts keywords, and an emotion engine recognizes the user's emotions, then provides the appropriate information and contact information for the appropriate person. This process allows users to quickly and efficiently obtain the information they need and connect with the appropriate expert. This system facilitates information sharing within a company and improves business efficiency.

[0283] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0284] Step 1:

[0285] The user inputs information or a question. The user uses a chat app or other communication tool to type, "What is the best way to design a database for a new project?" The device receives this message and stores the input message in an internal buffer.

[0286] Input: User's message

[0287] Output: Internal buffer storing received messages

[0288] Step 2:

[0289] The device then analyzes the received message using a natural language processing module, which uses a generative AI model (e.g., GPT-3) to analyze the message and extract keywords such as "database design," "new project," and "optimal method."

[0290] Input: Received message

[0291] Output: Extracted keywords

[0292] Step 3:

[0293] The device uses an emotion engine to recognize emotions from the user's input messages. This emotion engine (for example, Affectiva's emotion recognition API) analyzes the user's messages, detects emotions such as impatience or anxiety, and records the results in a log.

[0294] Input: Received message

[0295] Output: Recognized emotion

[0296] Step 4:

[0297] The server generates a database query based on the analysis results (extracted keywords) and emotion recognition results sent from the device. This generated query is then sent to the database server. Here, the emotion recognition results are also taken into consideration, and the query is constructed so that information appropriate to the user's emotion is searched for with priority.

[0298] Input: extracted keywords and recognized emotions

[0299] Output: Generated database query

[0300] Step 5:

[0301] The server sends the generated query to a database server to search for relevant information, and the database accepts the query, performs the search, and returns relevant documents and information.

[0302] Input: Generated database query

[0303] Output: Search results (related documents and information)

[0304] Step 6:

[0305] The server analyzes the search results retrieved from the database and adjusts the results based on the emotion engine's recognition. For example, if the user is feeling anxious, the server will prioritize documents with detailed explanations. It will also recommend contact details based on the user's emotion.

[0306] Input: Search results and recognized emotions

[0307] Output: Refined search results

[0308] Step 7:

[0309] The server generates an information packet to provide to the user based on the adjusted search results. This information packet includes links to related documents, detailed descriptions of the emotions, and contact information for the person in charge. The server then sends this information to the terminal.

[0310] Input: Refined search results

[0311] Output: Generated information packet

[0312] Step 8:

[0313] The device provides the generated information packet to the user. If the user enters a further question, such as "I would like to know more about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server searches the database again.

[0314] Input: Generated information packet, user's additional questions

[0315] Output: Information provided to the user, new search query

[0316] Step 9:

[0317] If the user wants to contact the person directly, the device will provide the contact information of the identified person, such as the email address and phone number of a database engineer. Based on the emotion recognition results, it will recommend a person who is comfortable contacting the person.

[0318] Input: User request, identified person

[0319] Output: Provide contact information for the person in charge

[0320] (Application example 2)

[0321] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0322] Conventional information retrieval systems cannot consider the user's emotional state when providing appropriate answers to user-entered questions or inquiries, which can lead to reduced user satisfaction. Furthermore, search results and information provided are uniform, making it difficult to provide information tailored to a user's specific needs and circumstances. Therefore, there is a need for a system that recognizes the user's emotional state and provides appropriate information.

[0323] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, emotion recognition means for recognizing the user's emotions during the message analysis process, means for searching a database based on the extracted keywords and topics and identifying related information and personnel, means for adjusting search results and information based on the results of emotion recognition, and means for providing the user with the identified related information and personnel. This makes it possible to provide detailed information that takes into account the user's search purpose and emotional state.

[0324] The "means for acquiring a message entered by a user" is a function for acquiring a message entered by a user through a communication tool.

[0325] "Natural language processing" is a technology that analyzes text messages and understands their meaning and context.

[0326] The "means for extracting keywords and topics" is a function for extracting important words and topics from messages.

[0327] "Means for searching a database and identifying relevant information and personnel" refers to a function that searches a database based on extracted keywords or topics to find relevant information and appropriate personnel.

[0328] The "means for providing the user with the identified related information and person in charge" is a function for providing the user with details of the searched information and person in charge.

[0329] "Emotion recognition means" is a technology for recognizing emotions from a user's input message.

[0330] "Means for adjusting search results and information" refers to a function that appropriately adjusts the information provided and search results based on the results of emotion recognition.

[0331] The present invention is a system that analyzes a message input by a user and provides appropriate information based on the message. The processing contents of a specific program for implementing the present invention will be described below.

[0332] The system first captures messages entered into a chat box by users on devices such as smartphones. These messages are then analyzed by a natural language processing (NLP) engine to extract important keywords and topics. This analysis uses generative AI models such as OpenAI's GPT-3.

[0333] The emotion engine then recognizes the user's emotions during message analysis. This emotion recognition uses emotion analysis libraries such as VADER. For example, if a user types, "What is the best model of a new smartphone?", the NLP engine extracts keywords such as "smartphone" and "best model," and the emotion engine recognizes that the user is excited.

[0334] Based on the extracted keywords and the emotion recognition results, a query is generated to search the database. This query is sent to the server, which then searches the database. The search results are adjusted based on the emotion recognition results. For example, if the user is excited, the newest or most popular products will be displayed preferentially.

[0335] The resulting search results are then organized and provided to the user, including product details, purchase links, and review links. If the user enters additional questions, the generative AI model is used again to analyze the question, generate a new search query, and provide updated information.

[0336] Additionally, the contact information of the identified representatives is also provided to the user, allowing the user to directly contact the customer service representative if necessary. Based on the results of emotion recognition, appropriate representatives are recommended.

[0337] As a specific example, consider the following user message:

[0338] For example: "What is the best model of the new smartphone?"

[0339] The prompt to input this to the generative AI model is as follows:

[0340] "Extract keywords from this user message: What's the best new smartphone model?"

[0341] This system is expected to improve user satisfaction by providing detailed information that takes into account the user's search purpose and emotional state. In addition, the emotion recognition function will recommend a more appropriate representative, allowing users to receive the support they need quickly.

[0342] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0343] Step 1:

[0344] The user enters a message. The user enters a question or inquiry into the smartphone's chat box. An example of an input is "What is the best model of new smartphone?" This message is sent to the device.

[0345] Step 2:

[0346] The terminal acquires the message. The communication module of the terminal receives the message sent by the user. This acquired message becomes the input for the next analysis step.

[0347] Step 3:

[0348] The device passes the message to the natural language processing engine. The device passes the message to the NLP engine and begins analysis. Specifically, the device sends a message to the API of a generative AI model (e.g., GPT-3) and prompts it to extract keywords and topics.

[0349] Step 4:

[0350] The NLP engine extracts keywords and topics. The generative AI model analyzes the message and extracts important keywords and topics. For example, the keywords "new smartphone" and "best model" are extracted from the message "What is the best new smartphone model?". This extraction result becomes the input for the next emotion recognition step.

[0351] Step 5:

[0352] The device passes the message to the emotion recognition engine. The device also passes the message it has acquired at the same time to the emotion recognition engine (e.g., VADER) and instructs it to analyze the user's emotions.

[0353] Step 6:

[0354] The emotion recognition engine recognizes emotions. The emotion recognition engine analyzes the message and recognizes the user's emotion. For example, the emotion "excited" is recognized from the user's message. This emotion recognition result is used to adjust the next database search.

[0355] Step 7:

[0356] The server generates a query for database search. The device sends the extracted keywords and emotion recognition results to the server, and the server generates a query for database search based on them. For example, the generated query might be in the form of "New smartphone and best model ORDER BY relevance_with_excitement DESC."

[0357] Step 8:

[0358] The server searches the database based on the query generated and retrieves relevant information. The search results include product details and a link to purchase.

[0359] Step 9:

[0360] The server adjusts the search results based on the emotion recognition results. For example, if the user is excited, the latest products or popular items will be given priority.

[0361] Step 10:

[0362] The server organizes the information and sends it to the device. The server organizes the tailored search results and generates an information packet to provide to the user. This information packet includes product details, purchase links, and review links. The organized information is sent to the device.

[0363] Step 11:

[0364] The terminal provides information to the user. The terminal displays search results to the user based on the information packets received from the server. This allows the user to quickly obtain the information they need.

[0365] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0366] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0367] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0368] [Second embodiment]

[0369] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0370] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0371] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0372] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0373] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0374] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0375] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0376] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0377] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0378] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0379] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0380] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0381] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[0382] 1. Getting the input message

[0383] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[0384] 2. Message Analysis

[0385] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[0386] 3. Database Search

[0387] The device generates a search query based on the keywords extracted by the generative AI module and sends it to the server. The server uses this query to search a database and identify relevant information and personnel. For example, guidelines and past project data related to "database design" may be returned as search results.

[0388] 4. Organizing and providing information

[0389] The server receives the search results, organizes them, and generates information packets to provide to the user. The organized information is sent to the user via their device. Specific examples include links to related documents and contact information for the person in charge.

[0390] 5. Processing of follow-up questions

[0391] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query, and the server again searches the database and provides updated information.

[0392] 6. Direct Contact Options

[0393] Additionally, if the user wants more information about a particular piece of information, the device will provide contact information for the identified person so the user can contact them directly, such as the database engineer's email address and phone number.

[0394] Specific examples

[0395] Example 1: Retrieving Database Design

[0396] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[0397] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[0398] 3. Database search: The server searches the database based on keywords to identify relevant documents and people.

[0399] 4. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[0400] 5. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[0401] 6. Direct Contact Option: If necessary, users will be provided with contact information for a database engineer.

[0402] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates information sharing within the company and improves business efficiency.

[0403] The processing flow will be explained below.

[0404] Step 1:

[0405] A user types a question through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[0406] Step 2:

[0407] The terminal receives the user's input message and forwards it to the generative AI module for analysis.

[0408] Step 3:

[0409] The generative AI module analyzes the user's message using natural language processing technology to extract important keywords and topics, such as "database design," "new project," and "optimal method."

[0410] Step 4:

[0411] The generative AI module uses the extracted keywords to generate a query for database search, which is then sent by the server to the database server.

[0412] Step 5:

[0413] The server searches the database server based on the generated query for relevant documents and contact information.

[0414] Step 6:

[0415] The server receives search results from the database server, which include a list of relevant documents and contacts.

[0416] Step 7:

[0417] The server organizes the retrieved information and generates an information packet to provide to the user, which includes a link to the document of the search result and contact information of the person in charge.

[0418] Step 8:

[0419] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[0420] Step 9:

[0421] If the user enters an additional question, the device again sends the new question to the generative AI module, and the next cycle begins. Similarly, the generative AI module analyzes the new question and performs a search again.

[0422] Step 10:

[0423] If the user wishes to contact a specific person directly, the device provides the contact information for the identified person, and the user can contact the person directly using the provided contact information.

[0424] Example 1

[0425] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0426] Conventional information search and inquiry systems have had difficulty efficiently analyzing user input messages and quickly providing relevant information. In particular, they often struggled to accurately extract keywords and provide appropriate information for complex questions in natural language. Furthermore, when users asked follow-up questions, the lack of a seamless system to respond often led to a decline in operational efficiency.

[0427] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0428] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and topics and sending it to the server, means for the server to search a database based on the search query and identify related information and personnel, and means for organizing the identified related information and personnel and generating an information packet to provide to the user. This makes it possible to quickly and accurately provide related information in response to user inquiries in complex natural language, thereby improving business efficiency.

[0429] An "input message" is text about a question or piece of information that a user sends through a chat app or other communication tool.

[0430] "Natural language processing" is a technology that allows computers to understand, interpret, and generate natural human language, and is used to analyze user input messages and extract keywords and topics.

[0431] "Keywords" are important words or phrases extracted from a user's input message and used to generate search queries.

[0432] "Topics" are major themes or subjects extracted from a user's input message and used to generate search queries.

[0433] A "search query" is a query generated based on the extracted keywords and topics, and is used by the server to search the database.

[0434] A "server" is a computer system that maintains a database and is responsible for retrieving relevant information based on a search query from a user.

[0435] A "database" is a collection of information that stores related information and data on personnel.

[0436] An "information packet" is a data structure containing relevant information that is generated by the server to organize and present search results to the user.

[0437] "Related Information" refers to necessary documentation, guidelines, and other reference materials identified based on an analysis of the user's input message and search query.

[0438] A "contact person" is someone with expertise or responsibilities related to the user's question or information, and whose contact information will be provided to the user as needed.

[0439] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[0440] First, a user inputs information or a question using a communication tool such as a chat app. For example, they might input, "I would like to know the best way to design a database for a new project." This message is received by the device and recorded as internal data.

[0441] The device then analyzes the received message using natural language processing (NLP) technology, using a generative AI model (e.g., OpenAI's GPT-4) installed on the device. The generative AI model analyzes the message and extracts important keywords and topics. For example, keywords such as "database design," "new project," and "optimal method" are extracted from the input message.

[0442] Based on the extracted keywords, the device generates a search query, which is then sent to the server. The query includes phrases such as "database design," "new project," and "best method."

[0443] The server searches its database based on the received search query. The database contains relevant information and personnel data. The server identifies relevant information and compiles it as search results. For example, it extracts relevant guidelines or past project data.

[0444] The server organizes the search results and generates an information packet to provide to the user. This information packet includes links to related documents and contact information for the person in charge. The server then sends the information packet to the device, which then provides it to the user. For example, the device may display links to related documents in a chat app and send the contact information for the person in charge.

[0445] If the user enters an additional question, the device analyzes the message again and generates a new search query. This query is also sent to the server, which searches the database again. For example, if the user enters "I'd like to know about specific design examples," the information will be provided again.

[0446] Furthermore, if the user needs more information, the device will provide the contact information of the identified person so that the user can contact them directly, for example, providing the database engineer's email address and phone number.

[0447] Specific examples

[0448] Example prompt sentence:

[0449] A user types "Please tell me the best way to design a database for a new project" into a chat app. The device receives the message, and the generative AI model extracts the keywords "database design," "new project," and "best method." The device generates a search query including keywords such as "database design," "new project," and "best method," and sends it to the server. The server searches the database based on the search query and identifies relevant documents and personnel. The server organizes links to relevant documents and contact information for personnel, generates an information packet, and sends it to the device. The device provides the user with the information packet received from the server. If the user additionally types "I would also like to know specific design examples," the device analyzes it again and sends a new search query to the server. If necessary, the device provides the user with contact information for database engineers.

[0450] This system allows users to quickly and efficiently obtain the information they need and connect them with the appropriate experts, facilitating smooth information sharing within the company and improving work efficiency.

[0451] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0452] Step 1:

[0453] A user uses a chat app to enter information or a question.

[0454] What happens: A user types into a chat app, "I'd like to know the best way to design a database for a new project."

[0455] Input: User message: "I'd like to know the best way to design a database for a new project."

[0456] Output: Message data sent to the chat app.

[0457] Step 2:

[0458] The message received by the terminal is analyzed using natural language processing.

[0459] How it works: A generative AI model (e.g., OpenAI's GPT-4) installed on the device analyzes the message and extracts keywords such as "database design," "new project," and "optimal method."

[0460] Input: The user's message data.

[0461] Output: Extracted keywords: "database design", "new project", "best practice".

[0462] Step 3:

[0463] The terminal generates a search query based on the extracted keywords and transmits it to the server.

[0464] Specific operation: The device uses keywords to create a search query in the format "database design new project best method" and sends it to the server.

[0465] Input: extracted keywords "database design", "new project", "best practice".

[0466] Output: Generated search query "database design new project best way".

[0467] Step 4:

[0468] The server searches the database based on the received search query.

[0469] What happens: The server searches its internal database for this search query to identify relevant information (e.g., guidelines, past project data) and responsible parties.

[0470] Input: Received search query "database design new project best way".

[0471] Output: Search results with relevant information and contact data.

[0472] Step 5:

[0473] The server organizes the search results, generates information packets, and sends them to the terminal.

[0474] Specific operation: The server organizes the search results, compiles relevant document links and contact information for the person in charge into an information packet, and sends it to the terminal.

[0475] Input: relevant information and contact data.

[0476] Output: Information packet (document links and contact information).

[0477] Step 6:

[0478] The terminal provides the information packets received from the server to the user.

[0479] What happens: The device interprets the information packet, displays a link to the relevant document in the chat app, and sends the contact information of the person in charge.

[0480] Input: Information packet sent by the server.

[0481] Output: Relevant document links and contact information of the person in charge displayed within the chat app.

[0482] Step 7:

[0483] If the user enters an additional question, the device analyzes the message again and generates a new search query.

[0484] Specific operation: When a user types "I would also like to know about specific design examples," the device analyzes the message, extracts new keywords, generates a search query, and sends it to the server.

[0485] Input: User's additional message: "I would also like to know specific design examples."

[0486] Output: The newly generated search query.

[0487] Step 8:

[0488] The server will then query the database again and provide updated information.

[0489] Specific operation: The server searches the database based on the newly received search query, collects relevant information, and sends it to the terminal.

[0490] Input: A new search query.

[0491] Output: Updated search results (relevant information).

[0492] Step 9:

[0493] The terminal again provides the received information to the user.

[0494] What happens: The device will provide the updated information to the user again, specifically by displaying the new information in the chat app.

[0495] Input: Updated search results (related information).

[0496] Output: Updates displayed within the chat app.

[0497] Step 10:

[0498] If the user wants to know more about a particular piece of information, the terminal will provide contact information for the person in charge.

[0499] Specific operation: Provide the user with contact information (e.g., email address, phone number) of the person whose device was identified.

[0500] Input: Contact information for the person.

[0501] Output: The contact information of the person provided to the user.

[0502] (Application example 1)

[0503] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0504] In today's commercial environment, speed and efficiency of customer service in brick-and-mortar stores is a key challenge. In particular, it is necessary for store associates to provide appropriate information in response to customer questions in real time. However, with current systems, it is difficult for store associates to grasp all product information, and they are often unable to respond immediately when customers request more detailed information. This can lead to lower customer satisfaction and missed sales opportunities.

[0505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0506] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and acquiring product information based on the created query, and means for providing the acquired product information to the user in an easy-to-understand format. This enables store clerks to provide optimal information in response to customer questions in real time, thereby improving customer satisfaction and maximizing sales opportunities.

[0507] A "user" is a person or customer who inputs information.

[0508] A "message" is text information containing a question or request entered by a user.

[0509] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0510] "Keywords" are important words extracted from a message.

[0511] A "topic" is an element that indicates the subject or topic of a message.

[0512] A "search query" is a form of question that is generated to search a database.

[0513] "Product information" refers to all product-related information retrieved from a database based on retrieved keywords.

[0514] "Information format" or "easy-to-understand format" refers to a way of presenting information in an organized manner that is easy for users to understand.

[0515] "Contact Person" refers to the individual or department responsible for providing relevant information or support.

[0516] This invention provides a "smart customer service assistant" system for improving customer service in brick-and-mortar stores. Specifically, this system allows store clerks to use smartphones or smart glasses to provide optimal information in response to customer questions in real time. The program processing and implementation method of this system are described in detail below.

[0517] First, the user (store clerk) inputs or speaks a customer's question through a smartphone or smart glasses. The captured message is then received by the device. The device's natural language processing (NLP) module then analyzes the message and extracts important keywords and topics. This can be done using Google Cloud's NLP API.

[0518] Next, a search query is generated based on the extracted keywords. This search query is sent to the store's database and used to search for related product information and contact information. The database contains detailed product information (such as ingredient lists, usage instructions, and warnings) and returns the relevant information.

[0519] The product information obtained as search results is organized in an easy-to-understand format by the terminal and provided to the user. This allows store staff to quickly respond to customer questions. Furthermore, if the user enters additional questions, the same process can be repeated to obtain and provide new information.

[0520] If necessary, users can make further inquiries based on the identified information. For example, if product expertise is required, the app will provide contact information for the customer support center or product development team, allowing users to quickly connect with an expert.

[0521] For example, if a customer asks, "What are the ingredients in this product?" the system will act as follows:

[0522] 1. Obtaining input message: The clerk enters, "Please tell me about the ingredients of this product."

[0523] 2. Message analysis: The device receives the message and the NLP module extracts keywords related to the "product ingredients."

[0524] 3. Database search: Retrieve product ingredient information from the database based on the search query.

[0525] 4. Providing information: The acquired information is provided to the salesperson in an easy-to-understand format. The display reads, "This product contains ingredients A, B, and C."

[0526] An example prompt is:

[0527] A customer asks, "What are the ingredients in this product?" How can I provide the necessary information for this question?

[0528] This invention will improve customer service in brick-and-mortar stores, allowing store associates to quickly and efficiently respond to customer questions, increasing customer satisfaction and maximizing sales opportunities.

[0529] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0530] Step 1:

[0531] The user receives the customer's question through a smartphone or smart glasses. The user inputs or speaks the customer's question into the terminal, which then receives the message. The input message is stored on the terminal as raw text data.

[0532] Step 2:

[0533] The device analyzes the received message. This analysis uses Google Cloud's NLP API to extract important keywords and topics from the message. Specifically, the NLP API parses the input message and identifies notable words such as nouns and verbs. The output is a list of extracted keywords.

[0534] Step 3:

[0535] A search query is generated based on the extracted keywords. The terminal combines the extracted keywords to create a search query for the database. This search query is structured in an appropriate format for retrieving product information. The output is the generated query.

[0536] Step 4:

[0537] The terminal sends the generated search query to the server and searches the database. The server searches the database based on the received query and retrieves the relevant product information. The database used here contains detailed product information (such as ingredient list, usage instructions, and precautions). The output is the product information as a search result.

[0538] Step 5:

[0539] The server then organizes the product information into an easy-to-understand format, including the product name, ingredient list, and usage instructions, and outputs the information as a data packet ready to be delivered to the user.

[0540] Step 6:

[0541] The server sends the organized product information to the terminal. The terminal displays the received information and provides it to the user. The display format can be a visual interface including text and images. This allows the user to quickly respond to the customer. Specifically, it will be displayed in the form of "The ingredients of this product include components A, B, and C."

[0542] Step 7:

[0543] If the user enters a follow-up question, the process from step 1 to step 6 is repeated. The follow-up question is also parsed using natural language processing, a database search is performed, and updated information is provided.

[0544] Step 8:

[0545] If necessary, the server will provide contact information for specific personnel for further inquiries or specialized support, for example, "If you need more information, please contact our support center here," allowing the user to directly contact an expert.

[0546] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0547] The system according to the present invention is realized by a program that allows users to quickly and efficiently search for information and respond to inquiries. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to provide more appropriate search results and personnel. The program for this system is explained below, along with the process flow and specific examples.

[0548] 1. Getting the input message

[0549] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[0550] 2. Message Analysis

[0551] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[0552] 3. Emotion recognition

[0553] As the device analyzes the message, the emotion engine recognizes emotions from the user's input. Emotions include joy, sadness, anger, surprise, fear, etc. For example, if the user inputs a message expressing impatience, that emotion will be recognized.

[0554] 4. Database Search

[0555] Based on the keywords extracted by the generative AI module, a query for database search is generated. This query is sent to the database server by the server. The search results may be adjusted taking into account the results of the emotion engine.

[0556] 5. Tailoring search results

[0557] The server searches the database server based on the generated query, and the search results are adjusted based on the emotions recognized by the emotion engine. For example, if the user is feeling anxious, documents with more detailed explanations will be prioritized.

[0558] 6. Organizing and providing information

[0559] The server organizes the information it has acquired and generates an information packet to provide to the user. This packet includes document links for search results and contact information for the agent. The agent recommendations may be adjusted based on the results of the emotion engine.

[0560] 7. Processing of follow-up questions

[0561] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query. The server again searches the database and provides updated information. The emotion engine also continues to analyze the input message and make adjustments based on emotion.

[0562] 8. Direct Contact Option

[0563] Additionally, if the user wants to know more about a specific piece of information, the device will provide the contact information of the identified person, allowing the user to contact them directly. For example, the device may provide the email address and phone number of a database engineer. Based on the results of the emotion engine, the device may also recommend a person the user feels comfortable contacting.

[0564] Specific examples

[0565] Example 1: Retrieving Database Design

[0566] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[0567] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[0568] 3. Emotion Recognition: The emotion engine recognizes anxiety from the user's message.

[0569] 4. Database search: The server searches the database based on keywords to identify relevant documents and people.

[0570] 5. Tailoring search results: Based on the results of the sentiment engine, documents with detailed descriptions are prioritized.

[0571] 6. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[0572] 7. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[0573] 8. Direct Contact Option: If needed, users will be provided with contact information for a database engineer. Based on the results of the sentiment engine, a reassuring contact will be recommended.

[0574] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates smooth information sharing within the company and improves work efficiency. In addition, the introduction of an emotion engine enables detailed responses tailored to the user's emotional state.

[0575] The processing flow will be explained below.

[0576] Step 1:

[0577] A user types a question or request for information through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[0578] Step 2:

[0579] The terminal receives the user's input message and forwards it to the emotion engine and generative AI module.

[0580] Step 3:

[0581] The emotion engine analyzes the user's input message and recognizes emotions, for example, "anxiety" from the user's message.

[0582] Step 4:

[0583] The generative AI module analyzes the content of the user's message using natural language processing technology and extracts important keywords and topics, such as "database design," "new project," and "optimal method."

[0584] Step 5:

[0585] The generative AI module generates search queries based on the extracted keywords. These queries also incorporate the results of the emotion engine and are adjusted to prioritize information that takes the user's emotions into consideration.

[0586] Step 6:

[0587] The server uses the generated search query to search the database server for related documents and contact information.

[0588] Step 7:

[0589] The server receives the search results from the database and organizes them, prioritizing documents with detailed descriptions that reduce the user's anxiety based on an emotion engine.

[0590] Step 8:

[0591] The server then uses the organized information to generate an information packet to present to the user, which includes links to related documentation and contact information for the appropriate person.

[0592] Step 9:

[0593] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[0594] Step 10:

[0595] If the user inputs an additional question, the device again sends the new question to the emotion engine and generative AI module, which continues to analyze the input message and make emotion-based adjustments.

[0596] Step 11:

[0597] The generative AI module analyzes the additional questions and generates new search queries, and the server again searches the database and provides updated information.

[0598] Step 12:

[0599] If the user wants to contact a specific agent directly, the device will provide the contact information of the identified agent, and based on the results of the emotion engine, a reassuring agent will be recommended.

[0600] Step 13:

[0601] The user can use the contact information provided to directly contact the appropriate person, for example, by providing an email address or phone number for the person.

[0602] Example 2

[0603] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0604] Conventional information retrieval systems only extract keywords from messages entered by users and perform database searches, but they have the problem of being unable to respond to the user's emotional state. Therefore, there is a need for a system that can respond in a detailed manner according to the user's emotional state.

[0605] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0606] In this invention, the server includes means for acquiring a message input by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for recognizing emotions from the user's input message, means for searching a database based on the extracted keywords and topics and the recognized emotions to identify related information and a person in charge, and means for providing the identified related information and a person in charge to the user, thereby enabling detailed information provision that takes into account the user's emotional state.

[0607] "User" refers to the end user who inputs information and uses the system.

[0608] "Message" refers to text data that a user sends to the system to display information or ask a question.

[0609] "Natural language processing" refers to the technology that enables computers to understand, analyze, and process human language.

[0610] "Keywords" refer to words that have a major meaning in message analysis.

[0611] "Topic" refers to the concept of the main topic or theme of a message.

[0612] "Emotion" refers to the psychological state perceived from a user's message.

[0613] "Database" refers to a data storage system for systematically storing and retrieving information.

[0614] "Contact Person" refers to a person designated to respond to user inquiries about specific information.

[0615] "Search" refers to the process of finding specific information from stored data.

[0616] "Contact Information" refers to information such as telephone numbers and email addresses used to contact a person in charge.

[0617] The system according to the present invention acquires a message input by a user and realizes efficient and appropriate information retrieval by combining natural language processing and emotion recognition. Specific embodiments of this system will be described below.

[0618] First, a user enters information or a question using a chat app or other communication tool. This sends a message to the device and receives it. For example, the message might be something like, "What's the best way to design a database for a new project?" The device then prepares the message to be saved in the database.

[0619] The device then analyzes the received message using a natural language processing (NLP) module. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords from the message, such as "database design," "new project," and "optimal method." This generative AI model has advanced natural language understanding capabilities and can extract keywords for a variety of topics.

[0620] Furthermore, the device uses an emotion engine (e.g., Affectiva's emotion recognition API) to analyze emotions from the user's input messages. The emotion engine detects emotions such as impatience or anxiety from the user's messages and records the results in a log. This information plays an important role in subsequent data retrieval and result adjustment.

[0621] Next, the server generates a database query based on the analysis results and keywords sent from the device. The generated query is sent to the database server, which searches for relevant information. At this stage, the emotion recognition results are taken into account and information that matches the user's emotions is searched for with priority. This process makes it possible to provide appropriate information according to the user's emotional state.

[0622] The server analyzes the search results retrieved from the database and adjusts them based on the emotion engine's recognition results. For example, if the user expresses anxiety, the server adjusts the results to prioritize detailed explanatory documents and support information.

[0623] The server then generates an information packet based on the adjusted search results, which includes links to related documents and contact information for the relevant person. The server also recommends the most appropriate contact information for the user based on the results of the emotion engine.

[0624] If the user has further questions, for example, by entering "I would also like to know about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server again searches the database. In the same way, it provides the user with search results according to their emotions.

[0625] Finally, if the user wants to know more about a particular piece of information, the device will provide contact information for the identified person, such as the email address and phone number of a database engineer. Based on the results of the emotion engine, the device will recommend the appropriate person for the user to contact.

[0626] As a concrete example, consider the prompt, "What is the best way to design a database for a new project?" Based on this prompt, a generative AI model extracts keywords, and an emotion engine recognizes the user's emotions, then provides the appropriate information and contact information for the appropriate person. This process allows users to quickly and efficiently obtain the information they need and connect with the appropriate expert. This system facilitates information sharing within a company and improves business efficiency.

[0627] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0628] Step 1:

[0629] The user inputs information or a question. The user uses a chat app or other communication tool to type, "What is the best way to design a database for a new project?" The device receives this message and stores the input message in an internal buffer.

[0630] Input: User's message

[0631] Output: Internal buffer storing received messages

[0632] Step 2:

[0633] The device then analyzes the received message using a natural language processing module, which uses a generative AI model (e.g., GPT-3) to analyze the message and extract keywords such as "database design," "new project," and "optimal method."

[0634] Input: Received message

[0635] Output: Extracted keywords

[0636] Step 3:

[0637] The device uses an emotion engine to recognize emotions from the user's input messages. This emotion engine (for example, Affectiva's emotion recognition API) analyzes the user's messages, detects emotions such as impatience or anxiety, and records the results in a log.

[0638] Input: Received message

[0639] Output: Recognized emotion

[0640] Step 4:

[0641] The server generates a database query based on the analysis results (extracted keywords) and emotion recognition results sent from the device. This generated query is then sent to the database server. Here, the emotion recognition results are also taken into consideration, and the query is constructed so that information appropriate to the user's emotion is searched for with priority.

[0642] Input: extracted keywords and recognized emotions

[0643] Output: Generated database query

[0644] Step 5:

[0645] The server sends the generated query to a database server to search for relevant information, and the database accepts the query, performs the search, and returns relevant documents and information.

[0646] Input: Generated database query

[0647] Output: Search results (related documents and information)

[0648] Step 6:

[0649] The server analyzes the search results retrieved from the database and adjusts the results based on the emotion engine's recognition. For example, if the user is feeling anxious, the server will prioritize documents with detailed explanations. It will also recommend contact details based on the user's emotion.

[0650] Input: Search results and recognized emotions

[0651] Output: Refined search results

[0652] Step 7:

[0653] The server generates an information packet to provide to the user based on the adjusted search results. This information packet includes links to related documents, detailed descriptions of the emotions, and contact information for the person in charge. The server then sends this information to the terminal.

[0654] Input: Refined search results

[0655] Output: Generated information packet

[0656] Step 8:

[0657] The device provides the generated information packet to the user. If the user enters a further question, such as "I would like to know more about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server searches the database again.

[0658] Input: Generated information packet, user's additional questions

[0659] Output: Information provided to the user, new search query

[0660] Step 9:

[0661] If the user wants to contact the person directly, the device will provide the contact information of the identified person, such as the email address and phone number of a database engineer. Based on the emotion recognition results, it will recommend a person who is comfortable contacting the person.

[0662] Input: User request, identified person

[0663] Output: Provide contact information for the person in charge

[0664] (Application example 2)

[0665] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0666] Conventional information retrieval systems cannot consider the user's emotional state when providing appropriate answers to user-entered questions or inquiries, which can lead to reduced user satisfaction. Furthermore, search results and information provided are uniform, making it difficult to provide information tailored to a user's specific needs and circumstances. Therefore, there is a need for a system that recognizes the user's emotional state and provides appropriate information.

[0667] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, emotion recognition means for recognizing the user's emotions during the message analysis process, means for searching a database based on the extracted keywords and topics and identifying related information and personnel, means for adjusting search results and information based on the results of emotion recognition, and means for providing the user with the identified related information and personnel. This makes it possible to provide detailed information that takes into account the user's search purpose and emotional state.

[0668] The "means for acquiring a message entered by a user" is a function for acquiring a message entered by a user through a communication tool.

[0669] "Natural language processing" is a technology that analyzes text messages and understands their meaning and context.

[0670] The "means for extracting keywords and topics" is a function for extracting important words and topics from messages.

[0671] "Means for searching a database and identifying relevant information and personnel" refers to a function that searches a database based on extracted keywords or topics to find relevant information and appropriate personnel.

[0672] The "means for providing the user with the identified related information and person in charge" is a function for providing the user with details of the searched information and person in charge.

[0673] "Emotion recognition means" is a technology for recognizing emotions from a user's input message.

[0674] "Means for adjusting search results and information" refers to a function that appropriately adjusts the information provided and search results based on the results of emotion recognition.

[0675] The present invention is a system that analyzes a message input by a user and provides appropriate information based on the message. The processing contents of a specific program for implementing the present invention will be described below.

[0676] The system first captures messages entered into a chat box by users on devices such as smartphones. These messages are then analyzed by a natural language processing (NLP) engine to extract important keywords and topics. This analysis uses generative AI models such as OpenAI's GPT-3.

[0677] The emotion engine then recognizes the user's emotions during message analysis. This emotion recognition uses emotion analysis libraries such as VADER. For example, if a user types, "What is the best model of a new smartphone?", the NLP engine extracts keywords such as "smartphone" and "best model," and the emotion engine recognizes that the user is excited.

[0678] Based on the extracted keywords and the emotion recognition results, a query is generated to search the database. This query is sent to the server, which then searches the database. The search results are adjusted based on the emotion recognition results. For example, if the user is excited, the newest or most popular products will be displayed preferentially.

[0679] The resulting search results are then organized and provided to the user, including product details, purchase links, and review links. If the user enters additional questions, the generative AI model is used again to analyze the question, generate a new search query, and provide updated information.

[0680] Additionally, the contact information of the identified representatives is also provided to the user, allowing the user to directly contact the customer service representative if necessary. Based on the results of emotion recognition, appropriate representatives are recommended.

[0681] As a specific example, consider the following user message:

[0682] For example: "What is the best model of the new smartphone?"

[0683] The prompt to input this to the generative AI model is as follows:

[0684] "Extract keywords from this user message: What's the best new smartphone model?"

[0685] This system is expected to improve user satisfaction by providing detailed information that takes into account the user's search purpose and emotional state. In addition, the emotion recognition function will recommend a more appropriate representative, allowing users to receive the support they need quickly.

[0686] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0687] Step 1:

[0688] The user enters a message. The user enters a question or inquiry into the smartphone's chat box. An example of an input is "What is the best model of new smartphone?" This message is sent to the device.

[0689] Step 2:

[0690] The terminal acquires the message. The communication module of the terminal receives the message sent by the user. This acquired message becomes the input for the next analysis step.

[0691] Step 3:

[0692] The device passes the message to the natural language processing engine. The device passes the message to the NLP engine and begins analysis. Specifically, the device sends a message to the API of a generative AI model (e.g., GPT-3) and prompts it to extract keywords and topics.

[0693] Step 4:

[0694] The NLP engine extracts keywords and topics. The generative AI model analyzes the message and extracts important keywords and topics. For example, the keywords "new smartphone" and "best model" are extracted from the message "What is the best new smartphone model?". This extraction result becomes the input for the next emotion recognition step.

[0695] Step 5:

[0696] The device passes the message to the emotion recognition engine. The device also passes the message it has acquired at the same time to the emotion recognition engine (e.g., VADER) and instructs it to analyze the user's emotions.

[0697] Step 6:

[0698] The emotion recognition engine recognizes emotions. The emotion recognition engine analyzes the message and recognizes the user's emotion. For example, the emotion "excited" is recognized from the user's message. This emotion recognition result is used to adjust the next database search.

[0699] Step 7:

[0700] The server generates a query for database search. The device sends the extracted keywords and emotion recognition results to the server, and the server generates a query for database search based on them. For example, the generated query might be in the form of "New smartphone and best model ORDER BY relevance_with_excitement DESC."

[0701] Step 8:

[0702] The server searches the database based on the query generated and retrieves relevant information. The search results include product details and a link to purchase.

[0703] Step 9:

[0704] The server adjusts the search results based on the emotion recognition results. For example, if the user is excited, the latest products or popular items will be given priority.

[0705] Step 10:

[0706] The server organizes the information and sends it to the device. The server organizes the tailored search results and generates an information packet to provide to the user. This information packet includes product details, purchase links, and review links. The organized information is sent to the device.

[0707] Step 11:

[0708] The terminal provides information to the user. The terminal displays search results to the user based on the information packets received from the server. This allows the user to quickly obtain the information they need.

[0709] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0710] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0711] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0712] [Third embodiment]

[0713] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0714] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0715] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0716] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0717] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0718] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0719] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0720] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0721] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0722] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0723] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0724] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0725] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[0726] 1. Getting the input message

[0727] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[0728] 2. Message Analysis

[0729] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[0730] 3. Database Search

[0731] The device generates a search query based on the keywords extracted by the generative AI module and sends it to the server. The server uses this query to search a database and identify relevant information and personnel. For example, guidelines and past project data related to "database design" may be returned as search results.

[0732] 4. Organizing and providing information

[0733] The server receives the search results, organizes them, and generates information packets to provide to the user. The organized information is sent to the user via their device. Specific examples include links to related documents and contact information for the person in charge.

[0734] 5. Processing of follow-up questions

[0735] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query, and the server again searches the database and provides updated information.

[0736] 6. Direct Contact Options

[0737] Additionally, if the user wants more information about a particular piece of information, the device will provide contact information for the identified person so the user can contact them directly, such as the database engineer's email address and phone number.

[0738] Specific examples

[0739] Example 1: Retrieving Database Design

[0740] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[0741] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[0742] 3. Database search: The server searches the database based on keywords to identify relevant documents and people.

[0743] 4. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[0744] 5. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[0745] 6. Direct Contact Option: If necessary, users will be provided with contact information for a database engineer.

[0746] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates information sharing within the company and improves business efficiency.

[0747] The processing flow will be explained below.

[0748] Step 1:

[0749] A user types a question through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[0750] Step 2:

[0751] The terminal receives the user's input message and forwards it to the generative AI module for analysis.

[0752] Step 3:

[0753] The generative AI module analyzes the user's message using natural language processing technology to extract important keywords and topics, such as "database design," "new project," and "optimal method."

[0754] Step 4:

[0755] The generative AI module uses the extracted keywords to generate a query for database search, which is then sent by the server to the database server.

[0756] Step 5:

[0757] The server searches the database server based on the generated query for relevant documents and contact information.

[0758] Step 6:

[0759] The server receives search results from the database server, which include a list of relevant documents and contacts.

[0760] Step 7:

[0761] The server organizes the retrieved information and generates an information packet to provide to the user, which includes a link to the document of the search result and contact information of the person in charge.

[0762] Step 8:

[0763] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[0764] Step 9:

[0765] If the user enters an additional question, the device again sends the new question to the generative AI module, and the next cycle begins. Similarly, the generative AI module analyzes the new question and performs a search again.

[0766] Step 10:

[0767] If the user wishes to contact a specific person directly, the device provides the contact information for the identified person, and the user can contact the person directly using the provided contact information.

[0768] Example 1

[0769] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0770] Conventional information search and inquiry systems have had difficulty efficiently analyzing user input messages and quickly providing relevant information. In particular, they often struggled to accurately extract keywords and provide appropriate information for complex questions in natural language. Furthermore, when users asked follow-up questions, the lack of a seamless system to respond often led to a decline in operational efficiency.

[0771] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0772] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and topics and sending it to the server, means for the server to search a database based on the search query and identify related information and personnel, and means for organizing the identified related information and personnel and generating an information packet to provide to the user. This makes it possible to quickly and accurately provide related information in response to user inquiries in complex natural language, thereby improving business efficiency.

[0773] An "input message" is text about a question or piece of information that a user sends through a chat app or other communication tool.

[0774] "Natural language processing" is a technology that allows computers to understand, interpret, and generate natural human language, and is used to analyze user input messages and extract keywords and topics.

[0775] "Keywords" are important words or phrases extracted from a user's input message and used to generate search queries.

[0776] "Topics" are major themes or subjects extracted from a user's input message and used to generate search queries.

[0777] A "search query" is a query generated based on the extracted keywords and topics, and is used by the server to search the database.

[0778] A "server" is a computer system that maintains a database and is responsible for retrieving relevant information based on a search query from a user.

[0779] A "database" is a collection of information that stores related information and data on personnel.

[0780] An "information packet" is a data structure containing relevant information that is generated by the server to organize and present search results to the user.

[0781] "Related Information" refers to necessary documentation, guidelines, and other reference materials identified based on an analysis of the user's input message and search query.

[0782] A "contact person" is someone with expertise or responsibilities related to the user's question or information, and whose contact information will be provided to the user as needed.

[0783] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[0784] First, a user inputs information or a question using a communication tool such as a chat app. For example, they might input, "I would like to know the best way to design a database for a new project." This message is received by the device and recorded as internal data.

[0785] The device then analyzes the received message using natural language processing (NLP) technology, using a generative AI model (e.g., OpenAI's GPT-4) installed on the device. The generative AI model analyzes the message and extracts important keywords and topics. For example, keywords such as "database design," "new project," and "optimal method" are extracted from the input message.

[0786] Based on the extracted keywords, the device generates a search query, which is then sent to the server. The query includes phrases such as "database design," "new project," and "best method."

[0787] The server searches its database based on the received search query. The database contains relevant information and personnel data. The server identifies relevant information and compiles it as search results. For example, it extracts relevant guidelines or past project data.

[0788] The server organizes the search results and generates an information packet to provide to the user. This information packet includes links to related documents and contact information for the person in charge. The server then sends the information packet to the device, which then provides it to the user. For example, the device may display links to related documents in a chat app and send the contact information for the person in charge.

[0789] If the user enters an additional question, the device analyzes the message again and generates a new search query. This query is also sent to the server, which searches the database again. For example, if the user enters "I'd like to know about specific design examples," the information will be provided again.

[0790] Furthermore, if the user needs more information, the device will provide the contact information of the identified person so that the user can contact them directly, for example, providing the database engineer's email address and phone number.

[0791] Specific examples

[0792] Example prompt sentence:

[0793] A user types "Please tell me the best way to design a database for a new project" into a chat app. The device receives the message, and the generative AI model extracts the keywords "database design," "new project," and "best method." The device generates a search query including keywords such as "database design," "new project," and "best method," and sends it to the server. The server searches the database based on the search query and identifies relevant documents and personnel. The server organizes links to relevant documents and contact information for personnel, generates an information packet, and sends it to the device. The device provides the user with the information packet received from the server. If the user additionally types "I would also like to know specific design examples," the device analyzes it again and sends a new search query to the server. If necessary, the device provides the user with contact information for database engineers.

[0794] This system allows users to quickly and efficiently obtain the information they need and connect them with the appropriate experts, facilitating smooth information sharing within the company and improving work efficiency.

[0795] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0796] Step 1:

[0797] A user uses a chat app to enter information or a question.

[0798] What happens: A user types into a chat app, "I'd like to know the best way to design a database for a new project."

[0799] Input: User message: "I'd like to know the best way to design a database for a new project."

[0800] Output: Message data sent to the chat app.

[0801] Step 2:

[0802] The message received by the terminal is analyzed using natural language processing.

[0803] How it works: A generative AI model (e.g., OpenAI's GPT-4) installed on the device analyzes the message and extracts keywords such as "database design," "new project," and "optimal method."

[0804] Input: The user's message data.

[0805] Output: Extracted keywords: "database design", "new project", "best practice".

[0806] Step 3:

[0807] The terminal generates a search query based on the extracted keywords and transmits it to the server.

[0808] Specific operation: The device uses keywords to create a search query in the format "database design new project best method" and sends it to the server.

[0809] Input: extracted keywords "database design", "new project", "best practice".

[0810] Output: Generated search query "database design new project best way".

[0811] Step 4:

[0812] The server searches the database based on the received search query.

[0813] What happens: The server searches its internal database for this search query to identify relevant information (e.g., guidelines, past project data) and responsible parties.

[0814] Input: Received search query "database design new project best way".

[0815] Output: Search results with relevant information and contact data.

[0816] Step 5:

[0817] The server organizes the search results, generates information packets, and sends them to the terminal.

[0818] Specific operation: The server organizes the search results, compiles relevant document links and contact information for the person in charge into an information packet, and sends it to the terminal.

[0819] Input: relevant information and contact data.

[0820] Output: Information packet (document links and contact information).

[0821] Step 6:

[0822] The terminal provides the information packets received from the server to the user.

[0823] What happens: The device interprets the information packet, displays a link to the relevant document in the chat app, and sends the contact information of the person in charge.

[0824] Input: Information packet sent by the server.

[0825] Output: Relevant document links and contact information of the person in charge displayed within the chat app.

[0826] Step 7:

[0827] If the user enters an additional question, the device analyzes the message again and generates a new search query.

[0828] Specific operation: When a user types "I would also like to know about specific design examples," the device analyzes the message, extracts new keywords, generates a search query, and sends it to the server.

[0829] Input: User's additional message: "I would also like to know specific design examples."

[0830] Output: The newly generated search query.

[0831] Step 8:

[0832] The server will then query the database again and provide updated information.

[0833] Specific operation: The server searches the database based on the newly received search query, collects relevant information, and sends it to the terminal.

[0834] Input: A new search query.

[0835] Output: Updated search results (relevant information).

[0836] Step 9:

[0837] The terminal again provides the received information to the user.

[0838] What happens: The device will provide the updated information to the user again, specifically by displaying the new information in the chat app.

[0839] Input: Updated search results (related information).

[0840] Output: Updates displayed within the chat app.

[0841] Step 10:

[0842] If the user wants to know more about a particular piece of information, the terminal will provide contact information for the person in charge.

[0843] Specific operation: Provide the user with contact information (e.g., email address, phone number) of the person whose device was identified.

[0844] Input: Contact information for the person.

[0845] Output: The contact information of the person provided to the user.

[0846] (Application example 1)

[0847] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0848] In today's commercial environment, speed and efficiency of customer service in brick-and-mortar stores is a key challenge. In particular, it is necessary for store associates to provide appropriate information in response to customer questions in real time. However, with current systems, it is difficult for store associates to grasp all product information, and they are often unable to respond immediately when customers request more detailed information. This can lead to lower customer satisfaction and missed sales opportunities.

[0849] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0850] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and acquiring product information based on the created query, and means for providing the acquired product information to the user in an easy-to-understand format. This enables store clerks to provide optimal information in response to customer questions in real time, thereby improving customer satisfaction and maximizing sales opportunities.

[0851] A "user" is a person or customer who inputs information.

[0852] A "message" is text information containing a question or request entered by a user.

[0853] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[0854] "Keywords" are important words extracted from a message.

[0855] A "topic" is an element that indicates the subject or topic of a message.

[0856] A "search query" is a form of question that is generated to search a database.

[0857] "Product information" refers to all product-related information retrieved from a database based on retrieved keywords.

[0858] "Information format" or "easy-to-understand format" refers to a way of presenting information in an organized manner that is easy for users to understand.

[0859] "Contact Person" refers to the individual or department responsible for providing relevant information or support.

[0860] This invention provides a "smart customer service assistant" system for improving customer service in brick-and-mortar stores. Specifically, this system allows store clerks to use smartphones or smart glasses to provide optimal information in response to customer questions in real time. The program processing and implementation method of this system are described in detail below.

[0861] First, the user (store clerk) inputs or speaks a customer's question through a smartphone or smart glasses. The captured message is then received by the device. The device's natural language processing (NLP) module then analyzes the message and extracts important keywords and topics. This can be done using Google Cloud's NLP API.

[0862] Next, a search query is generated based on the extracted keywords. This search query is sent to the store's database and used to search for related product information and contact information. The database contains detailed product information (such as ingredient lists, usage instructions, and warnings) and returns the relevant information.

[0863] The product information obtained as search results is organized in an easy-to-understand format by the terminal and provided to the user. This allows store staff to quickly respond to customer questions. Furthermore, if the user enters additional questions, the same process can be repeated to obtain and provide new information.

[0864] If necessary, users can make further inquiries based on the identified information. For example, if product expertise is required, the app will provide contact information for the customer support center or product development team, allowing users to quickly connect with an expert.

[0865] For example, if a customer asks, "What are the ingredients in this product?" the system will act as follows:

[0866] 1. Obtaining input message: The clerk enters, "Please tell me about the ingredients of this product."

[0867] 2. Message analysis: The device receives the message and the NLP module extracts keywords related to the "product ingredients."

[0868] 3. Database search: Retrieve product ingredient information from the database based on the search query.

[0869] 4. Providing information: The acquired information is provided to the salesperson in an easy-to-understand format. The display reads, "This product contains ingredients A, B, and C."

[0870] An example prompt is:

[0871] A customer asks, "What are the ingredients in this product?" How can I provide the necessary information for this question?

[0872] This invention will improve customer service in brick-and-mortar stores, allowing store associates to quickly and efficiently respond to customer questions, increasing customer satisfaction and maximizing sales opportunities.

[0873] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0874] Step 1:

[0875] The user receives the customer's question through a smartphone or smart glasses. The user inputs or speaks the customer's question into the terminal, which then receives the message. The input message is stored on the terminal as raw text data.

[0876] Step 2:

[0877] The device analyzes the received message. This analysis uses Google Cloud's NLP API to extract important keywords and topics from the message. Specifically, the NLP API parses the input message and identifies notable words such as nouns and verbs. The output is a list of extracted keywords.

[0878] Step 3:

[0879] A search query is generated based on the extracted keywords. The terminal combines the extracted keywords to create a search query for the database. This search query is structured in an appropriate format for retrieving product information. The output is the generated query.

[0880] Step 4:

[0881] The terminal sends the generated search query to the server and searches the database. The server searches the database based on the received query and retrieves the relevant product information. The database used here contains detailed product information (such as ingredient list, usage instructions, and precautions). The output is the product information as a search result.

[0882] Step 5:

[0883] The server then organizes the product information into an easy-to-understand format, including the product name, ingredient list, and usage instructions, and outputs the information as a data packet ready to be delivered to the user.

[0884] Step 6:

[0885] The server sends the organized product information to the terminal. The terminal displays the received information and provides it to the user. The display format can be a visual interface including text and images. This allows the user to quickly respond to the customer. Specifically, it will be displayed in the form of "The ingredients of this product include components A, B, and C."

[0886] Step 7:

[0887] If the user enters a follow-up question, the process from step 1 to step 6 is repeated. The follow-up question is also parsed using natural language processing, a database search is performed, and updated information is provided.

[0888] Step 8:

[0889] If necessary, the server will provide contact information for specific personnel for further inquiries or specialized support, for example, "If you need more information, please contact our support center here," allowing the user to directly contact an expert.

[0890] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0891] The system according to the present invention is realized by a program that allows users to quickly and efficiently search for information and respond to inquiries. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to provide more appropriate search results and personnel. The program for this system is explained below, along with the process flow and specific examples.

[0892] 1. Getting the input message

[0893] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[0894] 2. Message Analysis

[0895] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[0896] 3. Emotion recognition

[0897] As the device analyzes the message, the emotion engine recognizes emotions from the user's input. Emotions include joy, sadness, anger, surprise, fear, etc. For example, if the user inputs a message expressing impatience, that emotion will be recognized.

[0898] 4. Database Search

[0899] Based on the keywords extracted by the generative AI module, a query for database search is generated. This query is sent to the database server by the server. The search results may be adjusted taking into account the results of the emotion engine.

[0900] 5. Tailoring search results

[0901] The server searches the database server based on the generated query, and the search results are adjusted based on the emotions recognized by the emotion engine. For example, if the user is feeling anxious, documents with more detailed explanations will be prioritized.

[0902] 6. Organizing and providing information

[0903] The server organizes the information it has acquired and generates an information packet to provide to the user. This packet includes document links for search results and contact information for the agent. The agent recommendations may be adjusted based on the results of the emotion engine.

[0904] 7. Processing of follow-up questions

[0905] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query. The server again searches the database and provides updated information. The emotion engine also continues to analyze the input message and make adjustments based on emotion.

[0906] 8. Direct Contact Option

[0907] Additionally, if the user wants to know more about a specific piece of information, the device will provide the contact information of the identified person, allowing the user to contact them directly. For example, the device may provide the email address and phone number of a database engineer. Based on the results of the emotion engine, the device may also recommend a person the user feels comfortable contacting.

[0908] Specific examples

[0909] Example 1: Retrieving Database Design

[0910] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[0911] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[0912] 3. Emotion Recognition: The emotion engine recognizes anxiety from the user's message.

[0913] 4. Database search: The server searches the database based on keywords to identify relevant documents and people.

[0914] 5. Tailoring search results: Based on the results of the sentiment engine, documents with detailed descriptions are prioritized.

[0915] 6. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[0916] 7. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[0917] 8. Direct Contact Option: If needed, users will be provided with contact information for a database engineer. Based on the results of the sentiment engine, a reassuring contact will be recommended.

[0918] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates smooth information sharing within the company and improves work efficiency. In addition, the introduction of an emotion engine enables detailed responses tailored to the user's emotional state.

[0919] The processing flow will be explained below.

[0920] Step 1:

[0921] A user types a question or request for information through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[0922] Step 2:

[0923] The terminal receives the user's input message and forwards it to the emotion engine and generative AI module.

[0924] Step 3:

[0925] The emotion engine analyzes the user's input message and recognizes emotions, for example, "anxiety" from the user's message.

[0926] Step 4:

[0927] The generative AI module analyzes the content of the user's message using natural language processing technology and extracts important keywords and topics, such as "database design," "new project," and "optimal method."

[0928] Step 5:

[0929] The generative AI module generates search queries based on the extracted keywords. These queries also incorporate the results of the emotion engine and are adjusted to prioritize information that takes the user's emotions into consideration.

[0930] Step 6:

[0931] The server uses the generated search query to search the database server for related documents and contact information.

[0932] Step 7:

[0933] The server receives the search results from the database and organizes them, prioritizing documents with detailed descriptions that reduce the user's anxiety based on an emotion engine.

[0934] Step 8:

[0935] The server then uses the organized information to generate an information packet to present to the user, which includes links to related documentation and contact information for the appropriate person.

[0936] Step 9:

[0937] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[0938] Step 10:

[0939] If the user inputs an additional question, the device again sends the new question to the emotion engine and generative AI module, which continues to analyze the input message and make emotion-based adjustments.

[0940] Step 11:

[0941] The generative AI module analyzes the additional questions and generates new search queries, and the server again searches the database and provides updated information.

[0942] Step 12:

[0943] If the user wants to contact a specific agent directly, the device will provide the contact information of the identified agent, and based on the results of the emotion engine, a reassuring agent will be recommended.

[0944] Step 13:

[0945] The user can use the contact information provided to directly contact the appropriate person, for example, by providing an email address or phone number for the person.

[0946] Example 2

[0947] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0948] Conventional information retrieval systems only extract keywords from messages entered by users and perform database searches, but they have the problem of being unable to respond to the user's emotional state. Therefore, there is a need for a system that can respond in a detailed manner according to the user's emotional state.

[0949] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0950] In this invention, the server includes means for acquiring a message input by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for recognizing emotions from the user's input message, means for searching a database based on the extracted keywords and topics and the recognized emotions to identify related information and a person in charge, and means for providing the identified related information and a person in charge to the user, thereby enabling detailed information provision that takes into account the user's emotional state.

[0951] "User" refers to the end user who inputs information and uses the system.

[0952] "Message" refers to text data that a user sends to the system to display information or ask a question.

[0953] "Natural language processing" refers to the technology that enables computers to understand, analyze, and process human language.

[0954] "Keywords" refer to words that have a major meaning in message analysis.

[0955] "Topic" refers to the concept of the main topic or theme of a message.

[0956] "Emotion" refers to the psychological state perceived from a user's message.

[0957] "Database" refers to a data storage system for systematically storing and retrieving information.

[0958] "Contact Person" refers to a person designated to respond to user inquiries about specific information.

[0959] "Search" refers to the process of finding specific information from stored data.

[0960] "Contact Information" refers to information such as telephone numbers and email addresses used to contact a person in charge.

[0961] The system according to the present invention acquires a message input by a user and realizes efficient and appropriate information retrieval by combining natural language processing and emotion recognition. Specific embodiments of this system will be described below.

[0962] First, a user enters information or a question using a chat app or other communication tool. This sends a message to the device and receives it. For example, the message might be something like, "What's the best way to design a database for a new project?" The device then prepares the message to be saved in the database.

[0963] The device then analyzes the received message using a natural language processing (NLP) module. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords from the message, such as "database design," "new project," and "optimal method." This generative AI model has advanced natural language understanding capabilities and can extract keywords for a variety of topics.

[0964] Furthermore, the device uses an emotion engine (e.g., Affectiva's emotion recognition API) to analyze emotions from the user's input messages. The emotion engine detects emotions such as impatience or anxiety from the user's messages and records the results in a log. This information plays an important role in subsequent data retrieval and result adjustment.

[0965] Next, the server generates a database query based on the analysis results and keywords sent from the device. The generated query is sent to the database server, which searches for relevant information. At this stage, the emotion recognition results are taken into account and information that matches the user's emotions is searched for with priority. This process makes it possible to provide appropriate information according to the user's emotional state.

[0966] The server analyzes the search results retrieved from the database and adjusts them based on the emotion engine's recognition results. For example, if the user expresses anxiety, the server adjusts the results to prioritize detailed explanatory documents and support information.

[0967] The server then generates an information packet based on the adjusted search results, which includes links to related documents and contact information for the relevant person. The server also recommends the most appropriate contact information for the user based on the results of the emotion engine.

[0968] If the user has further questions, for example, by entering "I would also like to know about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server again searches the database. In the same way, it provides the user with search results according to their emotions.

[0969] Finally, if the user wants to know more about a particular piece of information, the device will provide contact information for the identified person, such as the email address and phone number of a database engineer. Based on the results of the emotion engine, the device will recommend the appropriate person for the user to contact.

[0970] As a concrete example, consider the prompt, "What is the best way to design a database for a new project?" Based on this prompt, a generative AI model extracts keywords, and an emotion engine recognizes the user's emotions, then provides the appropriate information and contact information for the appropriate person. This process allows users to quickly and efficiently obtain the information they need and connect with the appropriate expert. This system facilitates information sharing within a company and improves business efficiency.

[0971] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0972] Step 1:

[0973] The user inputs information or a question. The user uses a chat app or other communication tool to type, "What is the best way to design a database for a new project?" The device receives this message and stores the input message in an internal buffer.

[0974] Input: User's message

[0975] Output: Internal buffer storing received messages

[0976] Step 2:

[0977] The device then analyzes the received message using a natural language processing module, which uses a generative AI model (e.g., GPT-3) to analyze the message and extract keywords such as "database design," "new project," and "optimal method."

[0978] Input: Received message

[0979] Output: Extracted keywords

[0980] Step 3:

[0981] The device uses an emotion engine to recognize emotions from the user's input messages. This emotion engine (for example, Affectiva's emotion recognition API) analyzes the user's messages, detects emotions such as impatience or anxiety, and records the results in a log.

[0982] Input: Received message

[0983] Output: Recognized emotion

[0984] Step 4:

[0985] The server generates a database query based on the analysis results (extracted keywords) and emotion recognition results sent from the device. This generated query is then sent to the database server. Here, the emotion recognition results are also taken into consideration, and the query is constructed so that information appropriate to the user's emotion is searched for with priority.

[0986] Input: extracted keywords and recognized emotions

[0987] Output: Generated database query

[0988] Step 5:

[0989] The server sends the generated query to a database server to search for relevant information, and the database accepts the query, performs the search, and returns relevant documents and information.

[0990] Input: Generated database query

[0991] Output: Search results (related documents and information)

[0992] Step 6:

[0993] The server analyzes the search results retrieved from the database and adjusts the results based on the emotion engine's recognition. For example, if the user is feeling anxious, the server will prioritize documents with detailed explanations. It will also recommend contact details based on the user's emotion.

[0994] Input: Search results and recognized emotions

[0995] Output: Refined search results

[0996] Step 7:

[0997] The server generates an information packet to provide to the user based on the adjusted search results. This information packet includes links to related documents, detailed descriptions of the emotions, and contact information for the person in charge. The server then sends this information to the terminal.

[0998] Input: Refined search results

[0999] Output: Generated information packet

[1000] Step 8:

[1001] The device provides the generated information packet to the user. If the user enters a further question, such as "I would like to know more about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server searches the database again.

[1002] Input: Generated information packet, user's additional questions

[1003] Output: Information provided to the user, new search query

[1004] Step 9:

[1005] If the user wants to contact the person directly, the device will provide the contact information of the identified person, such as the email address and phone number of a database engineer. Based on the emotion recognition results, it will recommend a person who is comfortable contacting the person.

[1006] Input: User request, identified person

[1007] Output: Provide contact information for the person in charge

[1008] (Application example 2)

[1009] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] Conventional information retrieval systems cannot consider the user's emotional state when providing appropriate answers to user-entered questions or inquiries, which can lead to reduced user satisfaction. Furthermore, search results and information provided are uniform, making it difficult to provide information tailored to a user's specific needs and circumstances. Therefore, there is a need for a system that recognizes the user's emotional state and provides appropriate information.

[1011] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, emotion recognition means for recognizing the user's emotions during the message analysis process, means for searching a database based on the extracted keywords and topics and identifying related information and personnel, means for adjusting search results and information based on the results of emotion recognition, and means for providing the user with the identified related information and personnel. This makes it possible to provide detailed information that takes into account the user's search purpose and emotional state.

[1012] The "means for acquiring a message entered by a user" is a function for acquiring a message entered by a user through a communication tool.

[1013] "Natural language processing" is a technology that analyzes text messages and understands their meaning and context.

[1014] The "means for extracting keywords and topics" is a function for extracting important words and topics from messages.

[1015] "Means for searching a database and identifying relevant information and personnel" refers to a function that searches a database based on extracted keywords or topics to find relevant information and appropriate personnel.

[1016] The "means for providing the user with the identified related information and person in charge" is a function for providing the user with details of the searched information and person in charge.

[1017] "Emotion recognition means" is a technology for recognizing emotions from a user's input message.

[1018] "Means for adjusting search results and information" refers to a function that appropriately adjusts the information provided and search results based on the results of emotion recognition.

[1019] The present invention is a system that analyzes a message input by a user and provides appropriate information based on the message. The processing contents of a specific program for implementing the present invention will be described below.

[1020] The system first captures messages entered into a chat box by users on devices such as smartphones. These messages are then analyzed by a natural language processing (NLP) engine to extract important keywords and topics. This analysis uses generative AI models such as OpenAI's GPT-3.

[1021] The emotion engine then recognizes the user's emotions during message analysis. This emotion recognition uses emotion analysis libraries such as VADER. For example, if a user types, "What is the best model of a new smartphone?", the NLP engine extracts keywords such as "smartphone" and "best model," and the emotion engine recognizes that the user is excited.

[1022] Based on the extracted keywords and the emotion recognition results, a query is generated to search the database. This query is sent to the server, which then searches the database. The search results are adjusted based on the emotion recognition results. For example, if the user is excited, the newest or most popular products will be displayed preferentially.

[1023] The resulting search results are then organized and provided to the user, including product details, purchase links, and review links. If the user enters additional questions, the generative AI model is used again to analyze the question, generate a new search query, and provide updated information.

[1024] Additionally, the contact information of the identified representatives is also provided to the user, allowing the user to directly contact the customer service representative if necessary. Based on the results of emotion recognition, appropriate representatives are recommended.

[1025] As a specific example, consider the following user message:

[1026] For example: "What is the best model of the new smartphone?"

[1027] The prompt to input this to the generative AI model is as follows:

[1028] "Extract keywords from this user message: What's the best new smartphone model?"

[1029] This system is expected to improve user satisfaction by providing detailed information that takes into account the user's search purpose and emotional state. In addition, the emotion recognition function will recommend a more appropriate representative, allowing users to receive the support they need quickly.

[1030] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1031] Step 1:

[1032] The user enters a message. The user enters a question or inquiry into the smartphone's chat box. An example of an input is "What is the best model of new smartphone?" This message is sent to the device.

[1033] Step 2:

[1034] The terminal acquires the message. The communication module of the terminal receives the message sent by the user. This acquired message becomes the input for the next analysis step.

[1035] Step 3:

[1036] The device passes the message to the natural language processing engine. The device passes the message to the NLP engine and begins analysis. Specifically, the device sends a message to the API of a generative AI model (e.g., GPT-3) and prompts it to extract keywords and topics.

[1037] Step 4:

[1038] The NLP engine extracts keywords and topics. The generative AI model analyzes the message and extracts important keywords and topics. For example, the keywords "new smartphone" and "best model" are extracted from the message "What is the best new smartphone model?". This extraction result becomes the input for the next emotion recognition step.

[1039] Step 5:

[1040] The device passes the message to the emotion recognition engine. The device also passes the message it has acquired at the same time to the emotion recognition engine (e.g., VADER) and instructs it to analyze the user's emotions.

[1041] Step 6:

[1042] The emotion recognition engine recognizes emotions. The emotion recognition engine analyzes the message and recognizes the user's emotion. For example, the emotion "excited" is recognized from the user's message. This emotion recognition result is used to adjust the next database search.

[1043] Step 7:

[1044] The server generates a query for database search. The device sends the extracted keywords and emotion recognition results to the server, and the server generates a query for database search based on them. For example, the generated query might be in the form of "New smartphone and best model ORDER BY relevance_with_excitement DESC."

[1045] Step 8:

[1046] The server searches the database based on the query generated and retrieves relevant information. The search results include product details and a link to purchase.

[1047] Step 9:

[1048] The server adjusts the search results based on the emotion recognition results. For example, if the user is excited, the latest products or popular items will be given priority.

[1049] Step 10:

[1050] The server organizes the information and sends it to the device. The server organizes the tailored search results and generates an information packet to provide to the user. This information packet includes product details, purchase links, and review links. The organized information is sent to the device.

[1051] Step 11:

[1052] The terminal provides information to the user. The terminal displays search results to the user based on the information packets received from the server. This allows the user to quickly obtain the information they need.

[1053] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1054] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1055] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1056] [Fourth embodiment]

[1057] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1058] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1059] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1060] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1061] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1062] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1063] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1064] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1065] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1066] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1067] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1068] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1069] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1070] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[1071] 1. Getting the input message

[1072] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[1073] 2. Message Analysis

[1074] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[1075] 3. Database Search

[1076] The device generates a search query based on the keywords extracted by the generative AI module and sends it to the server. The server uses this query to search a database and identify relevant information and personnel. For example, guidelines and past project data related to "database design" may be returned as search results.

[1077] 4. Organizing and providing information

[1078] The server receives the search results, organizes them, and generates information packets to provide to the user. The organized information is sent to the user via their device. Specific examples include links to related documents and contact information for the person in charge.

[1079] 5. Processing of follow-up questions

[1080] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query, and the server again searches the database and provides updated information.

[1081] 6. Direct Contact Options

[1082] Additionally, if the user wants more information about a particular piece of information, the device will provide contact information for the identified person so the user can contact them directly, such as the database engineer's email address and phone number.

[1083] Specific examples

[1084] Example 1: Retrieving Database Design

[1085] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[1086] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[1087] 3. Database search: The server searches the database based on keywords to identify relevant documents and people.

[1088] 4. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[1089] 5. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[1090] 6. Direct Contact Option: If necessary, users will be provided with contact information for a database engineer.

[1091] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates information sharing within the company and improves business efficiency.

[1092] The processing flow will be explained below.

[1093] Step 1:

[1094] A user types a question through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[1095] Step 2:

[1096] The terminal receives the user's input message and forwards it to the generative AI module for analysis.

[1097] Step 3:

[1098] The generative AI module analyzes the user's message using natural language processing technology to extract important keywords and topics, such as "database design," "new project," and "optimal method."

[1099] Step 4:

[1100] The generative AI module uses the extracted keywords to generate a query for database search, which is then sent by the server to the database server.

[1101] Step 5:

[1102] The server searches the database server based on the generated query for relevant documents and contact information.

[1103] Step 6:

[1104] The server receives search results from the database server, which include a list of relevant documents and contacts.

[1105] Step 7:

[1106] The server organizes the retrieved information and generates an information packet to provide to the user, which includes a link to the document of the search result and contact information of the person in charge.

[1107] Step 8:

[1108] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[1109] Step 9:

[1110] If the user enters an additional question, the device again sends the new question to the generative AI module, and the next cycle begins. Similarly, the generative AI module analyzes the new question and performs a search again.

[1111] Step 10:

[1112] If the user wishes to contact a specific person directly, the device provides the contact information for the identified person, and the user can contact the person directly using the provided contact information.

[1113] Example 1

[1114] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1115] Conventional information search and inquiry systems have had difficulty efficiently analyzing user input messages and quickly providing relevant information. In particular, they often struggled to accurately extract keywords and provide appropriate information for complex questions in natural language. Furthermore, when users asked follow-up questions, the lack of a seamless system to respond often led to a decline in operational efficiency.

[1116] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1117] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and topics and sending it to the server, means for the server to search a database based on the search query and identify related information and personnel, and means for organizing the identified related information and personnel and generating an information packet to provide to the user. This makes it possible to quickly and accurately provide related information in response to user inquiries in complex natural language, thereby improving business efficiency.

[1118] An "input message" is text about a question or piece of information that a user sends through a chat app or other communication tool.

[1119] "Natural language processing" is a technology that allows computers to understand, interpret, and generate natural human language, and is used to analyze user input messages and extract keywords and topics.

[1120] "Keywords" are important words or phrases extracted from a user's input message and used to generate search queries.

[1121] "Topics" are major themes or subjects extracted from a user's input message and used to generate search queries.

[1122] A "search query" is a query generated based on the extracted keywords and topics, and is used by the server to search the database.

[1123] A "server" is a computer system that maintains a database and is responsible for retrieving relevant information based on a search query from a user.

[1124] A "database" is a collection of information that stores related information and data on personnel.

[1125] An "information packet" is a data structure containing relevant information that is generated by the server to organize and present search results to the user.

[1126] "Related Information" refers to necessary documentation, guidelines, and other reference materials identified based on an analysis of the user's input message and search query.

[1127] A "contact person" is someone with expertise or responsibilities related to the user's question or information, and whose contact information will be provided to the user as needed.

[1128] The system according to the present invention is realized by a program for quickly and efficiently performing user information searches and inquiries. The program for this system will be explained below, along with a processing flow and specific examples.

[1129] First, a user inputs information or a question using a communication tool such as a chat app. For example, they might input, "I would like to know the best way to design a database for a new project." This message is received by the device and recorded as internal data.

[1130] The device then analyzes the received message using natural language processing (NLP) technology, using a generative AI model (e.g., OpenAI's GPT-4) installed on the device. The generative AI model analyzes the message and extracts important keywords and topics. For example, keywords such as "database design," "new project," and "optimal method" are extracted from the input message.

[1131] Based on the extracted keywords, the device generates a search query, which is then sent to the server. The query includes phrases such as "database design," "new project," and "best method."

[1132] The server searches its database based on the received search query. The database contains relevant information and personnel data. The server identifies relevant information and compiles it as search results. For example, it extracts relevant guidelines or past project data.

[1133] The server organizes the search results and generates an information packet to provide to the user. This information packet includes links to related documents and contact information for the person in charge. The server then sends the information packet to the device, which then provides it to the user. For example, the device may display links to related documents in a chat app and send the contact information for the person in charge.

[1134] If the user enters an additional question, the device analyzes the message again and generates a new search query. This query is also sent to the server, which searches the database again. For example, if the user enters "I'd like to know about specific design examples," the information will be provided again.

[1135] Furthermore, if the user needs more information, the device will provide the contact information of the identified person so that the user can contact them directly, for example, providing the database engineer's email address and phone number.

[1136] Specific examples

[1137] Example prompt sentence:

[1138] A user types "Please tell me the best way to design a database for a new project" into a chat app. The device receives the message, and the generative AI model extracts the keywords "database design," "new project," and "best method." The device generates a search query including keywords such as "database design," "new project," and "best method," and sends it to the server. The server searches the database based on the search query and identifies relevant documents and personnel. The server organizes links to relevant documents and contact information for personnel, generates an information packet, and sends it to the device. The device provides the user with the information packet received from the server. If the user additionally types "I would also like to know specific design examples," the device analyzes it again and sends a new search query to the server. If necessary, the device provides the user with contact information for database engineers.

[1139] This system allows users to quickly and efficiently obtain the information they need and connect them with the appropriate experts, facilitating smooth information sharing within the company and improving work efficiency.

[1140] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1141] Step 1:

[1142] A user uses a chat app to enter information or a question.

[1143] What happens: A user types into a chat app, "I'd like to know the best way to design a database for a new project."

[1144] Input: User message: "I'd like to know the best way to design a database for a new project."

[1145] Output: Message data sent to the chat app.

[1146] Step 2:

[1147] The message received by the terminal is analyzed using natural language processing.

[1148] How it works: A generative AI model (e.g., OpenAI's GPT-4) installed on the device analyzes the message and extracts keywords such as "database design," "new project," and "optimal method."

[1149] Input: The user's message data.

[1150] Output: Extracted keywords: "database design", "new project", "best practice".

[1151] Step 3:

[1152] The terminal generates a search query based on the extracted keywords and transmits it to the server.

[1153] Specific operation: The device uses keywords to create a search query in the format "database design new project best method" and sends it to the server.

[1154] Input: extracted keywords "database design", "new project", "best practice".

[1155] Output: Generated search query "database design new project best way".

[1156] Step 4:

[1157] The server searches the database based on the received search query.

[1158] What happens: The server searches its internal database for this search query to identify relevant information (e.g., guidelines, past project data) and responsible parties.

[1159] Input: Received search query "database design new project best way".

[1160] Output: Search results with relevant information and contact data.

[1161] Step 5:

[1162] The server organizes the search results, generates information packets, and sends them to the terminal.

[1163] Specific operation: The server organizes the search results, compiles relevant document links and contact information for the person in charge into an information packet, and sends it to the terminal.

[1164] Input: relevant information and contact data.

[1165] Output: Information packet (document links and contact information).

[1166] Step 6:

[1167] The terminal provides the information packets received from the server to the user.

[1168] What happens: The device interprets the information packet, displays a link to the relevant document in the chat app, and sends the contact information of the person in charge.

[1169] Input: Information packet sent by the server.

[1170] Output: Relevant document links and contact information of the person in charge displayed within the chat app.

[1171] Step 7:

[1172] If the user enters an additional question, the device analyzes the message again and generates a new search query.

[1173] Specific operation: When a user types "I would also like to know about specific design examples," the device analyzes the message, extracts new keywords, generates a search query, and sends it to the server.

[1174] Input: User's additional message: "I would also like to know specific design examples."

[1175] Output: The newly generated search query.

[1176] Step 8:

[1177] The server will then query the database again and provide updated information.

[1178] Specific operation: The server searches the database based on the newly received search query, collects relevant information, and sends it to the terminal.

[1179] Input: A new search query.

[1180] Output: Updated search results (relevant information).

[1181] Step 9:

[1182] The terminal again provides the received information to the user.

[1183] What happens: The device will provide the updated information to the user again, specifically by displaying the new information in the chat app.

[1184] Input: Updated search results (related information).

[1185] Output: Updates displayed within the chat app.

[1186] Step 10:

[1187] If the user wants to know more about a particular piece of information, the terminal will provide contact information for the person in charge.

[1188] Specific operation: Provide the user with contact information (e.g., email address, phone number) of the person whose device was identified.

[1189] Input: Contact information for the person.

[1190] Output: The contact information of the person provided to the user.

[1191] (Application example 1)

[1192] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1193] In today's commercial environment, speed and efficiency of customer service in brick-and-mortar stores is a key challenge. In particular, it is necessary for store associates to provide appropriate information in response to customer questions in real time. However, with current systems, it is difficult for store associates to grasp all product information, and they are often unable to respond immediately when customers request more detailed information. This can lead to lower customer satisfaction and missed sales opportunities.

[1194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1195] In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for generating a search query based on the extracted keywords and acquiring product information based on the created query, and means for providing the acquired product information to the user in an easy-to-understand format. This enables store clerks to provide optimal information in response to customer questions in real time, thereby improving customer satisfaction and maximizing sales opportunities.

[1196] A "user" is a person or customer who inputs information.

[1197] A "message" is text information containing a question or request entered by a user.

[1198] "Natural language processing" is a technology that allows computers to understand and analyze human language.

[1199] "Keywords" are important words extracted from a message.

[1200] A "topic" is an element that indicates the subject or topic of a message.

[1201] A "search query" is a form of question that is generated to search a database.

[1202] "Product information" refers to all product-related information retrieved from a database based on retrieved keywords.

[1203] "Information format" or "easy-to-understand format" refers to a way of presenting information in an organized manner that is easy for users to understand.

[1204] "Contact Person" refers to the individual or department responsible for providing relevant information or support.

[1205] This invention provides a "smart customer service assistant" system for improving customer service in brick-and-mortar stores. Specifically, this system allows store clerks to use smartphones or smart glasses to provide optimal information in response to customer questions in real time. The program processing and implementation method of this system are described in detail below.

[1206] First, the user (store clerk) inputs or speaks a customer's question through a smartphone or smart glasses. The captured message is then received by the device. The device's natural language processing (NLP) module then analyzes the message and extracts important keywords and topics. This can be done using Google Cloud's NLP API.

[1207] Next, a search query is generated based on the extracted keywords. This search query is sent to the store's database and used to search for related product information and contact information. The database contains detailed product information (such as ingredient lists, usage instructions, and warnings) and returns the relevant information.

[1208] The product information obtained as search results is organized in an easy-to-understand format by the terminal and provided to the user. This allows store staff to quickly respond to customer questions. Furthermore, if the user enters additional questions, the same process can be repeated to obtain and provide new information.

[1209] If necessary, users can make further inquiries based on the identified information. For example, if product expertise is required, the app will provide contact information for the customer support center or product development team, allowing users to quickly connect with an expert.

[1210] For example, if a customer asks, "What are the ingredients in this product?" the system will act as follows:

[1211] 1. Obtaining input message: The clerk enters, "Please tell me about the ingredients of this product."

[1212] 2. Message analysis: The device receives the message and the NLP module extracts keywords related to the "product ingredients."

[1213] 3. Database search: Retrieve product ingredient information from the database based on the search query.

[1214] 4. Providing information: The acquired information is provided to the salesperson in an easy-to-understand format. The display reads, "This product contains ingredients A, B, and C."

[1215] An example prompt is:

[1216] A customer asks, "What are the ingredients in this product?" How can I provide the necessary information for this question?

[1217] This invention will improve customer service in brick-and-mortar stores, allowing store associates to quickly and efficiently respond to customer questions, increasing customer satisfaction and maximizing sales opportunities.

[1218] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1219] Step 1:

[1220] The user receives the customer's question through a smartphone or smart glasses. The user inputs or speaks the customer's question into the terminal, which then receives the message. The input message is stored on the terminal as raw text data.

[1221] Step 2:

[1222] The device analyzes the received message. This analysis uses Google Cloud's NLP API to extract important keywords and topics from the message. Specifically, the NLP API parses the input message and identifies notable words such as nouns and verbs. The output is a list of extracted keywords.

[1223] Step 3:

[1224] A search query is generated based on the extracted keywords. The terminal combines the extracted keywords to create a search query for the database. This search query is structured in an appropriate format for retrieving product information. The output is the generated query.

[1225] Step 4:

[1226] The terminal sends the generated search query to the server and searches the database. The server searches the database based on the received query and retrieves the relevant product information. The database used here contains detailed product information (such as ingredient list, usage instructions, and precautions). The output is the product information as a search result.

[1227] Step 5:

[1228] The server then organizes the product information into an easy-to-understand format, including the product name, ingredient list, and usage instructions, and outputs the information as a data packet ready to be delivered to the user.

[1229] Step 6:

[1230] The server sends the organized product information to the terminal. The terminal displays the received information and provides it to the user. The display format can be a visual interface including text and images. This allows the user to quickly respond to the customer. Specifically, it will be displayed in the form of "The ingredients of this product include components A, B, and C."

[1231] Step 7:

[1232] If the user enters a follow-up question, the process from step 1 to step 6 is repeated. The follow-up question is also parsed using natural language processing, a database search is performed, and updated information is provided.

[1233] Step 8:

[1234] If necessary, the server will provide contact information for specific personnel for further inquiries or specialized support, for example, "If you need more information, please contact our support center here," allowing the user to directly contact an expert.

[1235] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1236] The system according to the present invention is realized by a program that allows users to quickly and efficiently search for information and respond to inquiries. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it becomes possible to provide more appropriate search results and personnel. The program for this system is explained below, along with the process flow and specific examples.

[1237] 1. Getting the input message

[1238] First, a user inputs information or a question through a communication tool (e.g., a chat app). This message is received by the device. For example, a user might input, "I would like to know the best way to design a database for a new project."

[1239] 2. Message Analysis

[1240] Next, the device analyzes the received message using natural language processing (NLP). Specifically, a generative AI module analyzes the message and extracts important keywords and topics. For example, keywords and topics such as "database design," "new project," and "best practices" are extracted.

[1241] 3. Emotion recognition

[1242] As the device analyzes the message, the emotion engine recognizes emotions from the user's input. Emotions include joy, sadness, anger, surprise, fear, etc. For example, if the user inputs a message expressing impatience, that emotion will be recognized.

[1243] 4. Database Search

[1244] Based on the keywords extracted by the generative AI module, a query for database search is generated. This query is sent to the database server by the server. The search results may be adjusted taking into account the results of the emotion engine.

[1245] 5. Tailoring search results

[1246] The server searches the database server based on the generated query, and the search results are adjusted based on the emotions recognized by the emotion engine. For example, if the user is feeling anxious, documents with more detailed explanations will be prioritized.

[1247] 6. Organizing and providing information

[1248] The server organizes the information it has acquired and generates an information packet to provide to the user. This packet includes document links for search results and contact information for the agent. The agent recommendations may be adjusted based on the results of the emotion engine.

[1249] 7. Processing of follow-up questions

[1250] If the user enters an additional question, the device again uses the generative AI module to analyze the question and generate a new search query. The server again searches the database and provides updated information. The emotion engine also continues to analyze the input message and make adjustments based on emotion.

[1251] 8. Direct Contact Option

[1252] Additionally, if the user wants to know more about a specific piece of information, the device will provide the contact information of the identified person, allowing the user to contact them directly. For example, the device may provide the email address and phone number of a database engineer. Based on the results of the emotion engine, the device may also recommend a person the user feels comfortable contacting.

[1253] Specific examples

[1254] Example 1: Retrieving Database Design

[1255] 1. Capture input message: A user types in a chat app, "What's the best way to design a database for my new project?"

[1256] 2. Message analysis: The device receives the message, and the generative AI module extracts keywords such as "database design," "new project," and "optimal method."

[1257] 3. Emotion Recognition: The emotion engine recognizes anxiety from the user's message.

[1258] 4. Database search: The server searches the database based on keywords to identify relevant documents and people.

[1259] 5. Tailoring search results: Based on the results of the sentiment engine, documents with detailed descriptions are prioritized.

[1260] 6. Organizing and providing information: The server organizes links to related documents and contact information for responsible parties and provides them to the user.

[1261] 7. Processing additional questions: If the user further requests, "I would like to know more about specific design examples," the information will be searched and provided again.

[1262] 8. Direct Contact Option: If needed, users will be provided with contact information for a database engineer. Based on the results of the sentiment engine, a reassuring contact will be recommended.

[1263] In this way, users can quickly and efficiently obtain the information they need and connect with the appropriate experts. This system facilitates smooth information sharing within the company and improves work efficiency. In addition, the introduction of an emotion engine enables detailed responses tailored to the user's emotional state.

[1264] The processing flow will be explained below.

[1265] Step 1:

[1266] A user types a question or request for information through a communication tool. For example, a user types, "What is the best way to design a database for a new project?"

[1267] Step 2:

[1268] The terminal receives the user's input message and forwards it to the emotion engine and generative AI module.

[1269] Step 3:

[1270] The emotion engine analyzes the user's input message and recognizes emotions, for example, "anxiety" from the user's message.

[1271] Step 4:

[1272] The generative AI module analyzes the content of the user's message using natural language processing technology and extracts important keywords and topics, such as "database design," "new project," and "optimal method."

[1273] Step 5:

[1274] The generative AI module generates search queries based on the extracted keywords. These queries also incorporate the results of the emotion engine and are adjusted to prioritize information that takes the user's emotions into consideration.

[1275] Step 6:

[1276] The server uses the generated search query to search the database server for related documents and contact information.

[1277] Step 7:

[1278] The server receives the search results from the database and organizes them, prioritizing documents with detailed descriptions that reduce the user's anxiety based on an emotion engine.

[1279] Step 8:

[1280] The server then uses the organized information to generate an information packet to present to the user, which includes links to related documentation and contact information for the appropriate person.

[1281] Step 9:

[1282] The terminal sends the generated information packet to the user, who then checks the required information through the provided link or contact information.

[1283] Step 10:

[1284] If the user inputs an additional question, the device again sends the new question to the emotion engine and generative AI module, which continues to analyze the input message and make emotion-based adjustments.

[1285] Step 11:

[1286] The generative AI module analyzes the additional questions and generates new search queries, and the server again searches the database and provides updated information.

[1287] Step 12:

[1288] If the user wants to contact a specific agent directly, the device will provide the contact information of the identified agent, and based on the results of the emotion engine, a reassuring agent will be recommended.

[1289] Step 13:

[1290] The user can use the contact information provided to directly contact the appropriate person, for example, by providing an email address or phone number for the person.

[1291] Example 2

[1292] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1293] Conventional information retrieval systems only extract keywords from messages entered by users and perform database searches, but they have the problem of being unable to respond to the user's emotional state. Therefore, there is a need for a system that can respond in a detailed manner according to the user's emotional state.

[1294] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1295] In this invention, the server includes means for acquiring a message input by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, means for recognizing emotions from the user's input message, means for searching a database based on the extracted keywords and topics and the recognized emotions to identify related information and a person in charge, and means for providing the identified related information and a person in charge to the user, thereby enabling detailed information provision that takes into account the user's emotional state.

[1296] "User" refers to the end user who inputs information and uses the system.

[1297] "Message" refers to text data that a user sends to the system to display information or ask a question.

[1298] "Natural language processing" refers to the technology that enables computers to understand, analyze, and process human language.

[1299] "Keywords" refer to words that have a major meaning in message analysis.

[1300] "Topic" refers to the concept of the main topic or theme of a message.

[1301] "Emotion" refers to the psychological state perceived from a user's message.

[1302] "Database" refers to a data storage system for systematically storing and retrieving information.

[1303] "Contact Person" refers to a person designated to respond to user inquiries about specific information.

[1304] "Search" refers to the process of finding specific information from stored data.

[1305] "Contact Information" refers to information such as telephone numbers and email addresses used to contact a person in charge.

[1306] The system according to the present invention acquires a message input by a user and realizes efficient and appropriate information retrieval by combining natural language processing and emotion recognition. Specific embodiments of this system will be described below.

[1307] First, a user enters information or a question using a chat app or other communication tool. This sends a message to the device and receives it. For example, the message might be something like, "What's the best way to design a database for a new project?" The device then prepares the message to be saved in the database.

[1308] The device then analyzes the received message using a natural language processing (NLP) module. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-3) to extract important keywords from the message, such as "database design," "new project," and "optimal method." This generative AI model has advanced natural language understanding capabilities and can extract keywords for a variety of topics.

[1309] Furthermore, the device uses an emotion engine (e.g., Affectiva's emotion recognition API) to analyze emotions from the user's input messages. The emotion engine detects emotions such as impatience or anxiety from the user's messages and records the results in a log. This information plays an important role in subsequent data retrieval and result adjustment.

[1310] Next, the server generates a database query based on the analysis results and keywords sent from the device. The generated query is sent to the database server, which searches for relevant information. At this stage, the emotion recognition results are taken into account and information that matches the user's emotions is searched for with priority. This process makes it possible to provide appropriate information according to the user's emotional state.

[1311] The server analyzes the search results retrieved from the database and adjusts them based on the emotion engine's recognition results. For example, if the user expresses anxiety, the server adjusts the results to prioritize detailed explanatory documents and support information.

[1312] The server then generates an information packet based on the adjusted search results, which includes links to related documents and contact information for the relevant person. The server also recommends the most appropriate contact information for the user based on the results of the emotion engine.

[1313] If the user has further questions, for example, by entering "I would also like to know about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server again searches the database. In the same way, it provides the user with search results according to their emotions.

[1314] Finally, if the user wants to know more about a particular piece of information, the device will provide contact information for the identified person, such as the email address and phone number of a database engineer. Based on the results of the emotion engine, the device will recommend the appropriate person for the user to contact.

[1315] As a concrete example, consider the prompt, "What is the best way to design a database for a new project?" Based on this prompt, a generative AI model extracts keywords, and an emotion engine recognizes the user's emotions, then provides the appropriate information and contact information for the appropriate person. This process allows users to quickly and efficiently obtain the information they need and connect with the appropriate expert. This system facilitates information sharing within a company and improves business efficiency.

[1316] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1317] Step 1:

[1318] The user inputs information or a question. The user uses a chat app or other communication tool to type, "What is the best way to design a database for a new project?" The device receives this message and stores the input message in an internal buffer.

[1319] Input: User's message

[1320] Output: Internal buffer storing received messages

[1321] Step 2:

[1322] The device then analyzes the received message using a natural language processing module, which uses a generative AI model (e.g., GPT-3) to analyze the message and extract keywords such as "database design," "new project," and "optimal method."

[1323] Input: Received message

[1324] Output: Extracted keywords

[1325] Step 3:

[1326] The device uses an emotion engine to recognize emotions from the user's input messages. This emotion engine (for example, Affectiva's emotion recognition API) analyzes the user's messages, detects emotions such as impatience or anxiety, and records the results in a log.

[1327] Input: Received message

[1328] Output: Recognized emotion

[1329] Step 4:

[1330] The server generates a database query based on the analysis results (extracted keywords) and emotion recognition results sent from the device. This generated query is then sent to the database server. Here, the emotion recognition results are also taken into consideration, and the query is constructed so that information appropriate to the user's emotion is searched for with priority.

[1331] Input: extracted keywords and recognized emotions

[1332] Output: Generated database query

[1333] Step 5:

[1334] The server sends the generated query to a database server to search for relevant information, and the database accepts the query, performs the search, and returns relevant documents and information.

[1335] Input: Generated database query

[1336] Output: Search results (related documents and information)

[1337] Step 6:

[1338] The server analyzes the search results retrieved from the database and adjusts the results based on the emotion engine's recognition. For example, if the user is feeling anxious, the server will prioritize documents with detailed explanations. It will also recommend contact details based on the user's emotion.

[1339] Input: Search results and recognized emotions

[1340] Output: Refined search results

[1341] Step 7:

[1342] The server generates an information packet to provide to the user based on the adjusted search results. This information packet includes links to related documents, detailed descriptions of the emotions, and contact information for the person in charge. The server then sends this information to the terminal.

[1343] Input: Refined search results

[1344] Output: Generated information packet

[1345] Step 8:

[1346] The device provides the generated information packet to the user. If the user enters a further question, such as "I would like to know more about specific design examples," the device again uses the generative AI module and emotion engine to generate a new search query, and the server searches the database again.

[1347] Input: Generated information packet, user's additional questions

[1348] Output: Information provided to the user, new search query

[1349] Step 9:

[1350] If the user wants to contact the person directly, the device will provide the contact information of the identified person, such as the email address and phone number of a database engineer. Based on the emotion recognition results, it will recommend a person who is comfortable contacting the person.

[1351] Input: User request, identified person

[1352] Output: Provide contact information for the person in charge

[1353] (Application example 2)

[1354] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1355] Conventional information retrieval systems cannot consider the user's emotional state when providing appropriate answers to user-entered questions or inquiries, which can lead to reduced user satisfaction. Furthermore, search results and information provided are uniform, making it difficult to provide information tailored to a user's specific needs and circumstances. Therefore, there is a need for a system that recognizes the user's emotional state and provides appropriate information.

[1356] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring a message entered by a user, means for analyzing the acquired message using natural language processing and extracting keywords and topics, emotion recognition means for recognizing the user's emotions during the message analysis process, means for searching a database based on the extracted keywords and topics and identifying related information and personnel, means for adjusting search results and information based on the results of emotion recognition, and means for providing the user with the identified related information and personnel. This makes it possible to provide detailed information that takes into account the user's search purpose and emotional state.

[1357] The "means for acquiring a message entered by a user" is a function for acquiring a message entered by a user through a communication tool.

[1358] "Natural language processing" is a technology that analyzes text messages and understands their meaning and context.

[1359] The "means for extracting keywords and topics" is a function for extracting important words and topics from messages.

[1360] "Means for searching a database and identifying relevant information and personnel" refers to a function that searches a database based on extracted keywords or topics to find relevant information and appropriate personnel.

[1361] The "means for providing the user with the identified related information and person in charge" is a function for providing the user with details of the searched information and person in charge.

[1362] "Emotion recognition means" is a technology for recognizing emotions from a user's input message.

[1363] "Means for adjusting search results and information" refers to a function that appropriately adjusts the information provided and search results based on the results of emotion recognition.

[1364] The present invention is a system that analyzes a message input by a user and provides appropriate information based on the message. The processing contents of a specific program for implementing the present invention will be described below.

[1365] The system first captures messages entered into a chat box by users on devices such as smartphones. These messages are then analyzed by a natural language processing (NLP) engine to extract important keywords and topics. This analysis uses generative AI models such as OpenAI's GPT-3.

[1366] The emotion engine then recognizes the user's emotions during message analysis. This emotion recognition uses emotion analysis libraries such as VADER. For example, if a user types, "What is the best model of a new smartphone?", the NLP engine extracts keywords such as "smartphone" and "best model," and the emotion engine recognizes that the user is excited.

[1367] Based on the extracted keywords and the emotion recognition results, a query is generated to search the database. This query is sent to the server, which then searches the database. The search results are adjusted based on the emotion recognition results. For example, if the user is excited, the newest or most popular products will be displayed preferentially.

[1368] The resulting search results are then organized and provided to the user, including product details, purchase links, and review links. If the user enters additional questions, the generative AI model is used again to analyze the question, generate a new search query, and provide updated information.

[1369] Additionally, the contact information of the identified representatives is also provided to the user, allowing the user to directly contact the customer service representative if necessary. Based on the results of emotion recognition, appropriate representatives are recommended.

[1370] As a specific example, consider the following user message:

[1371] For example: "What is the best model of the new smartphone?"

[1372] The prompt to input this to the generative AI model is as follows:

[1373] "Extract keywords from this user message: What's the best new smartphone model?"

[1374] This system is expected to improve user satisfaction by providing detailed information that takes into account the user's search purpose and emotional state. In addition, the emotion recognition function will recommend a more appropriate representative, allowing users to receive the support they need quickly.

[1375] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1376] Step 1:

[1377] The user enters a message. The user enters a question or inquiry into the smartphone's chat box. An example of an input is "What is the best model of new smartphone?" This message is sent to the device.

[1378] Step 2:

[1379] The terminal acquires the message. The communication module of the terminal receives the message sent by the user. This acquired message becomes the input for the next analysis step.

[1380] Step 3:

[1381] The device passes the message to the natural language processing engine. The device passes the message to the NLP engine and begins analysis. Specifically, the device sends a message to the API of a generative AI model (e.g., GPT-3) and prompts it to extract keywords and topics.

[1382] Step 4:

[1383] The NLP engine extracts keywords and topics. The generative AI model analyzes the message and extracts important keywords and topics. For example, the keywords "new smartphone" and "best model" are extracted from the message "What is the best new smartphone model?". This extraction result becomes the input for the next emotion recognition step.

[1384] Step 5:

[1385] The device passes the message to the emotion recognition engine. The device also passes the message it has acquired at the same time to the emotion recognition engine (e.g., VADER) and instructs it to analyze the user's emotions.

[1386] Step 6:

[1387] The emotion recognition engine recognizes emotions. The emotion recognition engine analyzes the message and recognizes the user's emotion. For example, the emotion "excited" is recognized from the user's message. This emotion recognition result is used to adjust the next database search.

[1388] Step 7:

[1389] The server generates a query for database search. The device sends the extracted keywords and emotion recognition results to the server, and the server generates a query for database search based on them. For example, the generated query might be in the form of "New smartphone and best model ORDER BY relevance_with_excitement DESC."

[1390] Step 8:

[1391] The server searches the database based on the query generated and retrieves relevant information. The search results include product details and a link to purchase.

[1392] Step 9:

[1393] The server adjusts the search results based on the emotion recognition results. For example, if the user is excited, the latest products or popular items will be given priority.

[1394] Step 10:

[1395] The server organizes the information and sends it to the device. The server organizes the tailored search results and generates an information packet to provide to the user. This information packet includes product details, purchase links, and review links. The organized information is sent to the device.

[1396] Step 11:

[1397] The terminal provides information to the user. The terminal displays search results to the user based on the information packets received from the server. This allows the user to quickly obtain the information they need.

[1398] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1399] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1400] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1401] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1402] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1403] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1404] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1405] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1406] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1407] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1408] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1409] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1410] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1411] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1412] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1413] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1414] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1415] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1416] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1417] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1418] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1419] The following is further disclosed regarding the above embodiment.

[1420] (Claim 1)

[1421] means for obtaining a message entered by a user;

[1422] A means for analyzing the acquired messages by natural language processing and extracting keywords and topics;

[1423] A means for searching a database based on the extracted keywords and topics to identify relevant information and personnel;

[1424] A system including a means for providing identified relevant information and personnel to a user.

[1425] (Claim 2)

[1426] a means for the user to input a follow-up question;

[1427] 10. The system of claim 1, further comprising means for searching the database again based on an additional query to provide updated information.

[1428] (Claim 3)

[1429] 10. The system of claim 1, further comprising means for providing contact information for the identified personnel so that the user can contact them directly.

[1430] "Example 1"

[1431] (Claim 1)

[1432] means for obtaining a message entered by a user;

[1433] A means for analyzing the acquired messages by natural language processing and extracting keywords and topics;

[1434] means for generating a search query based on the extracted keywords and topics and transmitting the search query to a server;

[1435] a means for the server to search a database based on the search query to identify relevant information and personnel;

[1436] The system includes means for organizing the identified relevant information and personnel and generating an information packet for presentation to the user.

[1437] (Claim 2)

[1438] a means for the user to input a follow-up question;

[1439] 10. The system of claim 1, further comprising means for searching the database again based on an additional query to provide updated information.

[1440] (Claim 3)

[1441] 10. The system of claim 1, further comprising means for providing contact information for the identified personnel so that the user can contact them directly.

[1442] "Application Example 1"

[1443] (Claim 1)

[1444] means for obtaining a message entered by a user;

[1445] A means for analyzing the acquired messages by natural language processing and extracting keywords and topics;

[1446] A means for searching a database based on the extracted keywords and topics to identify relevant information and personnel;

[1447] 1. A system including a means for providing identified relevant information and personnel to a user,

[1448] Furthermore, a means for generating a search query based on the extracted keywords and acquiring product information based on the generated query;

[1449] A means for providing the acquired product information to the user in an easy-to-understand format;

[1450] A system including:

[1451] (Claim 2)

[1452] a means for the user to input a follow-up question;

[1453] and further including means for searching the database again based on additional questions and providing updated information.

[1454] 10. The system of claim 1.

[1455] (Claim 3)

[1456] providing contact information for the identified personnel and further including a means for the user to contact them directly;

[1457] 10. The system of claim 1.

[1458] "Example 2: Combining Emotion Engines"

[1459] (Claim 1)

[1460] means for obtaining a message entered by a user;

[1461] A means for analyzing the acquired messages by natural language processing and extracting keywords and topics;

[1462] means for recognizing emotions from a user's input message;

[1463] A means for searching a database based on the extracted keywords and topics and the recognized sentiment to identify relevant information and personnel;

[1464] A system including a means for providing identified relevant information and personnel to a user.

[1465] (Claim 2)

[1466] a means for the user to input a follow-up question;

[1467] 10. The system of claim 1, further comprising means for searching the database again based on an additional query to provide updated information.

[1468] (Claim 3)

[1469] 10. The system of claim 1, further comprising means for providing contact information for the identified personnel so that the user can contact them directly.

[1470] "Application example 2 when combining emotion engines"

[1471] (Claim 1)

[1472] means for obtaining a message entered by a user;

[1473] A means for analyzing the acquired messages by natural language processing and extracting keywords and topics;

[1474] A means for searching a database based on the extracted keywords and topics to identify relevant information and personnel;

[1475] means for providing the identified relevant information and personnel to the user;

[1476] emotion recognition means for recognizing a user's emotion in the process of analyzing the message;

[1477] The system includes a means for tailoring search results or information based on the results of emotion recognition.

[1478] (Claim 2)

[1479] a means for the user to input a follow-up question;

[1480] 10. The system of claim 1, further comprising means for searching the database again based on an additional query to provide updated information.

[1481] (Claim 3)

[1482] 10. The system of claim 1, further comprising means for providing contact information for the identified personnel so that the user can contact them directly. [Explanation of symbols]

[1483] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. means for obtaining a message entered by a user; A means for analyzing the acquired messages by natural language processing and extracting keywords and topics; A means for searching a database based on the extracted keywords and topics to identify relevant information and personnel; A system including a means for providing identified relevant information and personnel to a user.

2. a means for the user to input a follow-up question; 10. The system of claim 1, further comprising means for searching the database again based on additional queries to provide updated information.

3. The system of claim 1 , further comprising means for providing contact information for the identified personnel so that the user can contact them directly.

Images