system

Abstract

We provide the system. [Solution] A means for storing knowledge data and service information in a database, A means for constructing a generative AI that learns knowledge data and service information, A means of receiving questions from users, A means of analyzing the received question and generating an answer using a generative AI, A means of sending the generated response to the user, A means of displaying submitted responses to the user, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of the chatbot's character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Modern salespersons, especially new salespersons, need to quickly acquire knowledge about complex technologies and services. However, with conventional education methods and manuals, it may be difficult to obtain the necessary information quickly and accurately, resulting in a problem of reduced efficiency in sales activities. In particular, the inability to quickly respond to questions regarding technical backgrounds and detailed service information has an adverse effect on the quality of customer service and, ultimately, on sales results.

Means for Solving the Problems

[0005] To solve this problem, the present invention provides the following means. First, it includes means for storing knowledge data and service information in a database. Next, it includes means for constructing a generative AI that learns the knowledge data and service information. Furthermore, it includes means for receiving questions from users and means for analyzing the received questions and generating answers using the generative AI. It also includes means for sending the generated answers to users and means for displaying the sent answers to users. As a result, sales representatives can quickly and accurately obtain the necessary information, improve the quality of customer service, and improve sales results.

[0006] A "database" is a system that systematically manages electronically stored information and allows for quick access when needed.

[0007] "Knowledge data" refers to knowledge information such as technical documents, FAQ data, and training materials that are collected and managed within a company.

[0008] "Service information" refers to detailed descriptions, specifications, and usage instructions for the products and services offered.

[0009] "Generative AI" refers to artificial intelligence systems that learn from large amounts of data and generate answers in natural language to questions.

[0010] "A means of receiving questions from users" refers to an interface that incorporates questions entered by users into the system.

[0011] "Methods for analyzing questions" refer to the process of understanding the content of a user's question and extracting the information necessary to generate an appropriate answer.

[0012] "Methods for generating answers using generative AI" refer to the process of creating answers based on analyzed questions, utilizing learned knowledge data and service information.

[0013] "Means for sending the response to the user" refers to a communication method for sending the generated response to the user's device.

[0014] "Means of displaying responses to the user" refers to the process of outputting submitted responses to a display device so that the user can visually confirm them. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13]It is a sequence diagram showing the processing flow of the data processing system in Embodiment 2 when combined with an emotion engine. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when combined with an emotion engine.

Embodiments for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] This invention relates to a sales representative support system utilizing generative AI, and is particularly aimed at enabling new sales representatives to quickly acquire necessary knowledge. Specific embodiments for carrying out this invention are described below.

[0037] Data Ingestion

[0038] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[0039] Data Learning

[0040] The server retrieves knowledge data and service information stored in the database and provides it to the generative AI. The AI ​​learns from this data and becomes able to generate appropriate answers to various questions from users. This generative AI utilizes natural language processing technology to analyze the content of the questions and provide the optimal answer.

[0041] Questions accepted

[0042] The terminal has an interface for receiving questions from the user. The user inputs the question into the terminal in natural language, and the terminal sends the question to the server. This allows the user to quickly ask for the information they need.

[0043] Question Processing

[0044] The server receives questions from users and analyzes them. The analyzed questions are passed to a generative AI, which generates answers based on knowledge data and service information. In this answer generation process, the AI ​​utilizes its learned knowledge to provide the most appropriate information for the question.

[0045] Submitting and displaying responses

[0046] The generated answers are sent from the server to the user's device. The device receives the sent answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[0047] Specific example

[0048] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?" The terminal receives the question and sends it to the server. The server uses generative AI to analyze the question and generates the best answer based on knowledge data and service information in its database. The answer is, "The new product X has high processing power, excellent security features, and a user-friendly interface." The server sends this answer to Tanaka's terminal, where Tanaka can check the answer.

[0049] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[0050] The following describes the processing flow.

[0051] Step 1:

[0052] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database. The server structures the collected data and stores it in the database for quick access.

[0053] Step 2:

[0054] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds knowledge to generate appropriate answers to user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[0055] Step 3:

[0056] The user enters a question into the terminal. The terminal receives this input and sends the question to the server. The terminal provides an interface that accepts natural language input from the user and sends the question data, ready to be sent, to the server.

[0057] Step 4:

[0058] The server receives questions sent from terminals. Next, the server passes the received questions to a generative AI for analysis. The generative AI understands the content of the questions and generates the optimal answer based on knowledge data and service information in the database.

[0059] Step 5:

[0060] The server receives a response from the generative AI. This response provides specific and appropriate information in response to the user's question. The server then performs the necessary transmission processing to send this response to the user's terminal.

[0061] Step 6:

[0062] The terminal receives the response sent from the server. The terminal then visually displays this response to the user. The user can then review the displayed response and obtain information regarding their question.

[0063] Specific example

[0064] Step 1:

[0065] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[0066] Step 2:

[0067] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[0068] Step 3:

[0069] User Tanaka enters the question "What are the main features of the new product X?" into the terminal.

[0070] Step 4:

[0071] The terminal sends Tanaka's question to the server. The server receives the question, sends it to a generative AI, and analyzes it.

[0072] Step 5:

[0073] The generative AI generates the response, "New product X has high-speed processing capabilities, excellent security features, and a user-friendly interface." The server receives this response and sends it to Tanaka's terminal.

[0074] Step 6:

[0075] Tanaka's device receives the response from the server and displays the content visually to Tanaka. Tanaka checks the response regarding the main features of the new product X.

[0076] In this way, the present invention provides an efficient means for sales representatives to quickly and accurately acquire the necessary knowledge, thereby improving the quality of customer service.

[0077] (Example 1)

[0078] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0079] New sales representatives are required to acquire necessary knowledge quickly and efficiently to improve the quality of their customer service. However, traditional methods require them to individually search and understand vast amounts of knowledge data and product information, which is time-consuming and laborious. A system is needed to solve this problem.

[0080] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0081] In this invention, the server includes means for storing knowledge data and product information in a database, means for building a generative AI model that learns the knowledge data and product information, means for receiving user questions in natural language, means for analyzing the received questions and generating answers using the generative AI model, means for sending the generated answers to the user's terminal, and means for visually displaying the transmitted answers to the user. This enables new sales representatives to efficiently acquire necessary information and respond to customers quickly and accurately.

[0082] A "database" is an information storage system that efficiently manages and makes searchable knowledge data and product information.

[0083] "Knowledge data" refers to all data containing specific knowledge and information within a company, such as technical documents, FAQ data, and training materials.

[0084] "Product information" refers to detailed data about products and services offered by a company, including technical specifications, functions, and features.

[0085] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on knowledge data and product information to generate appropriate answers to questions.

[0086] A "user" refers to the entity that uses the system to input questions and receive answers. This is primarily intended for new sales representatives.

[0087] "Natural language" refers to the language that humans use on a daily basis, and specifically to the question format that users input into the system.

[0088] "Analysis" refers to the process of understanding the received question, extracting its contents, and organizing them.

[0089] "Answer generation" refers to the process of creating appropriate answers using a generative AI model based on the analyzed question content.

[0090] A "device" refers to a device used by a user to input questions and receive answers. This includes PCs and tablets.

[0091] "Visual display" refers to displaying the generated response on the device screen in a way that the user can see and understand.

[0092] This invention is a system that supports new sales representatives using a generative AI model. In particular, it aims to enable the rapid and accurate acquisition of necessary information by efficiently utilizing knowledge data and product information. Specific embodiments for carrying out this invention are described below.

[0093] The server periodically collects knowledge data such as technical documents, FAQ data, and training materials within the company, as well as detailed information about the products and services the company provides, and stores it in a database (e.g., MySQL® or PostgreSQL). This data later serves as foundational data for training generative AI models (e.g., OpenAI®'s GPT-3®).

[0094] The server provides the AI ​​model with knowledge data and product information stored in the database, and this AI model learns from this data. This allows it to generate appropriate answers to user questions.

[0095] The user (new sales representative) uses a provided device (e.g., a PC or tablet) to input a question in natural language. The device then sends the question to the server.

[0096] The server receives questions sent from terminals and analyzes them using a generative AI model. Based on the analyzed question, the AI ​​model refers to knowledge data and product information in the database to generate the optimal answer. This answer generation process fully utilizes the knowledge the AI ​​model has learned.

[0097] The generated answers are sent from the server to the user's device, which receives the answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[0098] As a concrete example, consider a scenario where a new sales representative enters the question, "What are the main features of the new product X?" into a terminal. In this case, the terminal sends the question to a server, which uses a generative AI model to analyze the question. As a result, an answer is generated, such as, "The new product X has high-speed processing capabilities, excellent security features, and a user-friendly interface," and this answer is sent to the user's terminal. The user can then view the answer on their terminal.

[0099] The following is an example of a prompt message.

[0100] In response to the question, "What are the main features of the new product X?", generate a detailed description of the features of product X.

[0101] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[0102] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0103] Step 1: Data Ingest

[0104] The server collects technical documents, FAQ data, training materials, and detailed information about products and services within the company. This data is periodically stored in a database by the server. The input consists of various documents and data within the company, while the output is organized knowledge data and product information stored in the database.

[0105] Step 2: Data Training

[0106] The server provides a generative AI model with knowledge data and product information stored in a database. The generative AI model (e.g., GPT-3) learns from this data. The input is knowledge data and product information in the database, and the output is a generative AI model capable of generating appropriate answers to questions.

[0107] Step 3: Question Acceptance

[0108] The user uses a terminal to input a question in natural language. The terminal sends this question to the server. The input is the user's question in natural language, and the output is that the question has been sent to the server.

[0109] Step 4: Questionnaire Analysis

[0110] The server receives questions sent from the terminal and analyzes them using a generative AI model. The input is the user's question, and the output is the analyzed question. The server uses natural language processing techniques to properly understand the questions.

[0111] Step 5: Generate Response

[0112] The server uses a generative AI model to generate the optimal answer based on the analyzed question content. The input consists of the analyzed question content, knowledge data, and product information, while the output is the generated answer. The generated answer is based on the knowledge learned by the AI ​​model.

[0113] Step 6: Submit your response

[0114] The server sends the generated response to the user's terminal. The input is the generated response, and the output is that the response is sent to the terminal.

[0115] Step 7: Display your answer

[0116] The terminal receives the response sent from the server and displays it visually to the user. The input is the response sent from the server, and the output is the response displayed on the terminal's screen. The user checks the response on the terminal and obtains the necessary information.

[0117] (Application Example 1)

[0118] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0119] In conventional factory equipment maintenance, it was difficult for workers to quickly and accurately obtain the necessary information. In particular, new workers were unfamiliar with equipment maintenance procedures and specific operating methods, and it took time for them to acquire the necessary knowledge. This led to decreased efficiency in maintenance work and an increased risk of equipment failure due to improper operation.

[0120] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0121] In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for receiving questions from users, means for analyzing the received questions and generating answers using the generative AI, means for sending the generated answers to the user, means for displaying the sent answers to the user, and means for receiving questions and displaying answers to robots operating in the factory. This makes it possible to quickly and accurately acquire information necessary for equipment maintenance work in the factory and improve work efficiency.

[0122] A "database" is a system for systematically accumulating and managing information, and it plays a role in storing knowledge data and service information.

[0123] "Knowledge data" refers to a collection of data containing specialized knowledge and information within a company, such as technical documents and FAQ data.

[0124] "Service information" refers to a collection of data containing detailed information about the products and services offered by a company.

[0125] "Generative AI" is artificial intelligence that learns from knowledge data and service information to generate the best possible answers to user questions.

[0126] "Means for receiving questions from users" refers to devices or systems that have an interface that allows users to input questions in natural language.

[0127] "Means for analyzing questions and generating answers using generative AI" refers to devices or systems that analyze questions received from users and execute a process to generate the most appropriate answer to those questions.

[0128] "Means for sending generated answers to users" refers to communication methods for providing users with answers generated by generative AI.

[0129] "Means for displaying submitted responses to the user" refers to devices or systems that allow the user to visually confirm the responses provided.

[0130] "Robots operating in factories" are robots designed to perform tasks in factory manufacturing areas or equipment maintenance sites.

[0131] "Means for receiving questions and displaying answers" refers to devices or systems that have the function of receiving questions from users and displaying corresponding answers, such as robots in a factory.

[0132] This invention provides a system for supporting equipment maintenance work within a factory. Specific embodiments are described below.

[0133] First, the server periodically collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores this data in a database. This database serves as the foundational data for generative AI to learn from. The hardware used will consist of a server computer and a large-capacity storage device. The specific software used will include a database management system (DBMS) and data collection tools.

[0134] Next, the server builds a generative AI model based on the knowledge data and service information stored in the database. This model uses OpenAI's GPT-3 or other natural language processing models. This enables the AI ​​to generate appropriate answers to various questions from the user.

[0135] Users (in this case, factory workers) input questions through robots operating within the factory. The robots accept questions in natural language and send them to a server. The robots are equipped with a voice recognition system and a touchscreen, allowing users to input questions by voice or text.

[0136] When the server receives a question, it analyzes the question using natural language processing technology and passes it on to a generative AI. The AI ​​generates the optimal answer based on pre-trained knowledge data and service information. This process utilizes natural language processing technology to understand the meaning of the question and select the appropriate answer.

[0137] The generated response is sent from the server to the robot. The robot then presents this response to the user visually or audibly. For example, if a worker asks, "How do I change the oil in a machine tool?", the robot will display or voice the response it received from the server, saying, "The oil change is performed using the following steps..."

[0138] As a concrete example, if a new worker in a factory asks a question about how to inspect a conveyor belt, the following exchange may occur:

[0139] Question: How do you inspect a conveyor belt?

[0140] Answer: Belt conveyor inspection is performed using the following procedure:

[0141] 1. Stop the machine and turn off the power.

[0142] 2. Check the tension of the belt.

[0143] 3. Visually inspect the belt for any abnormalities.

[0144] 4. If any abnormalities are found, appropriate repairs will be carried out.

[0145] 5. After the inspection, restart the machine and check if it is working correctly.

[0146] In this way, by providing the necessary information for equipment maintenance work within the factory quickly and accurately, work efficiency can be improved.

[0147] Furthermore, the generated responses are recorded as logs on the server and used for future improvements and evaluations. This system is expected to enable new workers to quickly acquire the necessary knowledge and significantly improve the efficiency of maintenance work within the factory.

[0148] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0149] Step 1:

[0150] The server collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores it in a database. This collection process is performed regularly, and the information is organized using a database management system (DBMS). The input is internal company data, and the output is a structured database.

[0151] Step 2:

[0152] The server extracts data from knowledge data and service information stored in the database and provides it to a generative AI (in this case, OpenAI's GPT-3). The AI ​​learns from this data and becomes capable of operating as a natural language processing model. The input is knowledge data from the database, and the output is the trained generative AI model. Specifically, the process involves data formatting and feeding the data to the AI ​​model.

[0153] Step 3:

[0154] Users input questions to factory robots via voice or text. The robots receive user questions using a voice recognition system or touchscreen. The input is the user's question, and the output is question data in digital format. Specifically, voice recognition processing is performed to convert speech to text.

[0155] Step 4:

[0156] The robot sends the received question to the server. The server receives this question and analyzes it using natural language processing technology. The input is the question data sent by the robot, and the output is the analyzed question data. Specifically, the operation includes semantic analysis of the question content and extraction of related data.

[0157] Step 5:

[0158] The server passes the analyzed question to the generative AI, which generates the optimal answer based on knowledge data and service information. The input is the analyzed question data, and the output is the generated answer. Specifically, the AI ​​model generates the answer. Example of a prompt: "Question: How do I change the oil in a machine tool? Answer: The oil change is performed using the following steps..."

[0159] Step 6:

[0160] The generated response is sent from the server to the robot. The robot receives this response and presents it to the user visually or audibly. The input is the response data from the server, and the output is a display or audio output in a format that the user can see. Specific actions include displaying the response on a screen and outputting audio from a speaker.

[0161] Step 7:

[0162] The user reviews the responses provided by the robot and performs the necessary maintenance tasks. The input is the robot's response information, and the output is the user's actions. Specific actions include the user performing maintenance tasks.

[0163] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0164] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions. Specific embodiments for carrying out this invention are described below.

[0165] Data Ingestion

[0166] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[0167] Data Learning

[0168] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to various user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[0169] Emotional engine integration

[0170] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. Specifically, when the user inputs a question through their device, the engine analyzes their emotions in real time using voice and camera data. The analysis results are fed back to a generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0171] Questions accepted

[0172] The device has an interface for receiving questions from the user. The user inputs questions into the device using natural language, and the device sends those questions to the server. It also collects the user's voice and camera footage in real time and sends it to the emotion engine.

[0173] Question Processing

[0174] The server receives questions from users and passes them to a generative AI for analysis. In parallel, the emotion engine analyzes audio and video data from the user and evaluates the user's emotional state. The obtained emotional information is fed back to the generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0175] Submitting and displaying responses

[0176] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[0177] Specific example

[0178] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?", and simultaneously reads the question aloud. The terminal sends the question along with Tanaka's voice data to the server. The server uses generative AI to analyze the question and an emotion engine to evaluate Tanaka's emotion from his voice tone. Based on knowledge data and service information in the database, the generative AI generates the answer, "The new product X has high processing power and excellent security features, and provides a user-friendly interface." Based on feedback from the emotion engine, the tone of the answer is adjusted to match Tanaka's emotion. The server sends the adjusted answer to Tanaka's terminal, where Tanaka confirms the answer.

[0179] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[0180] The following describes the processing flow.

[0181] Step 1:

[0182] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database, which provides the foundational data for generative AI to learn from.

[0183] Step 2:

[0184] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to user questions. In the learning process, natural language processing techniques are used to analyze the data and recognize patterns that fit the questions.

[0185] Step 3:

[0186] The server prepares to utilize the emotion engine. The emotion engine analyzes the user's voice data and camera footage to recognize the user's emotional state. The emotion engine evaluates emotions in real time based on voice tone and facial expressions, and feeds the obtained information back to the generative AI.

[0187] Step 4:

[0188] The user inputs a question into the device using natural language. The device provides an interface for inputting questions and receives them using text boxes or voice input functions.

[0189] Step 5:

[0190] The device sends the user's question text to the server. Simultaneously, if voice input is used, the voice data is also sent to the server. Furthermore, it may also send user facial expression data obtained from the camera feed.

[0191] Step 6:

[0192] The server receives question text, audio data, and facial expression data sent from the terminal. First, the received question text is passed to a generative AI to analyze the question. Next, the emotion engine analyzes the audio data and facial expression data to evaluate the user's emotional state. For example, it identifies emotions such as joy, sadness, surprise, and anxiety.

[0193] Step 7:

[0194] The generative AI searches the database for appropriate knowledge data and service information based on the analyzed question content and generates an answer. At the same time, it receives emotion evaluation data from the emotion engine and adjusts the tone and content of the answer according to the user's emotions. For example, if the user is agitated, it will adjust the answer to be provided in a calm tone.

[0195] Step 8:

[0196] The server sends the generated response to the terminal. The adjusted response data is sent in a format that is easy for the user to understand.

[0197] Step 9:

[0198] The device receives the response sent from the server. The device displays the response to the user visually or audibly. Text-based responses are displayed on the screen, and audio-based responses are played through the speaker.

[0199] Step 10:

[0200] The user reviews the provided answers. This ensures that the user receives appropriate answers to their questions, and that the answers are delivered in an emotionally sensitive tone, resulting in a more satisfying experience.

[0201] Specific example

[0202] Step 1:

[0203] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[0204] Step 2:

[0205] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[0206] Step 3:

[0207] The server sets up an emotion engine, enabling the analysis of voice and facial expression data.

[0208] Step 4:

[0209] User Tanaka types "What are the main features of the new product X?" into the device, and the question is read aloud. The camera captures Tanaka's facial expression.

[0210] Step 5:

[0211] The device sends Tanaka's question text, audio data, and facial expression data to the server.

[0212] Step 6:

[0213] The server passes the question text to the generative AI, and the voice data and facial expression data to the emotion engine.

[0214] Step 7:

[0215] The generative AI generates the response, "New product X has high-speed processing capabilities and excellent security features, and offers a user-friendly interface." The emotion engine determines from Tanaka's voice tone that he is excited and adjusts the tone of the response to be calmer.

[0216] Step 8:

[0217] The server sends the adjusted response to the terminal.

[0218] Step 9:

[0219] The terminal receives the response from the server, displays it to Tanaka, and reads it aloud.

[0220] Step 10:

[0221] Ms. Tanaka finds satisfaction in the appropriate answers to her questions and in the responses provided in an emotionally considerate tone.

[0222] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[0223] (Example 2)

[0224] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0225] There is a challenge in that sales representatives, especially new sales representatives, have difficulty quickly and accurately acquiring the necessary knowledge and providing appropriate responses while considering the user's feelings. In the current system, knowledge acquisition and recognition of user feelings are separated, and there is a lack of a system that functions as an integrated whole. As a result, the training efficiency of new sales representatives is low, and the quality of customer service cannot be guaranteed.

[0226] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, and means including an emotion engine that analyzes the user's voice data and video data to recognize their emotional state. As a result, sales representatives can quickly and accurately acquire the necessary knowledge and, furthermore, respond appropriately according to the user's emotions.

[0227] "Knowledge data" refers to data that compiles business knowledge and information, such as technical documents and FAQ data within a company.

[0228] "Service information" refers to data such as detailed information, specifications, and instruction manuals regarding the products and services offered.

[0229] "Generative AI" refers to artificial intelligence that learns from collected knowledge data and service information and generates appropriate answers to user questions.

[0230] A "natural language processing model" is a machine learning model used in generative AI to understand and generate natural human language.

[0231] An "emotion engine" refers to a technology that analyzes a user's voice and video data to recognize their emotional state.

[0232] A "database" is an information storage device for efficiently storing and managing knowledge data and service information.

[0233] "User" refers to a customer or sales representative who uses the system.

[0234] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions.

[0235] Data Ingestion

[0236] The server regularly collects knowledge data such as technical documents, FAQs, training materials, and detailed information about products and services offered within the company, and stores it in a database. This is done using scraping tools and APIs. By updating the information in the database in a timely manner, generative AI can always learn based on the latest information.

[0237] Data Learning

[0238] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from this data using natural language processing models such as OpenAI's GPT or Google's BERT. The AI ​​analyzes vast amounts of text data, recognizes patterns to respond to questions, and builds a knowledge base to generate answers.

[0239] Emotional engine integration

[0240] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. This emotion engine evaluates the user's emotions using technologies such as voice tone analysis and facial expression recognition. When the user inputs a question through the terminal, it uses voice and camera to perform real-time analysis and feeds the results back to the generative AI.

[0241] Questions accepted

[0242] The device has an interface for receiving questions from users, and users can input questions in natural language. Specifically, questions are submitted through a chatbot or text input field. The questions entered by the user are sent to the server along with audio and video data.

[0243] Question processing and response submission

[0244] The server passes the question received from the user to a generative AI, which analyzes the question. The generative AI retrieves relevant information from a database and generates the optimal answer. Simultaneously, an emotion engine evaluates the user's emotional state and feeds this information back to the generative AI. The generated answer is adjusted in tone and content according to the user's emotions. After the adjustments are complete, the server sends the answer to the terminal, which displays it to the user visually and audibly.

[0245] Specific example

[0246] Let's imagine a scenario where a new sales representative types the question, "What are the main features of the new product X?" and has it read aloud. The terminal collects the question and voice data and sends it to the server. The server's generative AI analyzes the question and derives the answer, "The new product X has high processing power, excellent security features, and a user-friendly interface." The emotion engine evaluates the user's voice tone as "interested" and adjusts the tone of the answer to match that emotion. As a result, the adjusted answer is sent to the user's terminal, and the user can confirm it in both voice and text.

[0247] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. This system is expected to significantly improve the quality of customer service.

[0248] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0249] Step 1: Data Ingest

[0250] The server periodically collects knowledge data, such as technical documents, FAQs, training materials, and detailed product and service information, from within the company and stores it in a database. This involves using scraping tools and APIs to retrieve data, normalizing it, and then storing it in the database. Input is data from various knowledge sources within the company, and output is knowledge data stored in the server's database. Specifically, the server periodically executes scripts and sends API requests to retrieve the latest data.

[0251] Step 2: Data Training

[0252] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from the data using natural language processing models such as OpenAI's GPT or Google's BERT. The input is knowledge data and service information in the database, and the output is the knowledge base built by the generative AI. Specifically, the AI ​​learns a model to analyze the data, recognize question-answer patterns, and generate relevant answers.

[0253] Step 3: Collecting emotional data

[0254] The device collects user audio and video data in real time and sends it to the emotion engine. This data is acquired using the microphone and camera built into the device. The input is the user's audio and video, and the output is the data sent to the emotion engine. Specifically, the device records audio, captures video with its camera, and sends this data to the server.

[0255] Step 4: Analysis of emotional state

[0256] The server's emotion engine analyzes audio and video data to recognize the user's emotional state. Specifically, it evaluates the emotional state using voice tone analysis and facial recognition technology. The input is the user's audio and video data, and the output is the analyzed emotional state information. The server evaluates emotions using voice tone analysis algorithms and facial recognition technology.

[0257] Step 5: Receiving the Question

[0258] The user inputs a question in natural language using the device. The device then sends the question to the server. The input is the text of the question entered by the user, and the output is the question data sent to the server. Specifically, the user submits a question through a chatbot or text input field, and the device forwards it to the server.

[0259] Step 6: Analyzing the Question

[0260] The server receives questions from users and passes them to a generative AI for analysis. The generative AI queries a knowledge base and generates the optimal answer. The input is the question data sent to the server, and the output is the generated answer data. Specifically, the AI ​​analyzes the question, retrieves relevant information from the knowledge base, and constructs the answer.

[0261] Step 7: Feedback on emotional information

[0262] The emotion engine feeds the analyzed emotional state back to the generative AI, which then adjusts the tone and content of the response. The input is the analyzed emotional state information, and the output is the adjusted response data. Specifically, the generative AI takes the emotional information into consideration and adjusts the tone of the response to include phrases such as "providing reassurance" or "including words of encouragement."

[0263] Step 8: Submit and view your response

[0264] The server sends the adjusted response to the user's device. The device displays the received response to the user. The input is the adjusted response data, and the output is the response information that the user receives visually and audibly. Specifically, the device provides the response to the user by reading it aloud using speech synthesis or by displaying it as a text message.

[0265] Through the processing steps described above, the system provides prompt, emotionally sensitive, and appropriate answers to user questions. By combining specific technologies and functions, it enables sales representatives to acquire knowledge and improves the quality of customer service.

[0266] (Application Example 2)

[0267] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0268] Conventional sales support systems can provide accurate answers to user questions, but they do not take into account the user's emotional state. This can lead to insufficient improvement in customer satisfaction and a decline in the quality of the customer experience. Furthermore, it is difficult for new sales representatives to quickly acquire the necessary knowledge. Therefore, there is a need for a system that allows new sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses tailored to the user's emotional state.

[0269] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for analyzing the user's voice data and video data and evaluating their emotional state, and means for adjusting the tone and content of the generated response according to the user's emotions. This enables sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses that take into account the user's emotions.

[0270] A "database" is an information system for storing knowledge data and service information, providing the foundational data that AI will later learn from.

[0271] "Knowledge data" refers to data that systematically compiles specific knowledge and information, such as technical documents and FAQ data within a company.

[0272] "Service information" refers to detailed information about the products and services offered, which is necessary to generate appropriate answers to user inquiries.

[0273] "Generative AI" refers to artificial intelligence that generates appropriate answers to user questions based on training data.

[0274] A "natural language processing model" is a machine learning model that understands and analyzes natural language text, such as questions from users, to generate appropriate answers.

[0275] "User voice data" refers to voice information provided by the user through a microphone, and is used to analyze the user's emotional state.

[0276] "Video data" refers to image information captured in real time via a camera, such as the user's facial expressions, and is used for emotion analysis.

[0277] "Emotional state" refers to the feelings and moods a user experiences at a particular moment, and is recognized by analyzing audio and video data.

[0278] "Response tone" refers to the tone and nuances of expression in the generated response, which are adjusted according to the user's emotional state.

[0279] A "terminal" is a device used by a user to input questions and receive answers, and includes smartphones and smart glasses.

[0280] This invention relates to a sales representative support system that combines a generative AI and an emotion engine, and will be described with a particular focus on its application in physical stores. Specific embodiments for carrying out this invention are described below.

[0281] Database and Server Configuration

[0282] The server periodically collects knowledge data such as technical documents and FAQs within the company, as well as detailed information about the products and services offered, and stores it in a database. This database provides the foundational data for generative AI to learn from.

[0283] Building Generative AI

[0284] The server provides the generative AI with the knowledge data and service information stored in the database. The generative AI learns these data and constructs a knowledge base for generating appropriate answers to various questions from users. It is important to recognize the patterns for answering questions using natural language processing technology.

[0285] Introduction of the Emotion Engine

[0286] The server is equipped with an emotion engine, which has the function of analyzing voice data and facial expression data from users and recognizing the emotional state of users. Specifically, when a user inputs a question through a terminal, the emotion at that time is analyzed in real time using voice and camera. The EmotionRecognizer library is used for emotion analysis.

[0287] Question Reception and Analysis

[0288] The terminal (for example, a smartphone or smart glasses) has an interface for receiving questions from users. The user inputs a question in natural language, and the terminal sends the question to the server. Also, the user's voice and camera video are collected in real time and sent to the emotion engine.

[0289] Answer Generation and Emotion Adjustment

[0290] The server receives the question from the user, passes it to the generative AI, and analyzes the question. In parallel, the emotion engine analyzes the voice and video data from the user and evaluates the emotional state of the user. The obtained emotion information is fed back to the generative AI, and the tone and content of the answer are adjusted according to the user's emotion.

[0291] Answer Sending and Display

[0292] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[0293] Specific example

[0294] For example, if a user asks "What is the warranty period for this product?" in a physical store, the smart glasses' camera might capture the user's facial expression and recognize that they are angry. Based on this information, a generative AI might generate a response such as "Excuse me, are you dissatisfied? The warranty period for this product is two years," and display it on the device in a tone appropriate to the user's emotions.

[0295] Example of a prompt

[0296] For example, here are some examples of prompt statements to input into a generative AI model:

[0297] Q: What is the warranty period for this product?

[0298] A:

[0299] Thus, the present invention provides a system that enables sales representatives to quickly and accurately acquire necessary knowledge, while also considering the user's feelings and providing appropriate responses. This can significantly improve the quality of customer service.

[0300] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0301] Step 1:

[0302] The user enters a question through a device (smartphone or smart glasses). The user enters the question in natural language and has it read aloud. At this time, the device's microphone captures the audio data.

[0303] Input: User's question (text format), user's voice data

[0304] Output: Question (text data), voice data

[0305] Step 2:

[0306] The terminal uses a speech recognition library (SpeechRecognition) to convert the captured voice data into text. The conversion from voice data to text data is performed.

[0307] Input: Voice data

[0308] Output: Question (text data)

[0309] <00​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​​The server inputs question data into a generative AI model and generates an appropriate answer. The generative AI generates the answer based on knowledge data and service information. This answer is provided in text format.

[0319] Input: Question (text data)

[0320] Output: Generated response (text data)

[0321] Step 6:

[0322] The server analyzes the video data using an emotion engine to recognize the user's emotional state. The emotion engine analyzes the video data and evaluates the user's emotions (e.g., anger, joy, dissatisfaction, etc.).

[0323] Input: Video data

[0324] Output: User's emotional state

[0325] Step 7:

[0326] The server adjusts the tone of the generated response based on emotional state information fed back from the emotion engine. The tone and content are adjusted according to the emotion.

[0327] Input: Generated response (text data), user's emotional state

[0328] Output: Emotionally balanced and adjusted responses (text data)

[0329] Step 8:

[0330] The server sends the adjusted response to the terminal. The terminal displays the received adjusted response to the user visually and audibly. The user can receive the appropriate response in an adjusted tone.

[0331] Input: Adjusted response (text data)

[0332] Output: Adjusted response displayed to the user

[0333] Specific actions:

[0334] For example, if a user asks, "What is the warranty period for this product?" and the device's camera captures an angry expression, the server uses generative AI to generate the answer, "The warranty period for this product is two years." After the emotion engine evaluates the anger, it adjusts the tone of the response to, "I'm sorry, are you dissatisfied? The warranty period for this product is two years."

[0335] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0336] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0337] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0338] [Second Embodiment]

[0339] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0340] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0341] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0342] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0343] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0344] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0345] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0346] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0347] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0348] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0349] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0350] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0351] This invention relates to a sales representative support system utilizing generative AI, and is particularly aimed at enabling new sales representatives to quickly acquire necessary knowledge. Specific embodiments for carrying out this invention are described below.

[0352] Data Ingestion

[0353] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[0354] Data Learning

[0355] The server retrieves knowledge data and service information stored in the database and provides it to the generative AI. The AI ​​learns from this data and becomes able to generate appropriate answers to various questions from users. This generative AI utilizes natural language processing technology to analyze the content of the questions and provide the optimal answer.

[0356] Questions accepted

[0357] The terminal has an interface for receiving questions from the user. The user inputs the question into the terminal in natural language, and the terminal sends the question to the server. This allows the user to quickly ask for the information they need.

[0358] Question Processing

[0359] The server receives questions from users and analyzes them. The analyzed questions are passed to a generative AI, which generates answers based on knowledge data and service information. In this answer generation process, the AI ​​utilizes its learned knowledge to provide the most appropriate information for the question.

[0360] Submitting and displaying responses

[0361] The generated answers are sent from the server to the user's device. The device receives the sent answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[0362] Specific example

[0363] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?" The terminal receives the question and sends it to the server. The server uses generative AI to analyze the question and generates the best answer based on knowledge data and service information in its database. The answer is, "The new product X has high processing power, excellent security features, and a user-friendly interface." The server sends this answer to Tanaka's terminal, where Tanaka can check the answer.

[0364] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[0365] The following describes the processing flow.

[0366] Step 1:

[0367] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database. The server structures the collected data and stores it in the database for quick access.

[0368] Step 2:

[0369] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds knowledge to generate appropriate answers to user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[0370] Step 3:

[0371] The user enters a question into the terminal. The terminal receives this input and sends the question to the server. The terminal provides an interface that accepts natural language input from the user and sends the question data, ready to be sent, to the server.

[0372] Step 4:

[0373] The server receives questions sent from terminals. Next, the server passes the received questions to a generative AI for analysis. The generative AI understands the content of the questions and generates the optimal answer based on knowledge data and service information in the database.

[0374] Step 5:

[0375] The server receives a response from the generative AI. This response provides specific and appropriate information in response to the user's question. The server then performs the necessary transmission processing to send this response to the user's terminal.

[0376] Step 6:

[0377] The terminal receives the response sent from the server. The terminal then visually displays this response to the user. The user can then review the displayed response and obtain information regarding their question.

[0378] Specific example

[0379] Step 1:

[0380] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[0381] Step 2:

[0382] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[0383] Step 3:

[0384] User Tanaka enters the question "What are the main features of the new product X?" into the terminal.

[0385] Step 4:

[0386] The terminal sends Tanaka's question to the server. The server receives the question, sends it to a generative AI, and analyzes it.

[0387] Step 5:

[0388] The generative AI generates the response, "New product X has high-speed processing capabilities, excellent security features, and a user-friendly interface." The server receives this response and sends it to Tanaka's terminal.

[0389] Step 6:

[0390] Tanaka's device receives the response from the server and displays the content visually to Tanaka. Tanaka checks the response regarding the main features of the new product X.

[0391] In this way, the present invention provides an efficient means for sales representatives to quickly and accurately acquire the necessary knowledge, thereby improving the quality of customer service.

[0392] (Example 1)

[0393] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0394] New sales representatives are required to acquire necessary knowledge quickly and efficiently to improve the quality of their customer service. However, traditional methods require them to individually search and understand vast amounts of knowledge data and product information, which is time-consuming and laborious. A system is needed to solve this problem.

[0395] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0396] In this invention, the server includes means for storing knowledge data and product information in a database, means for building a generative AI model that learns the knowledge data and product information, means for receiving user questions in natural language, means for analyzing the received questions and generating answers using the generative AI model, means for sending the generated answers to the user's terminal, and means for visually displaying the transmitted answers to the user. This enables new sales representatives to efficiently acquire necessary information and respond to customers quickly and accurately.

[0397] A "database" is an information storage system that efficiently manages and makes searchable knowledge data and product information.

[0398] "Knowledge data" refers to all data containing specific knowledge and information within a company, such as technical documents, FAQ data, and training materials.

[0399] "Product information" refers to detailed data about products and services offered by a company, including technical specifications, functions, and features.

[0400] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on knowledge data and product information to generate appropriate answers to questions.

[0401] A "user" refers to the entity that uses the system to input questions and receive answers. This is primarily intended for new sales representatives.

[0402] "Natural language" refers to the language that humans use on a daily basis, and specifically to the question format that users input into the system.

[0403] "Analysis" refers to the process of understanding the received question, extracting its contents, and organizing them.

[0404] "Answer generation" refers to the process of creating appropriate answers using a generative AI model based on the analyzed question content.

[0405] A "device" refers to a device used by a user to input questions and receive answers. This includes PCs and tablets.

[0406] "Visual display" refers to displaying the generated response on the device screen in a way that the user can see and understand.

[0407] This invention is a system that supports new sales representatives using a generative AI model. In particular, it aims to enable the rapid and accurate acquisition of necessary information by efficiently utilizing knowledge data and product information. Specific embodiments for carrying out this invention are described below.

[0408] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services the company provides, and stores it in a database (e.g., MySQL or PostgreSQL). This data later serves as foundational data for training generative AI models (e.g., OpenAI's GPT-3).

[0409] The server provides the AI ​​model with knowledge data and product information stored in the database, and this AI model learns from this data. This allows it to generate appropriate answers to user questions.

[0410] The user (new sales representative) uses a provided device (e.g., a PC or tablet) to input a question in natural language. The device then sends the question to the server.

[0411] The server receives questions sent from terminals and analyzes them using a generative AI model. Based on the analyzed question, the AI ​​model refers to knowledge data and product information in the database to generate the optimal answer. This answer generation process fully utilizes the knowledge the AI ​​model has learned.

[0412] The generated answers are sent from the server to the user's device, which receives the answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[0413] As a concrete example, consider a scenario where a new sales representative enters the question, "What are the main features of the new product X?" into a terminal. In this case, the terminal sends the question to a server, which uses a generative AI model to analyze the question. As a result, an answer is generated, such as, "The new product X has high-speed processing capabilities, excellent security features, and a user-friendly interface," and this answer is sent to the user's terminal. The user can then view the answer on their terminal.

[0414] The following is an example of a prompt message.

[0415] In response to the question, "What are the main features of the new product X?", generate a detailed description of the features of product X.

[0416] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[0417] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0418] Step 1: Data Ingest

[0419] The server collects technical documents, FAQ data, training materials, and detailed information about products and services within the company. This data is periodically stored in a database by the server. The input consists of various documents and data within the company, while the output is organized knowledge data and product information stored in the database.

[0420] Step 2: Data Training

[0421] The server provides a generative AI model with knowledge data and product information stored in a database. The generative AI model (e.g., GPT-3) learns from this data. The input is knowledge data and product information in the database, and the output is a generative AI model capable of generating appropriate answers to questions.

[0422] Step 3: Question Acceptance

[0423] The user uses a terminal to input a question in natural language. The terminal sends this question to the server. The input is the user's question in natural language, and the output is that the question has been sent to the server.

[0424] Step 4: Questionnaire Analysis

[0425] The server receives questions sent from the terminal and analyzes them using a generative AI model. The input is the user's question, and the output is the analyzed question. The server uses natural language processing techniques to properly understand the questions.

[0426] Step 5: Generate Response

[0427] The server uses a generative AI model to generate the optimal answer based on the analyzed question content. The input consists of the analyzed question content, knowledge data, and product information, while the output is the generated answer. The generated answer is based on the knowledge learned by the AI ​​model.

[0428] Step 6: Submit your response

[0429] The server sends the generated response to the user's terminal. The input is the generated response, and the output is that the response is sent to the terminal.

[0430] Step 7: Display your answer

[0431] The terminal receives the response sent from the server and displays it visually to the user. The input is the response sent from the server, and the output is the response displayed on the terminal's screen. The user checks the response on the terminal and obtains the necessary information.

[0432] (Application Example 1)

[0433] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0434] In conventional factory equipment maintenance, it was difficult for workers to quickly and accurately obtain the necessary information. In particular, new workers were unfamiliar with equipment maintenance procedures and specific operating methods, and it took time for them to acquire the necessary knowledge. This led to decreased efficiency in maintenance work and an increased risk of equipment failure due to improper operation.

[0435] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0436] In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for receiving questions from users, means for analyzing the received questions and generating answers using the generative AI, means for sending the generated answers to the user, means for displaying the sent answers to the user, and means for receiving questions and displaying answers to robots operating in the factory. This makes it possible to quickly and accurately acquire information necessary for equipment maintenance work in the factory and improve work efficiency.

[0437] A "database" is a system for systematically accumulating and managing information, and it plays a role in storing knowledge data and service information.

[0438] "Knowledge data" refers to a collection of data containing specialized knowledge and information within a company, such as technical documents and FAQ data.

[0439] "Service information" refers to a collection of data containing detailed information about the products and services offered by a company.

[0440] "Generative AI" is artificial intelligence that learns from knowledge data and service information to generate the best possible answers to user questions.

[0441] "Means for receiving questions from users" refers to devices or systems that have an interface that allows users to input questions in natural language.

[0442] "Means for analyzing questions and generating answers using generative AI" refers to devices or systems that analyze questions received from users and execute a process to generate the most appropriate answer to those questions.

[0443] "Means for sending generated answers to users" refers to communication methods for providing users with answers generated by generative AI.

[0444] "Means for displaying submitted responses to the user" refers to devices or systems that allow the user to visually confirm the responses provided.

[0445] "Robots operating in factories" are robots designed to perform tasks in factory manufacturing areas or equipment maintenance sites.

[0446] "Means for receiving questions and displaying answers" refers to devices or systems that have the function of receiving questions from users and displaying corresponding answers, such as robots in a factory.

[0447] This invention provides a system for supporting equipment maintenance work within a factory. Specific embodiments are described below.

[0448] First, the server periodically collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores this data in a database. This database serves as the foundational data for generative AI to learn from. The hardware used will consist of a server computer and a large-capacity storage device. The specific software used will include a database management system (DBMS) and data collection tools.

[0449] Next, the server builds a generative AI model based on the knowledge data and service information stored in the database. This model uses OpenAI's GPT-3 or other natural language processing models. This enables the AI ​​to generate appropriate answers to various questions from the user.

[0450] Users (in this case, factory workers) input questions through robots operating within the factory. The robots accept questions in natural language and send them to a server. The robots are equipped with a voice recognition system and a touchscreen, allowing users to input questions by voice or text.

[0451] When the server receives a question, it analyzes the question using natural language processing technology and passes it on to a generative AI. The AI ​​generates the optimal answer based on pre-trained knowledge data and service information. This process utilizes natural language processing technology to understand the meaning of the question and select the appropriate answer.

[0452] The generated response is sent from the server to the robot. The robot then presents this response to the user visually or audibly. For example, if a worker asks, "How do I change the oil in a machine tool?", the robot will display or voice the response it received from the server, saying, "The oil change is performed using the following steps..."

[0453] As a concrete example, if a new worker in a factory asks a question about how to inspect a conveyor belt, the following exchange may occur:

[0454] Question: How do you inspect a conveyor belt?

[0455] Answer: Belt conveyor inspection is performed using the following procedure:

[0456] 1. Stop the machine and turn off the power.

[0457] 2. Check the tension of the belt.

[0458] 3. Visually inspect the belt for any abnormalities.

[0459] 4. If any abnormalities are found, appropriate repairs will be carried out.

[0460] 5. After the inspection, restart the machine and check if it is working correctly.

[0461] In this way, by providing the necessary information for equipment maintenance work within the factory quickly and accurately, work efficiency can be improved.

[0462] Furthermore, the generated responses are recorded as logs on the server and used for future improvements and evaluations. This system is expected to enable new workers to quickly acquire the necessary knowledge and significantly improve the efficiency of maintenance work within the factory.

[0463] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0464] Step 1:

[0465] The server collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores it in a database. This collection process is performed regularly, and the information is organized using a database management system (DBMS). The input is internal company data, and the output is a structured database.

[0466] Step 2:

[0467] The server extracts data from knowledge data and service information stored in the database and provides it to a generative AI (in this case, OpenAI's GPT-3). The AI ​​learns from this data and becomes capable of operating as a natural language processing model. The input is knowledge data from the database, and the output is the trained generative AI model. Specifically, the process involves data formatting and feeding the data to the AI ​​model.

[0468] Step 3:

[0469] Users input questions to factory robots via voice or text. The robots receive user questions using a voice recognition system or touchscreen. The input is the user's question, and the output is question data in digital format. Specifically, voice recognition processing is performed to convert speech to text.

[0470] Step 4:

[0471] The robot sends the received question to the server. The server receives this question and analyzes it using natural language processing technology. The input is the question data sent by the robot, and the output is the analyzed question data. Specifically, the operation includes semantic analysis of the question content and extraction of related data.

[0472] Step 5:

[0473] The server passes the analyzed question to the generative AI, which generates the optimal answer based on knowledge data and service information. The input is the analyzed question data, and the output is the generated answer. Specifically, the AI ​​model generates the answer. Example of a prompt: "Question: How do I change the oil in a machine tool? Answer: The oil change is performed using the following steps..."

[0474] Step 6:

[0475] The generated response is sent from the server to the robot. The robot receives this response and presents it to the user visually or audibly. The input is the response data from the server, and the output is a display or audio output in a format that the user can see. Specific actions include displaying the response on a screen and outputting audio from a speaker.

[0476] Step 7:

[0477] The user reviews the responses provided by the robot and performs the necessary maintenance tasks. The input is the robot's response information, and the output is the user's actions. Specific actions include the user performing maintenance tasks.

[0478] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0479] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions. Specific embodiments for carrying out this invention are described below.

[0480] Data Ingestion

[0481] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[0482] Data Learning

[0483] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to various user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[0484] Emotional engine integration

[0485] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. Specifically, when the user inputs a question through their device, the engine analyzes their emotions in real time using voice and camera data. The analysis results are fed back to a generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0486] Questions accepted

[0487] The device has an interface for receiving questions from the user. The user inputs questions into the device using natural language, and the device sends those questions to the server. It also collects the user's voice and camera footage in real time and sends it to the emotion engine.

[0488] Question Processing

[0489] The server receives questions from users and passes them to a generative AI for analysis. In parallel, the emotion engine analyzes audio and video data from the user and evaluates the user's emotional state. The obtained emotional information is fed back to the generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0490] Submitting and displaying responses

[0491] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[0492] Specific example

[0493] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?", and simultaneously reads the question aloud. The terminal sends the question along with Tanaka's voice data to the server. The server uses generative AI to analyze the question and an emotion engine to evaluate Tanaka's emotion from his voice tone. Based on knowledge data and service information in the database, the generative AI generates the answer, "The new product X has high processing power and excellent security features, and provides a user-friendly interface." Based on feedback from the emotion engine, the tone of the answer is adjusted to match Tanaka's emotion. The server sends the adjusted answer to Tanaka's terminal, where Tanaka confirms the answer.

[0494] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[0495] The following describes the processing flow.

[0496] Step 1:

[0497] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database, which provides the foundational data for generative AI to learn from.

[0498] Step 2:

[0499] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to user questions. In the learning process, natural language processing techniques are used to analyze the data and recognize patterns that fit the questions.

[0500] Step 3:

[0501] The server prepares to utilize the emotion engine. The emotion engine analyzes the user's voice data and camera footage to recognize the user's emotional state. The emotion engine evaluates emotions in real time based on voice tone and facial expressions, and feeds the obtained information back to the generative AI.

[0502] Step 4:

[0503] The user inputs a question into the device using natural language. The device provides an interface for inputting questions and receives them using text boxes or voice input functions.

[0504] Step 5:

[0505] The device sends the user's question text to the server. Simultaneously, if voice input is used, the voice data is also sent to the server. Furthermore, it may also send user facial expression data obtained from the camera feed.

[0506] Step 6:

[0507] The server receives question text, audio data, and facial expression data sent from the terminal. First, the received question text is passed to a generative AI to analyze the question. Next, the emotion engine analyzes the audio data and facial expression data to evaluate the user's emotional state. For example, it identifies emotions such as joy, sadness, surprise, and anxiety.

[0508] Step 7:

[0509] The generative AI searches the database for appropriate knowledge data and service information based on the analyzed question content and generates an answer. At the same time, it receives emotion evaluation data from the emotion engine and adjusts the tone and content of the answer according to the user's emotions. For example, if the user is agitated, it will adjust the answer to be provided in a calm tone.

[0510] Step 8:

[0511] The server sends the generated response to the terminal. The adjusted response data is sent in a format that is easy for the user to understand.

[0512] Step 9:

[0513] The device receives the response sent from the server. The device displays the response to the user visually or audibly. Text-based responses are displayed on the screen, and audio-based responses are played through the speaker.

[0514] Step 10:

[0515] The user reviews the provided answers. This ensures that the user receives appropriate answers to their questions, and that the answers are delivered in an emotionally sensitive tone, resulting in a more satisfying experience.

[0516] Specific example

[0517] Step 1:

[0518] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[0519] Step 2:

[0520] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[0521] Step 3:

[0522] The server sets up an emotion engine, enabling the analysis of voice and facial expression data.

[0523] Step 4:

[0524] User Tanaka types "What are the main features of the new product X?" into the device, and the question is read aloud. The camera captures Tanaka's facial expression.

[0525] Step 5:

[0526] The device sends Tanaka's question text, audio data, and facial expression data to the server.

[0527] Step 6:

[0528] The server passes the question text to the generative AI, and the voice data and facial expression data to the emotion engine.

[0529] Step 7:

[0530] The generative AI generates the response, "New product X has high-speed processing capabilities and excellent security features, and offers a user-friendly interface." The emotion engine determines from Tanaka's voice tone that he is excited and adjusts the tone of the response to be calmer.

[0531] Step 8:

[0532] The server sends the adjusted response to the terminal.

[0533] Step 9:

[0534] The terminal receives the response from the server, displays it to Tanaka, and reads it aloud.

[0535] Step 10:

[0536] Ms. Tanaka finds satisfaction in the appropriate answers to her questions and in the responses provided in an emotionally considerate tone.

[0537] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[0538] (Example 2)

[0539] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0540] There is a challenge in that sales representatives, especially new sales representatives, have difficulty quickly and accurately acquiring the necessary knowledge and providing appropriate responses while considering the user's feelings. In the current system, knowledge acquisition and recognition of user feelings are separated, and there is a lack of a system that functions as an integrated whole. As a result, the training efficiency of new sales representatives is low, and the quality of customer service cannot be guaranteed.

[0541] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, and means including an emotion engine that analyzes the user's voice data and video data to recognize their emotional state. As a result, sales representatives can quickly and accurately acquire the necessary knowledge and, furthermore, respond appropriately according to the user's emotions.

[0542] "Knowledge data" refers to data that compiles business knowledge and information, such as technical documents and FAQ data within a company.

[0543] "Service information" refers to data such as detailed information, specifications, and instruction manuals regarding the products and services offered.

[0544] "Generative AI" refers to artificial intelligence that learns from collected knowledge data and service information and generates appropriate answers to user questions.

[0545] A "natural language processing model" is a machine learning model used in generative AI to understand and generate natural human language.

[0546] An "emotion engine" refers to a technology that analyzes a user's voice and video data to recognize their emotional state.

[0547] A "database" is an information storage device for efficiently storing and managing knowledge data and service information.

[0548] "User" refers to a customer or sales representative who uses the system.

[0549] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions.

[0550] Data Ingestion

[0551] The server regularly collects knowledge data such as technical documents, FAQs, training materials, and detailed information about products and services offered within the company, and stores it in a database. This is done using scraping tools and APIs. By updating the information in the database in a timely manner, generative AI can always learn based on the latest information.

[0552] Data Learning

[0553] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from this data using natural language processing models such as OpenAI's GPT or Google's BERT. The AI ​​analyzes vast amounts of text data, recognizes patterns to respond to questions, and builds a knowledge base to generate answers.

[0554] Emotional engine integration

[0555] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. This emotion engine evaluates the user's emotions using technologies such as voice tone analysis and facial expression recognition. When the user inputs a question through the terminal, it uses voice and camera to perform real-time analysis and feeds the results back to the generative AI.

[0556] Questions accepted

[0557] The device has an interface for receiving questions from users, and users can input questions in natural language. Specifically, questions are submitted through a chatbot or text input field. The questions entered by the user are sent to the server along with audio and video data.

[0558] Question processing and response submission

[0559] The server passes the question received from the user to a generative AI, which analyzes the question. The generative AI retrieves relevant information from a database and generates the optimal answer. Simultaneously, an emotion engine evaluates the user's emotional state and feeds this information back to the generative AI. The generated answer is adjusted in tone and content according to the user's emotions. After the adjustments are complete, the server sends the answer to the terminal, which displays it to the user visually and audibly.

[0560] Specific example

[0561] Let's imagine a scenario where a new sales representative types the question, "What are the main features of the new product X?" and has it read aloud. The terminal collects the question and voice data and sends it to the server. The server's generative AI analyzes the question and derives the answer, "The new product X has high processing power, excellent security features, and a user-friendly interface." The emotion engine evaluates the user's voice tone as "interested" and adjusts the tone of the answer to match that emotion. As a result, the adjusted answer is sent to the user's terminal, and the user can confirm it in both voice and text.

[0562] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. This system is expected to significantly improve the quality of customer service.

[0563] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0564] Step 1: Data Ingest

[0565] The server periodically collects knowledge data, such as technical documents, FAQs, training materials, and detailed product and service information, from within the company and stores it in a database. This involves using scraping tools and APIs to retrieve data, normalizing it, and then storing it in the database. Input is data from various knowledge sources within the company, and output is knowledge data stored in the server's database. Specifically, the server periodically executes scripts and sends API requests to retrieve the latest data.

[0566] Step 2: Data Training

[0567] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from the data using natural language processing models such as OpenAI's GPT or Google's BERT. The input is knowledge data and service information in the database, and the output is the knowledge base built by the generative AI. Specifically, the AI ​​learns a model to analyze the data, recognize question-answer patterns, and generate relevant answers.

[0568] Step 3: Collecting emotional data

[0569] The device collects user audio and video data in real time and sends it to the emotion engine. This data is acquired using the microphone and camera built into the device. The input is the user's audio and video, and the output is the data sent to the emotion engine. Specifically, the device records audio, captures video with its camera, and sends this data to the server.

[0570] Step 4: Analysis of emotional state

[0571] The server's emotion engine analyzes audio and video data to recognize the user's emotional state. Specifically, it evaluates the emotional state using voice tone analysis and facial recognition technology. The input is the user's audio and video data, and the output is the analyzed emotional state information. The server evaluates emotions using voice tone analysis algorithms and facial recognition technology.

[0572] Step 5: Receiving the Question

[0573] The user inputs a question in natural language using the device. The device then sends the question to the server. The input is the text of the question entered by the user, and the output is the question data sent to the server. Specifically, the user submits a question through a chatbot or text input field, and the device forwards it to the server.

[0574] Step 6: Analyzing the Question

[0575] The server receives questions from users and passes them to a generative AI for analysis. The generative AI queries a knowledge base and generates the optimal answer. The input is the question data sent to the server, and the output is the generated answer data. Specifically, the AI ​​analyzes the question, retrieves relevant information from the knowledge base, and constructs the answer.

[0576] Step 7: Feedback on emotional information

[0577] The emotion engine feeds the analyzed emotional state back to the generative AI, which then adjusts the tone and content of the response. The input is the analyzed emotional state information, and the output is the adjusted response data. Specifically, the generative AI takes the emotional information into consideration and adjusts the tone of the response to include phrases such as "providing reassurance" or "including words of encouragement."

[0578] Step 8: Submit and view your response

[0579] The server sends the adjusted response to the user's device. The device displays the received response to the user. The input is the adjusted response data, and the output is the response information that the user receives visually and audibly. Specifically, the device provides the response to the user by reading it aloud using speech synthesis or by displaying it as a text message.

[0580] Through the processing steps described above, the system provides prompt, emotionally sensitive, and appropriate answers to user questions. By combining specific technologies and functions, it enables sales representatives to acquire knowledge and improves the quality of customer service.

[0581] (Application Example 2)

[0582] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0583] Conventional sales support systems can provide accurate answers to user questions, but they do not take into account the user's emotional state. This can lead to insufficient improvement in customer satisfaction and a decline in the quality of the customer experience. Furthermore, it is difficult for new sales representatives to quickly acquire the necessary knowledge. Therefore, there is a need for a system that allows new sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses tailored to the user's emotional state.

[0584] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for analyzing the user's voice data and video data and evaluating their emotional state, and means for adjusting the tone and content of the generated response according to the user's emotions. This enables sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses that take into account the user's emotions.

[0585] A "database" is an information system for storing knowledge data and service information, providing the foundational data that AI will later learn from.

[0586] "Knowledge data" refers to data that systematically compiles specific knowledge and information, such as technical documents and FAQ data within a company.

[0587] "Service information" refers to detailed information about the products and services offered, which is necessary to generate appropriate answers to user inquiries.

[0588] "Generative AI" refers to artificial intelligence that generates appropriate answers to user questions based on training data.

[0589] A "natural language processing model" is a machine learning model that understands and analyzes natural language text, such as questions from users, to generate appropriate answers.

[0590] "User voice data" refers to voice information provided by the user through a microphone, and is used to analyze the user's emotional state.

[0591] "Video data" refers to image information captured in real time via a camera, such as the user's facial expressions, and is used for emotion analysis.

[0592] "Emotional state" refers to the feelings and moods a user experiences at a particular moment, and is recognized by analyzing audio and video data.

[0593] "Response tone" refers to the tone and nuances of expression in the generated response, which are adjusted according to the user's emotional state.

[0594] A "terminal" is a device used by a user to input questions and receive answers, and includes smartphones and smart glasses.

[0595] This invention relates to a sales representative support system that combines a generative AI and an emotion engine, and will be described with a particular focus on its application in physical stores. Specific embodiments for carrying out this invention are described below.

[0596] Database and Server Configuration

[0597] The server periodically collects knowledge data such as technical documents and FAQs within the company, as well as detailed information about the products and services offered, and stores it in a database. This database provides the foundational data for generative AI to learn from.

[0598] Building Generative AI

[0599] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to various user questions. It is important to recognize patterns for responding to questions using natural language processing techniques.

[0600] Introducing an emotional engine

[0601] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. Specifically, when the user inputs a question through the terminal, the system uses voice and camera data to analyze their emotions in real time. The EmotionRecognizer library is used for emotion analysis.

[0602] Question reception and analysis

[0603] The device (e.g., a smartphone or smart glasses) has an interface that receives questions from the user. The user inputs the question in natural language, and the device sends the question to the server. It also collects the user's voice and camera footage in real time and sends it to the emotion engine.

[0604] Response generation and sentiment adjustment

[0605] The server receives questions from users and passes them to a generative AI for analysis. In parallel, the emotion engine analyzes audio and video data from the user and evaluates the user's emotional state. The obtained emotional information is fed back to the generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0606] Displayed as "Response submitted".

[0607] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[0608] Specific example

[0609] For example, if a user asks "What is the warranty period for this product?" in a physical store, the smart glasses' camera might capture the user's facial expression and recognize that they are angry. Based on this information, a generative AI might generate a response such as "Excuse me, are you dissatisfied? The warranty period for this product is two years," and display it on the device in a tone appropriate to the user's emotions.

[0610] Example of a prompt

[0611] For example, here are some examples of prompt statements to input into a generative AI model:

[0612] Q: What is the warranty period for this product?

[0613] A:

[0614] Thus, the present invention provides a system that enables sales representatives to quickly and accurately acquire necessary knowledge, while also considering the user's feelings and providing appropriate responses. This can significantly improve the quality of customer service.

[0615] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0616] Step 1:

[0617] The user enters a question through a device (smartphone or smart glasses). The user enters the question in natural language and has it read aloud. At this time, the device's microphone captures the audio data.

[0618] Input: User's question (text format), user's voice data

[0619] Output: Question (text data), audio data

[0620] Step 2:

[0621] The device uses a speech recognition library to convert the captured audio data into text. The conversion from audio data to text data takes place.

[0622] Input: Audio data

[0623] Output: Question (text data)

[0624] Step 3:

[0625] The device captures the user's facial expressions with its camera and generates video data. This prepares it for real-time analysis of the user's emotional state.

[0626] Input: User's facial expression (video data)

[0627] Output: Video data

[0628] Step 4:

[0629] The device sends text data and video data of the question to the server. The server receives this data and passes the question data to the generative AI and the video data to the emotion engine.

[0630] Input: Question (text data), video data

[0631] Output: Sending question data and video data to the server.

[0632] Step 5:

[0633] The server inputs question data into a generative AI model and generates an appropriate answer. The generative AI generates the answer based on knowledge data and service information. This answer is provided in text format.

[0634] Input: Question (text data)

[0635] Output: Generated response (text data)

[0636] Step 6:

[0637] The server analyzes the video data using an emotion engine to recognize the user's emotional state. The emotion engine analyzes the video data and evaluates the user's emotions (e.g., anger, joy, dissatisfaction, etc.).

[0638] Input: Video data

[0639] Output: User's emotional state

[0640] Step 7:

[0641] The server adjusts the tone of the generated response based on emotional state information fed back from the emotion engine. The tone and content are adjusted according to the emotion.

[0642] Input: Generated response (text data), user's emotional state

[0643] Output: Emotionally balanced and adjusted responses (text data)

[0644] Step 8:

[0645] The server sends the adjusted response to the terminal. The terminal displays the received adjusted response to the user visually and audibly. The user can receive the appropriate response in an adjusted tone.

[0646] Input: Adjusted response (text data)

[0647] Output: Adjusted response displayed to the user

[0648] Specific actions:

[0649] For example, if a user asks, "What is the warranty period for this product?" and the device's camera captures an angry expression, the server uses generative AI to generate the answer, "The warranty period for this product is two years." After the emotion engine evaluates the anger, it adjusts the tone of the response to, "I'm sorry, are you dissatisfied? The warranty period for this product is two years."

[0650] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0651] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0652] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0653] [Third Embodiment]

[0654] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0655] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0656] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0657] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0658] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0659] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0660] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0661] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0662] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0663] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0664] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0665] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0666] This invention relates to a sales representative support system utilizing generative AI, and is particularly aimed at enabling new sales representatives to quickly acquire necessary knowledge. Specific embodiments for carrying out this invention are described below.

[0667] Data Ingestion

[0668] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[0669] Data Learning

[0670] The server retrieves knowledge data and service information stored in the database and provides it to the generative AI. The AI ​​learns from this data and becomes able to generate appropriate answers to various questions from users. This generative AI utilizes natural language processing technology to analyze the content of the questions and provide the optimal answer.

[0671] Questions accepted

[0672] The terminal has an interface for receiving questions from the user. The user inputs the question into the terminal in natural language, and the terminal sends the question to the server. This allows the user to quickly ask for the information they need.

[0673] Question Processing

[0674] The server receives questions from users and analyzes them. The analyzed questions are passed to a generative AI, which generates answers based on knowledge data and service information. In this answer generation process, the AI ​​utilizes its learned knowledge to provide the most appropriate information for the question.

[0675] Submitting and displaying responses

[0676] The generated answers are sent from the server to the user's device. The device receives the sent answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[0677] Specific example

[0678] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?" The terminal receives the question and sends it to the server. The server uses generative AI to analyze the question and generates the best answer based on knowledge data and service information in its database. The answer is, "The new product X has high processing power, excellent security features, and a user-friendly interface." The server sends this answer to Tanaka's terminal, where Tanaka can check the answer.

[0679] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[0680] The following describes the processing flow.

[0681] Step 1:

[0682] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database. The server structures the collected data and stores it in the database for quick access.

[0683] Step 2:

[0684] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds knowledge to generate appropriate answers to user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[0685] Step 3:

[0686] The user enters a question into the terminal. The terminal receives this input and sends the question to the server. The terminal provides an interface that accepts natural language input from the user and sends the question data, ready to be sent, to the server.

[0687] Step 4:

[0688] The server receives questions sent from terminals. Next, the server passes the received questions to a generative AI for analysis. The generative AI understands the content of the questions and generates the optimal answer based on knowledge data and service information in the database.

[0689] Step 5:

[0690] The server receives a response from the generative AI. This response provides specific and appropriate information in response to the user's question. The server then performs the necessary transmission processing to send this response to the user's terminal.

[0691] Step 6:

[0692] The terminal receives the response sent from the server. The terminal then visually displays this response to the user. The user can then review the displayed response and obtain information regarding their question.

[0693] Specific example

[0694] Step 1:

[0695] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[0696] Step 2:

[0697] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[0698] Step 3:

[0699] User Tanaka enters the question "What are the main features of the new product X?" into the terminal.

[0700] Step 4:

[0701] The terminal sends Tanaka's question to the server. The server receives the question, sends it to a generative AI, and analyzes it.

[0702] Step 5:

[0703] The generative AI generates the response, "New product X has high-speed processing capabilities, excellent security features, and a user-friendly interface." The server receives this response and sends it to Tanaka's terminal.

[0704] Step 6:

[0705] Tanaka's device receives the response from the server and displays the content visually to Tanaka. Tanaka checks the response regarding the main features of the new product X.

[0706] In this way, the present invention provides an efficient means for sales representatives to quickly and accurately acquire the necessary knowledge, thereby improving the quality of customer service.

[0707] (Example 1)

[0708] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0709] New sales representatives are required to acquire necessary knowledge quickly and efficiently to improve the quality of their customer service. However, traditional methods require them to individually search and understand vast amounts of knowledge data and product information, which is time-consuming and laborious. A system is needed to solve this problem.

[0710] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0711] In this invention, the server includes means for storing knowledge data and product information in a database, means for building a generative AI model that learns the knowledge data and product information, means for receiving user questions in natural language, means for analyzing the received questions and generating answers using the generative AI model, means for sending the generated answers to the user's terminal, and means for visually displaying the transmitted answers to the user. This enables new sales representatives to efficiently acquire necessary information and respond to customers quickly and accurately.

[0712] A "database" is an information storage system that efficiently manages and makes searchable knowledge data and product information.

[0713] "Knowledge data" refers to all data containing specific knowledge and information within a company, such as technical documents, FAQ data, and training materials.

[0714] "Product information" refers to detailed data about products and services offered by a company, including technical specifications, functions, and features.

[0715] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on knowledge data and product information to generate appropriate answers to questions.

[0716] A "user" refers to the entity that uses the system to input questions and receive answers. This is primarily intended for new sales representatives.

[0717] "Natural language" refers to the language that humans use on a daily basis, and specifically to the question format that users input into the system.

[0718] "Analysis" refers to the process of understanding the received question, extracting its contents, and organizing them.

[0719] "Answer generation" refers to the process of creating appropriate answers using a generative AI model based on the analyzed question content.

[0720] A "device" refers to a device used by a user to input questions and receive answers. This includes PCs and tablets.

[0721] "Visual display" refers to displaying the generated response on the device screen in a way that the user can see and understand.

[0722] This invention is a system that supports new sales representatives using a generative AI model. In particular, it aims to enable the rapid and accurate acquisition of necessary information by efficiently utilizing knowledge data and product information. Specific embodiments for carrying out this invention are described below.

[0723] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services the company provides, and stores it in a database (e.g., MySQL or PostgreSQL). This data later serves as foundational data for training generative AI models (e.g., OpenAI's GPT-3).

[0724] The server provides the AI ​​model with knowledge data and product information stored in the database, and this AI model learns from this data. This allows it to generate appropriate answers to user questions.

[0725] The user (new sales representative) uses a provided device (e.g., a PC or tablet) to input a question in natural language. The device then sends the question to the server.

[0726] The server receives questions sent from terminals and analyzes them using a generative AI model. Based on the analyzed question, the AI ​​model refers to knowledge data and product information in the database to generate the optimal answer. This answer generation process fully utilizes the knowledge the AI ​​model has learned.

[0727] The generated answers are sent from the server to the user's device, which receives the answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[0728] As a concrete example, consider a scenario where a new sales representative enters the question, "What are the main features of the new product X?" into a terminal. In this case, the terminal sends the question to a server, which uses a generative AI model to analyze the question. As a result, an answer is generated, such as, "The new product X has high-speed processing capabilities, excellent security features, and a user-friendly interface," and this answer is sent to the user's terminal. The user can then view the answer on their terminal.

[0729] The following is an example of a prompt message.

[0730] In response to the question, "What are the main features of the new product X?", generate a detailed description of the features of product X.

[0731] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[0732] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0733] Step 1: Data Ingest

[0734] The server collects technical documents, FAQ data, training materials, and detailed information about products and services within the company. This data is periodically stored in a database by the server. The input consists of various documents and data within the company, while the output is organized knowledge data and product information stored in the database.

[0735] Step 2: Data Training

[0736] The server provides a generative AI model with knowledge data and product information stored in a database. The generative AI model (e.g., GPT-3) learns from this data. The input is knowledge data and product information in the database, and the output is a generative AI model capable of generating appropriate answers to questions.

[0737] Step 3: Question Acceptance

[0738] The user uses a terminal to input a question in natural language. The terminal sends this question to the server. The input is the user's question in natural language, and the output is that the question has been sent to the server.

[0739] Step 4: Questionnaire Analysis

[0740] The server receives questions sent from the terminal and analyzes them using a generative AI model. The input is the user's question, and the output is the analyzed question. The server uses natural language processing techniques to properly understand the questions.

[0741] Step 5: Generate Response

[0742] The server uses a generative AI model to generate the optimal answer based on the analyzed question content. The input consists of the analyzed question content, knowledge data, and product information, while the output is the generated answer. The generated answer is based on the knowledge learned by the AI ​​model.

[0743] Step 6: Submit your response

[0744] The server sends the generated response to the user's terminal. The input is the generated response, and the output is that the response is sent to the terminal.

[0745] Step 7: Display your answer

[0746] The terminal receives the response sent from the server and displays it visually to the user. The input is the response sent from the server, and the output is the response displayed on the terminal's screen. The user checks the response on the terminal and obtains the necessary information.

[0747] (Application Example 1)

[0748] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0749] In conventional factory equipment maintenance, it was difficult for workers to quickly and accurately obtain the necessary information. In particular, new workers were unfamiliar with equipment maintenance procedures and specific operating methods, and it took time for them to acquire the necessary knowledge. This led to decreased efficiency in maintenance work and an increased risk of equipment failure due to improper operation.

[0750] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0751] In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for receiving questions from users, means for analyzing the received questions and generating answers using the generative AI, means for sending the generated answers to the user, means for displaying the sent answers to the user, and means for receiving questions and displaying answers to robots operating in the factory. This makes it possible to quickly and accurately acquire information necessary for equipment maintenance work in the factory and improve work efficiency.

[0752] A "database" is a system for systematically accumulating and managing information, and it plays a role in storing knowledge data and service information.

[0753] "Knowledge data" refers to a collection of data containing specialized knowledge and information within a company, such as technical documents and FAQ data.

[0754] "Service information" refers to a collection of data containing detailed information about the products and services offered by a company.

[0755] "Generative AI" is artificial intelligence that learns from knowledge data and service information to generate the best possible answers to user questions.

[0756] "Means for receiving questions from users" refers to devices or systems that have an interface that allows users to input questions in natural language.

[0757] "Means for analyzing questions and generating answers using generative AI" refers to devices or systems that analyze questions received from users and execute a process to generate the most appropriate answer to those questions.

[0758] "Means for sending generated answers to users" refers to communication methods for providing users with answers generated by generative AI.

[0759] "Means for displaying submitted responses to the user" refers to devices or systems that allow the user to visually confirm the responses provided.

[0760] "Robots operating in factories" are robots designed to perform tasks in factory manufacturing areas or equipment maintenance sites.

[0761] "Means for receiving questions and displaying answers" refers to devices or systems that have the function of receiving questions from users and displaying corresponding answers, such as robots in a factory.

[0762] This invention provides a system for supporting equipment maintenance work within a factory. Specific embodiments are described below.

[0763] First, the server periodically collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores this data in a database. This database serves as the foundational data for generative AI to learn from. The hardware used will consist of a server computer and a large-capacity storage device. The specific software used will include a database management system (DBMS) and data collection tools.

[0764] Next, the server builds a generative AI model based on the knowledge data and service information stored in the database. This model uses OpenAI's GPT-3 or other natural language processing models. This enables the AI ​​to generate appropriate answers to various questions from the user.

[0765] Users (in this case, factory workers) input questions through robots operating within the factory. The robots accept questions in natural language and send them to a server. The robots are equipped with a voice recognition system and a touchscreen, allowing users to input questions by voice or text.

[0766] When the server receives a question, it analyzes the question using natural language processing technology and passes it on to a generative AI. The AI ​​generates the optimal answer based on pre-trained knowledge data and service information. This process utilizes natural language processing technology to understand the meaning of the question and select the appropriate answer.

[0767] The generated response is sent from the server to the robot. The robot then presents this response to the user visually or audibly. For example, if a worker asks, "How do I change the oil in a machine tool?", the robot will display or voice the response it received from the server, saying, "The oil change is performed using the following steps..."

[0768] As a concrete example, if a new worker in a factory asks a question about how to inspect a conveyor belt, the following exchange may occur:

[0769] Question: How do you inspect a conveyor belt?

[0770] Answer: Belt conveyor inspection is performed using the following procedure:

[0771] 1. Stop the machine and turn off the power.

[0772] 2. Check the tension of the belt.

[0773] 3. Visually inspect the belt for any abnormalities.

[0774] 4. If any abnormalities are found, appropriate repairs will be carried out.

[0775] 5. After the inspection, restart the machine and check if it is working correctly.

[0776] In this way, by providing the necessary information for equipment maintenance work within the factory quickly and accurately, work efficiency can be improved.

[0777] Furthermore, the generated responses are recorded as logs on the server and used for future improvements and evaluations. This system is expected to enable new workers to quickly acquire the necessary knowledge and significantly improve the efficiency of maintenance work within the factory.

[0778] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0779] Step 1:

[0780] The server collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores it in a database. This collection process is performed regularly, and the information is organized using a database management system (DBMS). The input is internal company data, and the output is a structured database.

[0781] Step 2:

[0782] The server extracts data from knowledge data and service information stored in the database and provides it to a generative AI (in this case, OpenAI's GPT-3). The AI ​​learns from this data and becomes capable of operating as a natural language processing model. The input is knowledge data from the database, and the output is the trained generative AI model. Specifically, the process involves data formatting and feeding the data to the AI ​​model.

[0783] Step 3:

[0784] Users input questions to factory robots via voice or text. The robots receive user questions using a voice recognition system or touchscreen. The input is the user's question, and the output is question data in digital format. Specifically, voice recognition processing is performed to convert speech to text.

[0785] Step 4:

[0786] The robot sends the received question to the server. The server receives this question and analyzes it using natural language processing technology. The input is the question data sent by the robot, and the output is the analyzed question data. Specifically, the operation includes semantic analysis of the question content and extraction of related data.

[0787] Step 5:

[0788] The server passes the analyzed question to the generative AI, which generates the optimal answer based on knowledge data and service information. The input is the analyzed question data, and the output is the generated answer. Specifically, the AI ​​model generates the answer. Example of a prompt: "Question: How do I change the oil in a machine tool? Answer: The oil change is performed using the following steps..."

[0789] Step 6:

[0790] The generated response is sent from the server to the robot. The robot receives this response and presents it to the user visually or audibly. The input is the response data from the server, and the output is a display or audio output in a format that the user can see. Specific actions include displaying the response on a screen and outputting audio from a speaker.

[0791] Step 7:

[0792] The user reviews the responses provided by the robot and performs the necessary maintenance tasks. The input is the robot's response information, and the output is the user's actions. Specific actions include the user performing maintenance tasks.

[0793] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0794] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions. Specific embodiments for carrying out this invention are described below.

[0795] Data Ingestion

[0796] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[0797] Data Learning

[0798] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to various user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[0799] Emotional engine integration

[0800] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. Specifically, when the user inputs a question through their device, the engine analyzes their emotions in real time using voice and camera data. The analysis results are fed back to a generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0801] Questions accepted

[0802] The device has an interface for receiving questions from the user. The user inputs questions into the device using natural language, and the device sends those questions to the server. It also collects the user's voice and camera footage in real time and sends it to the emotion engine.

[0803] Question Processing

[0804] The server receives questions from users and passes them to a generative AI for analysis. In parallel, the emotion engine analyzes audio and video data from the user and evaluates the user's emotional state. The obtained emotional information is fed back to the generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0805] Submitting and displaying responses

[0806] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[0807] Specific example

[0808] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?", and simultaneously reads the question aloud. The terminal sends the question along with Tanaka's voice data to the server. The server uses generative AI to analyze the question and an emotion engine to evaluate Tanaka's emotion from his voice tone. Based on knowledge data and service information in the database, the generative AI generates the answer, "The new product X has high processing power and excellent security features, and provides a user-friendly interface." Based on feedback from the emotion engine, the tone of the answer is adjusted to match Tanaka's emotion. The server sends the adjusted answer to Tanaka's terminal, where Tanaka confirms the answer.

[0809] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[0810] The following describes the processing flow.

[0811] Step 1:

[0812] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database, which provides the foundational data for generative AI to learn from.

[0813] Step 2:

[0814] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to user questions. In the learning process, natural language processing techniques are used to analyze the data and recognize patterns that fit the questions.

[0815] Step 3:

[0816] The server prepares to utilize the emotion engine. The emotion engine analyzes the user's voice data and camera footage to recognize the user's emotional state. The emotion engine evaluates emotions in real time based on voice tone and facial expressions, and feeds the obtained information back to the generative AI.

[0817] Step 4:

[0818] The user inputs a question into the device using natural language. The device provides an interface for inputting questions and receives them using text boxes or voice input functions.

[0819] Step 5:

[0820] The device sends the user's question text to the server. Simultaneously, if voice input is used, the voice data is also sent to the server. Furthermore, it may also send user facial expression data obtained from the camera feed.

[0821] Step 6:

[0822] The server receives question text, audio data, and facial expression data sent from the terminal. First, the received question text is passed to a generative AI to analyze the question. Next, the emotion engine analyzes the audio data and facial expression data to evaluate the user's emotional state. For example, it identifies emotions such as joy, sadness, surprise, and anxiety.

[0823] Step 7:

[0824] The generative AI searches the database for appropriate knowledge data and service information based on the analyzed question content and generates an answer. At the same time, it receives emotion evaluation data from the emotion engine and adjusts the tone and content of the answer according to the user's emotions. For example, if the user is agitated, it will adjust the answer to be provided in a calm tone.

[0825] Step 8:

[0826] The server sends the generated response to the terminal. The adjusted response data is sent in a format that is easy for the user to understand.

[0827] Step 9:

[0828] The device receives the response sent from the server. The device displays the response to the user visually or audibly. Text-based responses are displayed on the screen, and audio-based responses are played through the speaker.

[0829] Step 10:

[0830] The user reviews the provided answers. This ensures that the user receives appropriate answers to their questions, and that the answers are delivered in an emotionally sensitive tone, resulting in a more satisfying experience.

[0831] Specific example

[0832] Step 1:

[0833] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[0834] Step 2:

[0835] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[0836] Step 3:

[0837] The server sets up an emotion engine, enabling the analysis of voice and facial expression data.

[0838] Step 4:

[0839] User Tanaka types "What are the main features of the new product X?" into the device, and the question is read aloud. The camera captures Tanaka's facial expression.

[0840] Step 5:

[0841] The device sends Tanaka's question text, audio data, and facial expression data to the server.

[0842] Step 6:

[0843] The server passes the question text to the generative AI, and the voice data and facial expression data to the emotion engine.

[0844] Step 7:

[0845] The generative AI generates the response, "New product X has high-speed processing capabilities and excellent security features, and offers a user-friendly interface." The emotion engine determines from Tanaka's voice tone that he is excited and adjusts the tone of the response to be calmer.

[0846] Step 8:

[0847] The server sends the adjusted response to the terminal.

[0848] Step 9:

[0849] The terminal receives the response from the server, displays it to Tanaka, and reads it aloud.

[0850] Step 10:

[0851] Ms. Tanaka finds satisfaction in the appropriate answers to her questions and in the responses provided in an emotionally considerate tone.

[0852] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[0853] (Example 2)

[0854] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0855] There is a challenge in that sales representatives, especially new sales representatives, have difficulty quickly and accurately acquiring the necessary knowledge and providing appropriate responses while considering the user's feelings. In the current system, knowledge acquisition and recognition of user feelings are separated, and there is a lack of a system that functions as an integrated whole. As a result, the training efficiency of new sales representatives is low, and the quality of customer service cannot be guaranteed.

[0856] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, and means including an emotion engine that analyzes the user's voice data and video data to recognize their emotional state. As a result, sales representatives can quickly and accurately acquire the necessary knowledge and, furthermore, respond appropriately according to the user's emotions.

[0857] "Knowledge data" refers to data that compiles business knowledge and information, such as technical documents and FAQ data within a company.

[0858] "Service information" refers to data such as detailed information, specifications, and instruction manuals regarding the products and services offered.

[0859] "Generative AI" refers to artificial intelligence that learns from collected knowledge data and service information and generates appropriate answers to user questions.

[0860] A "natural language processing model" is a machine learning model used in generative AI to understand and generate natural human language.

[0861] An "emotion engine" refers to a technology that analyzes a user's voice and video data to recognize their emotional state.

[0862] A "database" is an information storage device for efficiently storing and managing knowledge data and service information.

[0863] "User" refers to a customer or sales representative who uses the system.

[0864] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions.

[0865] Data Ingestion

[0866] The server regularly collects knowledge data such as technical documents, FAQs, training materials, and detailed information about products and services offered within the company, and stores it in a database. This is done using scraping tools and APIs. By updating the information in the database in a timely manner, generative AI can always learn based on the latest information.

[0867] Data Learning

[0868] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from this data using natural language processing models such as OpenAI's GPT or Google's BERT. The AI ​​analyzes vast amounts of text data, recognizes patterns to respond to questions, and builds a knowledge base to generate answers.

[0869] Emotional engine integration

[0870] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. This emotion engine evaluates the user's emotions using technologies such as voice tone analysis and facial expression recognition. When the user inputs a question through the terminal, it uses voice and camera to perform real-time analysis and feeds the results back to the generative AI.

[0871] Questions accepted

[0872] The device has an interface for receiving questions from users, and users can input questions in natural language. Specifically, questions are submitted through a chatbot or text input field. The questions entered by the user are sent to the server along with audio and video data.

[0873] Question processing and response submission

[0874] The server passes the question received from the user to a generative AI, which analyzes the question. The generative AI retrieves relevant information from a database and generates the optimal answer. Simultaneously, an emotion engine evaluates the user's emotional state and feeds this information back to the generative AI. The generated answer is adjusted in tone and content according to the user's emotions. After the adjustments are complete, the server sends the answer to the terminal, which displays it to the user visually and audibly.

[0875] Specific example

[0876] Let's imagine a scenario where a new sales representative types the question, "What are the main features of the new product X?" and has it read aloud. The terminal collects the question and voice data and sends it to the server. The server's generative AI analyzes the question and derives the answer, "The new product X has high processing power, excellent security features, and a user-friendly interface." The emotion engine evaluates the user's voice tone as "interested" and adjusts the tone of the answer to match that emotion. As a result, the adjusted answer is sent to the user's terminal, and the user can confirm it in both voice and text.

[0877] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. This system is expected to significantly improve the quality of customer service.

[0878] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0879] Step 1: Data Ingest

[0880] The server periodically collects knowledge data, such as technical documents, FAQs, training materials, and detailed product and service information, from within the company and stores it in a database. This involves using scraping tools and APIs to retrieve data, normalizing it, and then storing it in the database. Input is data from various knowledge sources within the company, and output is knowledge data stored in the server's database. Specifically, the server periodically executes scripts and sends API requests to retrieve the latest data.

[0881] Step 2: Data Training

[0882] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from the data using natural language processing models such as OpenAI's GPT or Google's BERT. The input is knowledge data and service information in the database, and the output is the knowledge base built by the generative AI. Specifically, the AI ​​learns a model to analyze the data, recognize question-answer patterns, and generate relevant answers.

[0883] Step 3: Collecting emotional data

[0884] The device collects user audio and video data in real time and sends it to the emotion engine. This data is acquired using the microphone and camera built into the device. The input is the user's audio and video, and the output is the data sent to the emotion engine. Specifically, the device records audio, captures video with its camera, and sends this data to the server.

[0885] Step 4: Analysis of emotional state

[0886] The server's emotion engine analyzes audio and video data to recognize the user's emotional state. Specifically, it evaluates the emotional state using voice tone analysis and facial recognition technology. The input is the user's audio and video data, and the output is the analyzed emotional state information. The server evaluates emotions using voice tone analysis algorithms and facial recognition technology.

[0887] Step 5: Receiving the Question

[0888] The user inputs a question in natural language using the device. The device then sends the question to the server. The input is the text of the question entered by the user, and the output is the question data sent to the server. Specifically, the user submits a question through a chatbot or text input field, and the device forwards it to the server.

[0889] Step 6: Analyzing the Question

[0890] The server receives questions from users and passes them to a generative AI for analysis. The generative AI queries a knowledge base and generates the optimal answer. The input is the question data sent to the server, and the output is the generated answer data. Specifically, the AI ​​analyzes the question, retrieves relevant information from the knowledge base, and constructs the answer.

[0891] Step 7: Feedback on emotional information

[0892] The emotion engine feeds the analyzed emotional state back to the generative AI, which then adjusts the tone and content of the response. The input is the analyzed emotional state information, and the output is the adjusted response data. Specifically, the generative AI takes the emotional information into consideration and adjusts the tone of the response to include phrases such as "providing reassurance" or "including words of encouragement."

[0893] Step 8: Submit and view your response

[0894] The server sends the adjusted response to the user's device. The device displays the received response to the user. The input is the adjusted response data, and the output is the response information that the user receives visually and audibly. Specifically, the device provides the response to the user by reading it aloud using speech synthesis or by displaying it as a text message.

[0895] Through the processing steps described above, the system provides prompt, emotionally sensitive, and appropriate answers to user questions. By combining specific technologies and functions, it enables sales representatives to acquire knowledge and improves the quality of customer service.

[0896] (Application Example 2)

[0897] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0898] Conventional sales support systems can provide accurate answers to user questions, but they do not take into account the user's emotional state. This can lead to insufficient improvement in customer satisfaction and a decline in the quality of the customer experience. Furthermore, it is difficult for new sales representatives to quickly acquire the necessary knowledge. Therefore, there is a need for a system that allows new sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses tailored to the user's emotional state.

[0899] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for analyzing the user's voice data and video data and evaluating their emotional state, and means for adjusting the tone and content of the generated response according to the user's emotions. This enables sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses that take into account the user's emotions.

[0900] A "database" is an information system for storing knowledge data and service information, providing the foundational data that AI will later learn from.

[0901] "Knowledge data" refers to data that systematically compiles specific knowledge and information, such as technical documents and FAQ data within a company.

[0902] "Service information" refers to detailed information about the products and services offered, which is necessary to generate appropriate answers to user inquiries.

[0903] "Generative AI" refers to artificial intelligence that generates appropriate answers to user questions based on training data.

[0904] A "natural language processing model" is a machine learning model that understands and analyzes natural language text, such as questions from users, to generate appropriate answers.

[0905] "User voice data" refers to voice information provided by the user through a microphone, and is used to analyze the user's emotional state.

[0906] "Video data" refers to image information captured in real time via a camera, such as the user's facial expressions, and is used for emotion analysis.

[0907] "Emotional state" refers to the feelings and moods a user experiences at a particular moment, and is recognized by analyzing audio and video data.

[0908] "Response tone" refers to the tone and nuances of expression in the generated response, which are adjusted according to the user's emotional state.

[0909] A "terminal" is a device used by a user to input questions and receive answers, and includes smartphones and smart glasses.

[0910] This invention relates to a sales representative support system that combines a generative AI and an emotion engine, and will be described with a particular focus on its application in physical stores. Specific embodiments for carrying out this invention are described below.

[0911] Database and Server Configuration

[0912] The server periodically collects knowledge data such as technical documents and FAQs within the company, as well as detailed information about the products and services offered, and stores it in a database. This database provides the foundational data for generative AI to learn from.

[0913] Building Generative AI

[0914] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to various user questions. It is important to recognize patterns for responding to questions using natural language processing techniques.

[0915] Introducing an emotional engine

[0916] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. Specifically, when the user inputs a question through the terminal, the system uses voice and camera data to analyze their emotions in real time. The EmotionRecognizer library is used for emotion analysis.

[0917] Question reception and analysis

[0918] The device (e.g., a smartphone or smart glasses) has an interface that receives questions from the user. The user inputs the question in natural language, and the device sends the question to the server. It also collects the user's voice and camera footage in real time and sends it to the emotion engine.

[0919] Response generation and sentiment adjustment

[0920] The server receives questions from users and passes them to a generative AI for analysis. In parallel, the emotion engine analyzes audio and video data from the user and evaluates the user's emotional state. The obtained emotional information is fed back to the generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[0921] Displayed as "Response submitted".

[0922] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[0923] Specific example

[0924] For example, if a user asks "What is the warranty period for this product?" in a physical store, the smart glasses' camera might capture the user's facial expression and recognize that they are angry. Based on this information, a generative AI might generate a response such as "Excuse me, are you dissatisfied? The warranty period for this product is two years," and display it on the device in a tone appropriate to the user's emotions.

[0925] Example of a prompt

[0926] For example, here are some examples of prompt statements to input into a generative AI model:

[0927] Q: What is the warranty period for this product?

[0928] A:

[0929] Thus, the present invention provides a system that enables sales representatives to quickly and accurately acquire necessary knowledge, while also considering the user's feelings and providing appropriate responses. This can significantly improve the quality of customer service.

[0930] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0931] Step 1:

[0932] The user enters a question through a device (smartphone or smart glasses). The user enters the question in natural language and has it read aloud. At this time, the device's microphone captures the audio data.

[0933] Input: User's question (text format), user's voice data

[0934] Output: Question (text data), audio data

[0935] Step 2:

[0936] The device uses a speech recognition library to convert the captured audio data into text. The conversion from audio data to text data takes place.

[0937] Input: Audio data

[0938] Output: Question (text data)

[0939] Step 3:

[0940] The device captures the user's facial expressions with its camera and generates video data. This prepares it for real-time analysis of the user's emotional state.

[0941] Input: User's facial expression (video data)

[0942] Output: Video data

[0943] Step 4:

[0944] The device sends text data and video data of the question to the server. The server receives this data and passes the question data to the generative AI and the video data to the emotion engine.

[0945] Input: Question (text data), video data

[0946] Output: Sending question data and video data to the server.

[0947] Step 5:

[0948] The server inputs question data into a generative AI model and generates an appropriate answer. The generative AI generates the answer based on knowledge data and service information. This answer is provided in text format.

[0949] Input: Question (text data)

[0950] Output: Generated response (text data)

[0951] Step 6:

[0952] The server analyzes the video data using an emotion engine to recognize the user's emotional state. The emotion engine analyzes the video data and evaluates the user's emotions (e.g., anger, joy, dissatisfaction, etc.).

[0953] Input: Video data

[0954] Output: User's emotional state

[0955] Step 7:

[0956] The server adjusts the tone of the generated response based on emotional state information fed back from the emotion engine. The tone and content are adjusted according to the emotion.

[0957] Input: Generated response (text data), user's emotional state

[0958] Output: Emotionally balanced and adjusted responses (text data)

[0959] Step 8:

[0960] The server sends the adjusted response to the terminal. The terminal displays the received adjusted response to the user visually and audibly. The user can receive the appropriate response in an adjusted tone.

[0961] Input: Adjusted response (text data)

[0962] Output: Adjusted response displayed to the user

[0963] Specific actions:

[0964] For example, if a user asks, "What is the warranty period for this product?" and the device's camera captures an angry expression, the server uses generative AI to generate the answer, "The warranty period for this product is two years." After the emotion engine evaluates the anger, it adjusts the tone of the response to, "I'm sorry, are you dissatisfied? The warranty period for this product is two years."

[0965] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0966] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0967] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0968] [Fourth Embodiment]

[0969] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0970] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0971] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0972] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0973] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0974] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0975] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0976] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0977] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0978] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0979] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0980] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0981] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0982] This invention relates to a sales representative support system utilizing generative AI, and is particularly aimed at enabling new sales representatives to quickly acquire necessary knowledge. Specific embodiments for carrying out this invention are described below.

[0983] Data Ingestion

[0984] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[0985] Data Learning

[0986] The server retrieves knowledge data and service information stored in the database and provides it to the generative AI. The AI ​​learns from this data and becomes able to generate appropriate answers to various questions from users. This generative AI utilizes natural language processing technology to analyze the content of the questions and provide the optimal answer.

[0987] Questions accepted

[0988] The terminal has an interface for receiving questions from the user. The user inputs the question into the terminal in natural language, and the terminal sends the question to the server. This allows the user to quickly ask for the information they need.

[0989] Question Processing

[0990] The server receives questions from users and analyzes them. The analyzed questions are passed to a generative AI, which generates answers based on knowledge data and service information. In this answer generation process, the AI ​​utilizes its learned knowledge to provide the most appropriate information for the question.

[0991] Submitting and displaying responses

[0992] The generated answers are sent from the server to the user's device. The device receives the sent answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[0993] Specific example

[0994] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?" The terminal receives the question and sends it to the server. The server uses generative AI to analyze the question and generates the best answer based on knowledge data and service information in its database. The answer is, "The new product X has high processing power, excellent security features, and a user-friendly interface." The server sends this answer to Tanaka's terminal, where Tanaka can check the answer.

[0995] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[0996] The following describes the processing flow.

[0997] Step 1:

[0998] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database. The server structures the collected data and stores it in the database for quick access.

[0999] Step 2:

[1000] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds knowledge to generate appropriate answers to user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[1001] Step 3:

[1002] The user enters a question into the terminal. The terminal receives this input and sends the question to the server. The terminal provides an interface that accepts natural language input from the user and sends the question data, ready to be sent, to the server.

[1003] Step 4:

[1004] The server receives questions sent from terminals. Next, the server passes the received questions to a generative AI for analysis. The generative AI understands the content of the questions and generates the optimal answer based on knowledge data and service information in the database.

[1005] Step 5:

[1006] The server receives a response from the generative AI. This response provides specific and appropriate information in response to the user's question. The server then performs the necessary transmission processing to send this response to the user's terminal.

[1007] Step 6:

[1008] The terminal receives the response sent from the server. The terminal then visually displays this response to the user. The user can then review the displayed response and obtain information regarding their question.

[1009] Specific example

[1010] Step 1:

[1011] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[1012] Step 2:

[1013] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[1014] Step 3:

[1015] User Tanaka enters the question "What are the main features of the new product X?" into the terminal.

[1016] Step 4:

[1017] The terminal sends Tanaka's question to the server. The server receives the question, sends it to a generative AI, and analyzes it.

[1018] Step 5:

[1019] The generative AI generates the response, "New product X has high-speed processing capabilities, excellent security features, and a user-friendly interface." The server receives this response and sends it to Tanaka's terminal.

[1020] Step 6:

[1021] Tanaka's device receives the response from the server and displays the content visually to Tanaka. Tanaka checks the response regarding the main features of the new product X.

[1022] In this way, the present invention provides an efficient means for sales representatives to quickly and accurately acquire the necessary knowledge, thereby improving the quality of customer service.

[1023] (Example 1)

[1024] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1025] New sales representatives are required to acquire necessary knowledge quickly and efficiently to improve the quality of their customer service. However, traditional methods require them to individually search and understand vast amounts of knowledge data and product information, which is time-consuming and laborious. A system is needed to solve this problem.

[1026] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1027] In this invention, the server includes means for storing knowledge data and product information in a database, means for building a generative AI model that learns the knowledge data and product information, means for receiving user questions in natural language, means for analyzing the received questions and generating answers using the generative AI model, means for sending the generated answers to the user's terminal, and means for visually displaying the transmitted answers to the user. This enables new sales representatives to efficiently acquire necessary information and respond to customers quickly and accurately.

[1028] A "database" is an information storage system that efficiently manages and makes searchable knowledge data and product information.

[1029] "Knowledge data" refers to all data containing specific knowledge and information within a company, such as technical documents, FAQ data, and training materials.

[1030] "Product information" refers to detailed data about products and services offered by a company, including technical specifications, functions, and features.

[1031] A "generative AI model" refers to an artificial intelligence model that performs natural language processing based on knowledge data and product information to generate appropriate answers to questions.

[1032] A "user" refers to the entity that uses the system to input questions and receive answers. This is primarily intended for new sales representatives.

[1033] "Natural language" refers to the language that humans use on a daily basis, and specifically to the question format that users input into the system.

[1034] "Analysis" refers to the process of understanding the received question, extracting its contents, and organizing them.

[1035] "Answer generation" refers to the process of creating appropriate answers using a generative AI model based on the analyzed question content.

[1036] A "device" refers to a device used by a user to input questions and receive answers. This includes PCs and tablets.

[1037] "Visual display" refers to displaying the generated response on the device screen in a way that the user can see and understand.

[1038] This invention is a system that supports new sales representatives using a generative AI model. In particular, it aims to enable the rapid and accurate acquisition of necessary information by efficiently utilizing knowledge data and product information. Specific embodiments for carrying out this invention are described below.

[1039] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services the company provides, and stores it in a database (e.g., MySQL or PostgreSQL). This data later serves as foundational data for training generative AI models (e.g., OpenAI's GPT-3).

[1040] The server provides the AI ​​model with knowledge data and product information stored in the database, and this AI model learns from this data. This allows it to generate appropriate answers to user questions.

[1041] The user (new sales representative) uses a provided device (e.g., a PC or tablet) to input a question in natural language. The device then sends the question to the server.

[1042] The server receives questions sent from terminals and analyzes them using a generative AI model. Based on the analyzed question, the AI ​​model refers to knowledge data and product information in the database to generate the optimal answer. This answer generation process fully utilizes the knowledge the AI ​​model has learned.

[1043] The generated answers are sent from the server to the user's device, which receives the answers and displays them visually to the user. This allows the user to quickly obtain appropriate answers to their questions.

[1044] As a concrete example, consider a scenario where a new sales representative enters the question, "What are the main features of the new product X?" into a terminal. In this case, the terminal sends the question to a server, which uses a generative AI model to analyze the question. As a result, an answer is generated, such as, "The new product X has high-speed processing capabilities, excellent security features, and a user-friendly interface," and this answer is sent to the user's terminal. The user can then view the answer on their terminal.

[1045] The following is an example of a prompt message.

[1046] In response to the question, "What are the main features of the new product X?", generate a detailed description of the features of product X.

[1047] In this way, the present invention provides a means for sales representatives to quickly and accurately obtain necessary information, thereby improving the quality of customer service.

[1048] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1049] Step 1: Data Ingest

[1050] The server collects technical documents, FAQ data, training materials, and detailed information about products and services within the company. This data is periodically stored in a database by the server. The input consists of various documents and data within the company, while the output is organized knowledge data and product information stored in the database.

[1051] Step 2: Data Training

[1052] The server provides a generative AI model with knowledge data and product information stored in a database. The generative AI model (e.g., GPT-3) learns from this data. The input is knowledge data and product information in the database, and the output is a generative AI model capable of generating appropriate answers to questions.

[1053] Step 3: Question Acceptance

[1054] The user uses a terminal to input a question in natural language. The terminal sends this question to the server. The input is the user's question in natural language, and the output is that the question has been sent to the server.

[1055] Step 4: Questionnaire Analysis

[1056] The server receives questions sent from the terminal and analyzes them using a generative AI model. The input is the user's question, and the output is the analyzed question. The server uses natural language processing techniques to properly understand the questions.

[1057] Step 5: Generate Response

[1058] The server uses a generative AI model to generate the optimal answer based on the analyzed question content. The input consists of the analyzed question content, knowledge data, and product information, while the output is the generated answer. The generated answer is based on the knowledge learned by the AI ​​model.

[1059] Step 6: Submit your response

[1060] The server sends the generated response to the user's terminal. The input is the generated response, and the output is that the response is sent to the terminal.

[1061] Step 7: Display your answer

[1062] The terminal receives the response sent from the server and displays it visually to the user. The input is the response sent from the server, and the output is the response displayed on the terminal's screen. The user checks the response on the terminal and obtains the necessary information.

[1063] (Application Example 1)

[1064] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1065] In conventional factory equipment maintenance, it was difficult for workers to quickly and accurately obtain the necessary information. In particular, new workers were unfamiliar with equipment maintenance procedures and specific operating methods, and it took time for them to acquire the necessary knowledge. This led to decreased efficiency in maintenance work and an increased risk of equipment failure due to improper operation.

[1066] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1067] In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for receiving questions from users, means for analyzing the received questions and generating answers using the generative AI, means for sending the generated answers to the user, means for displaying the sent answers to the user, and means for receiving questions and displaying answers to robots operating in the factory. This makes it possible to quickly and accurately acquire information necessary for equipment maintenance work in the factory and improve work efficiency.

[1068] A "database" is a system for systematically accumulating and managing information, and it plays a role in storing knowledge data and service information.

[1069] "Knowledge data" refers to a collection of data containing specialized knowledge and information within a company, such as technical documents and FAQ data.

[1070] "Service information" refers to a collection of data containing detailed information about the products and services offered by a company.

[1071] "Generative AI" is artificial intelligence that learns from knowledge data and service information to generate the best possible answers to user questions.

[1072] "Means for receiving questions from users" refers to devices or systems that have an interface that allows users to input questions in natural language.

[1073] "Means for analyzing questions and generating answers using generative AI" refers to devices or systems that analyze questions received from users and execute a process to generate the most appropriate answer to those questions.

[1074] "Means for sending generated answers to users" refers to communication methods for providing users with answers generated by generative AI.

[1075] "Means for displaying submitted responses to the user" refers to devices or systems that allow the user to visually confirm the responses provided.

[1076] "Robots operating in factories" are robots designed to perform tasks in factory manufacturing areas or equipment maintenance sites.

[1077] "Means for receiving questions and displaying answers" refers to devices or systems that have the function of receiving questions from users and displaying corresponding answers, such as robots in a factory.

[1078] This invention provides a system for supporting equipment maintenance work within a factory. Specific embodiments are described below.

[1079] First, the server periodically collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores this data in a database. This database serves as the foundational data for generative AI to learn from. The hardware used will consist of a server computer and a large-capacity storage device. The specific software used will include a database management system (DBMS) and data collection tools.

[1080] Next, the server builds a generative AI model based on the knowledge data and service information stored in the database. This model uses OpenAI's GPT-3 or other natural language processing models. This enables the AI ​​to generate appropriate answers to various questions from the user.

[1081] Users (in this case, factory workers) input questions through robots operating within the factory. The robots accept questions in natural language and send them to a server. The robots are equipped with a voice recognition system and a touchscreen, allowing users to input questions by voice or text.

[1082] When the server receives a question, it analyzes the question using natural language processing technology and passes it on to a generative AI. The AI ​​generates the optimal answer based on pre-trained knowledge data and service information. This process utilizes natural language processing technology to understand the meaning of the question and select the appropriate answer.

[1083] The generated response is sent from the server to the robot. The robot then presents this response to the user visually or audibly. For example, if a worker asks, "How do I change the oil in a machine tool?", the robot will display or voice the response it received from the server, saying, "The oil change is performed using the following steps..."

[1084] As a concrete example, if a new worker in a factory asks a question about how to inspect a conveyor belt, the following exchange may occur:

[1085] Question: How do you inspect a conveyor belt?

[1086] Answer: Belt conveyor inspection is performed using the following procedure:

[1087] 1. Stop the machine and turn off the power.

[1088] 2. Check the tension of the belt.

[1089] 3. Visually inspect the belt for any abnormalities.

[1090] 4. If any abnormalities are found, appropriate repairs will be carried out.

[1091] 5. After the inspection, restart the machine and check if it is working correctly.

[1092] In this way, by providing the necessary information for equipment maintenance work within the factory quickly and accurately, work efficiency can be improved.

[1093] Furthermore, the generated responses are recorded as logs on the server and used for future improvements and evaluations. This system is expected to enable new workers to quickly acquire the necessary knowledge and significantly improve the efficiency of maintenance work within the factory.

[1094] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1095] Step 1:

[1096] The server collects knowledge data and service information, such as technical documents, FAQ data, training materials, and product information, from within the company and stores it in a database. This collection process is performed regularly, and the information is organized using a database management system (DBMS). The input is internal company data, and the output is a structured database.

[1097] Step 2:

[1098] The server extracts data from knowledge data and service information stored in the database and provides it to a generative AI (in this case, OpenAI's GPT-3). The AI ​​learns from this data and becomes capable of operating as a natural language processing model. The input is knowledge data from the database, and the output is the trained generative AI model. Specifically, the process involves data formatting and feeding the data to the AI ​​model.

[1099] Step 3:

[1100] Users input questions to factory robots via voice or text. The robots receive user questions using a voice recognition system or touchscreen. The input is the user's question, and the output is question data in digital format. Specifically, voice recognition processing is performed to convert speech to text.

[1101] Step 4:

[1102] The robot sends the received question to the server. The server receives this question and analyzes it using natural language processing technology. The input is the question data sent by the robot, and the output is the analyzed question data. Specifically, the operation includes semantic analysis of the question content and extraction of related data.

[1103] Step 5:

[1104] The server passes the analyzed question to the generative AI, which generates the optimal answer based on knowledge data and service information. The input is the analyzed question data, and the output is the generated answer. Specifically, the AI ​​model generates the answer. Example of a prompt: "Question: How do I change the oil in a machine tool? Answer: The oil change is performed using the following steps..."

[1105] Step 6:

[1106] The generated response is sent from the server to the robot. The robot receives this response and presents it to the user visually or audibly. The input is the response data from the server, and the output is a display or audio output in a format that the user can see. Specific actions include displaying the response on a screen and outputting audio from a speaker.

[1107] Step 7:

[1108] The user reviews the responses provided by the robot and performs the necessary maintenance tasks. The input is the robot's response information, and the output is the user's actions. Specific actions include the user performing maintenance tasks.

[1109] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1110] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions. Specific embodiments for carrying out this invention are described below.

[1111] Data Ingestion

[1112] The server periodically collects knowledge data such as technical documents, FAQs, and training materials from within the company, as well as detailed information about the products and services offered, and stores it in a database. This database later serves as foundational data for generative AI to learn from.

[1113] Data Learning

[1114] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to various user questions. During the AI's learning process, it analyzes the data using natural language processing techniques and recognizes patterns to respond to questions.

[1115] Emotional engine integration

[1116] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. Specifically, when the user inputs a question through their device, the engine analyzes their emotions in real time using voice and camera data. The analysis results are fed back to a generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[1117] Questions accepted

[1118] The device has an interface for receiving questions from the user. The user inputs questions into the device using natural language, and the device sends those questions to the server. It also collects the user's voice and camera footage in real time and sends it to the emotion engine.

[1119] Question Processing

[1120] The server receives questions from users and passes them to a generative AI for analysis. In parallel, the emotion engine analyzes audio and video data from the user and evaluates the user's emotional state. The obtained emotional information is fed back to the generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[1121] Submitting and displaying responses

[1122] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[1123] Specific example

[1124] As a concrete example, let's assume that Tanaka, a new sales representative, wants to know about a new feature of a certain product. Tanaka uses a terminal to input the question, "What are the main features of the new product X?", and simultaneously reads the question aloud. The terminal sends the question along with Tanaka's voice data to the server. The server uses generative AI to analyze the question and an emotion engine to evaluate Tanaka's emotion from his voice tone. Based on knowledge data and service information in the database, the generative AI generates the answer, "The new product X has high processing power and excellent security features, and provides a user-friendly interface." Based on feedback from the emotion engine, the tone of the answer is adjusted to match Tanaka's emotion. The server sends the adjusted answer to Tanaka's terminal, where Tanaka confirms the answer.

[1125] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[1126] The following describes the processing flow.

[1127] Step 1:

[1128] The server collects knowledge data such as technical documents, FAQs, and training materials within the company, as well as detailed information about the products and services offered. This information is stored in a database, which provides the foundational data for generative AI to learn from.

[1129] Step 2:

[1130] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to user questions. In the learning process, natural language processing techniques are used to analyze the data and recognize patterns that fit the questions.

[1131] Step 3:

[1132] The server prepares to utilize the emotion engine. The emotion engine analyzes the user's voice data and camera footage to recognize the user's emotional state. The emotion engine evaluates emotions in real time based on voice tone and facial expressions, and feeds the obtained information back to the generative AI.

[1133] Step 4:

[1134] The user inputs a question into the device using natural language. The device provides an interface for inputting questions and receives them using text boxes or voice input functions.

[1135] Step 5:

[1136] The device sends the user's question text to the server. Simultaneously, if voice input is used, the voice data is also sent to the server. Furthermore, it may also send user facial expression data obtained from the camera feed.

[1137] Step 6:

[1138] The server receives question text, audio data, and facial expression data sent from the terminal. First, the received question text is passed to a generative AI to analyze the question. Next, the emotion engine analyzes the audio data and facial expression data to evaluate the user's emotional state. For example, it identifies emotions such as joy, sadness, surprise, and anxiety.

[1139] Step 7:

[1140] The generative AI searches the database for appropriate knowledge data and service information based on the analyzed question content and generates an answer. At the same time, it receives emotion evaluation data from the emotion engine and adjusts the tone and content of the answer according to the user's emotions. For example, if the user is agitated, it will adjust the answer to be provided in a calm tone.

[1141] Step 8:

[1142] The server sends the generated response to the terminal. The adjusted response data is sent in a format that is easy for the user to understand.

[1143] Step 9:

[1144] The device receives the response sent from the server. The device displays the response to the user visually or audibly. Text-based responses are displayed on the screen, and audio-based responses are played through the speaker.

[1145] Step 10:

[1146] The user reviews the provided answers. This ensures that the user receives appropriate answers to their questions, and that the answers are delivered in an emotionally sensitive tone, resulting in a more satisfying experience.

[1147] Specific example

[1148] Step 1:

[1149] The server collects knowledge data such as "technical specifications for new products" and "user guidelines," and stores it in a database.

[1150] Step 2:

[1151] The server provides this knowledge data to the generative AI, which then learns from it. The generative AI acquires detailed knowledge about the product's features and usage.

[1152] Step 3:

[1153] The server sets up an emotion engine, enabling the analysis of voice and facial expression data.

[1154] Step 4:

[1155] User Tanaka types "What are the main features of the new product X?" into the device, and the question is read aloud. The camera captures Tanaka's facial expression.

[1156] Step 5:

[1157] The device sends Tanaka's question text, audio data, and facial expression data to the server.

[1158] Step 6:

[1159] The server passes the question text to the generative AI, and the voice data and facial expression data to the emotion engine.

[1160] Step 7:

[1161] The generative AI generates the response, "New product X has high-speed processing capabilities and excellent security features, and offers a user-friendly interface." The emotion engine determines from Tanaka's voice tone that he is excited and adjusts the tone of the response to be calmer.

[1162] Step 8:

[1163] The server sends the adjusted response to the terminal.

[1164] Step 9:

[1165] The terminal receives the response from the server, displays it to Tanaka, and reads it aloud.

[1166] Step 10:

[1167] Ms. Tanaka finds satisfaction in the appropriate answers to her questions and in the responses provided in an emotionally considerate tone.

[1168] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. As a result, the quality of customer service can be improved.

[1169] (Example 2)

[1170] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1171] There is a challenge in that sales representatives, especially new sales representatives, have difficulty quickly and accurately acquiring the necessary knowledge and providing appropriate responses while considering the user's feelings. In the current system, knowledge acquisition and recognition of user feelings are separated, and there is a lack of a system that functions as an integrated whole. As a result, the training efficiency of new sales representatives is low, and the quality of customer service cannot be guaranteed.

[1172] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, and means including an emotion engine that analyzes the user's voice data and video data to recognize their emotional state. As a result, sales representatives can quickly and accurately acquire the necessary knowledge and, furthermore, respond appropriately according to the user's emotions.

[1173] "Knowledge data" refers to data that compiles business knowledge and information, such as technical documents and FAQ data within a company.

[1174] "Service information" refers to data such as detailed information, specifications, and instruction manuals regarding the products and services offered.

[1175] "Generative AI" refers to artificial intelligence that learns from collected knowledge data and service information and generates appropriate answers to user questions.

[1176] A "natural language processing model" is a machine learning model used in generative AI to understand and generate natural human language.

[1177] An "emotion engine" refers to a technology that analyzes a user's voice and video data to recognize their emotional state.

[1178] A "database" is an information storage device for efficiently storing and managing knowledge data and service information.

[1179] "User" refers to a customer or sales representative who uses the system.

[1180] This invention relates to a sales representative support system that combines generative AI and an emotion engine, and aims to enable new sales representatives to quickly acquire necessary knowledge and to provide more appropriate responses by recognizing the user's emotions.

[1181] Data Ingestion

[1182] The server regularly collects knowledge data such as technical documents, FAQs, training materials, and detailed information about products and services offered within the company, and stores it in a database. This is done using scraping tools and APIs. By updating the information in the database in a timely manner, generative AI can always learn based on the latest information.

[1183] Data Learning

[1184] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from this data using natural language processing models such as OpenAI's GPT or Google's BERT. The AI ​​analyzes vast amounts of text data, recognizes patterns to respond to questions, and builds a knowledge base to generate answers.

[1185] Emotional engine integration

[1186] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. This emotion engine evaluates the user's emotions using technologies such as voice tone analysis and facial expression recognition. When the user inputs a question through the terminal, it uses voice and camera to perform real-time analysis and feeds the results back to the generative AI.

[1187] Questions accepted

[1188] The device has an interface for receiving questions from users, and users can input questions in natural language. Specifically, questions are submitted through a chatbot or text input field. The questions entered by the user are sent to the server along with audio and video data.

[1189] Question processing and response submission

[1190] The server passes the question received from the user to a generative AI, which analyzes the question. The generative AI retrieves relevant information from a database and generates the optimal answer. Simultaneously, an emotion engine evaluates the user's emotional state and feeds this information back to the generative AI. The generated answer is adjusted in tone and content according to the user's emotions. After the adjustments are complete, the server sends the answer to the terminal, which displays it to the user visually and audibly.

[1191] Specific example

[1192] Let's imagine a scenario where a new sales representative types the question, "What are the main features of the new product X?" and has it read aloud. The terminal collects the question and voice data and sends it to the server. The server's generative AI analyzes the question and derives the answer, "The new product X has high processing power, excellent security features, and a user-friendly interface." The emotion engine evaluates the user's voice tone as "interested" and adjusts the tone of the answer to match that emotion. As a result, the adjusted answer is sent to the user's terminal, and the user can confirm it in both voice and text.

[1193] In this way, the present invention provides a means for sales representatives to quickly and accurately acquire the necessary knowledge, and further enables appropriate responses that take into account the user's feelings. This system is expected to significantly improve the quality of customer service.

[1194] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1195] Step 1: Data Ingest

[1196] The server periodically collects knowledge data, such as technical documents, FAQs, training materials, and detailed product and service information, from within the company and stores it in a database. This involves using scraping tools and APIs to retrieve data, normalizing it, and then storing it in the database. Input is data from various knowledge sources within the company, and output is knowledge data stored in the server's database. Specifically, the server periodically executes scripts and sends API requests to retrieve the latest data.

[1197] Step 2: Data Training

[1198] The server provides knowledge data and service information stored in the database to the generative AI. The generative AI learns from the data using natural language processing models such as OpenAI's GPT or Google's BERT. The input is knowledge data and service information in the database, and the output is the knowledge base built by the generative AI. Specifically, the AI ​​learns a model to analyze the data, recognize question-answer patterns, and generate relevant answers.

[1199] Step 3: Collecting emotional data

[1200] The device collects user audio and video data in real time and sends it to the emotion engine. This data is acquired using the microphone and camera built into the device. The input is the user's audio and video, and the output is the data sent to the emotion engine. Specifically, the device records audio, captures video with its camera, and sends this data to the server.

[1201] Step 4: Analysis of emotional state

[1202] The server's emotion engine analyzes audio and video data to recognize the user's emotional state. Specifically, it evaluates the emotional state using voice tone analysis and facial recognition technology. The input is the user's audio and video data, and the output is the analyzed emotional state information. The server evaluates emotions using voice tone analysis algorithms and facial recognition technology.

[1203] Step 5: Receiving the Question

[1204] The user inputs a question in natural language using the device. The device then sends the question to the server. The input is the text of the question entered by the user, and the output is the question data sent to the server. Specifically, the user submits a question through a chatbot or text input field, and the device forwards it to the server.

[1205] Step 6: Analyzing the Question

[1206] The server receives questions from users and passes them to a generative AI for analysis. The generative AI queries a knowledge base and generates the optimal answer. The input is the question data sent to the server, and the output is the generated answer data. Specifically, the AI ​​analyzes the question, retrieves relevant information from the knowledge base, and constructs the answer.

[1207] Step 7: Feedback on emotional information

[1208] The emotion engine feeds the analyzed emotional state back to the generative AI, which then adjusts the tone and content of the response. The input is the analyzed emotional state information, and the output is the adjusted response data. Specifically, the generative AI takes the emotional information into consideration and adjusts the tone of the response to include phrases such as "providing reassurance" or "including words of encouragement."

[1209] Step 8: Submit and view your response

[1210] The server sends the adjusted response to the user's device. The device displays the received response to the user. The input is the adjusted response data, and the output is the response information that the user receives visually and audibly. Specifically, the device provides the response to the user by reading it aloud using speech synthesis or by displaying it as a text message.

[1211] Through the processing steps described above, the system provides prompt, emotionally sensitive, and appropriate answers to user questions. By combining specific technologies and functions, it enables sales representatives to acquire knowledge and improves the quality of customer service.

[1212] (Application Example 2)

[1213] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1214] Conventional sales support systems can provide accurate answers to user questions, but they do not take into account the user's emotional state. This can lead to insufficient improvement in customer satisfaction and a decline in the quality of the customer experience. Furthermore, it is difficult for new sales representatives to quickly acquire the necessary knowledge. Therefore, there is a need for a system that allows new sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses tailored to the user's emotional state.

[1215] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for storing knowledge data and service information in a database, means for constructing a generative AI that learns the knowledge data and service information, means for analyzing the user's voice data and video data and evaluating their emotional state, and means for adjusting the tone and content of the generated response according to the user's emotions. This enables sales representatives to quickly acquire the necessary knowledge and to provide appropriate responses that take into account the user's emotions.

[1216] A "database" is an information system for storing knowledge data and service information, providing the foundational data that AI will later learn from.

[1217] "Knowledge data" refers to data that systematically compiles specific knowledge and information, such as technical documents and FAQ data within a company.

[1218] "Service information" refers to detailed information about the products and services offered, which is necessary to generate appropriate answers to user inquiries.

[1219] "Generative AI" refers to artificial intelligence that generates appropriate answers to user questions based on training data.

[1220] A "natural language processing model" is a machine learning model that understands and analyzes natural language text, such as questions from users, to generate appropriate answers.

[1221] "User voice data" refers to voice information provided by the user through a microphone, and is used to analyze the user's emotional state.

[1222] "Video data" refers to image information captured in real time via a camera, such as the user's facial expressions, and is used for emotion analysis.

[1223] "Emotional state" refers to the feelings and moods a user experiences at a particular moment, and is recognized by analyzing audio and video data.

[1224] "Response tone" refers to the tone and nuances of expression in the generated response, which are adjusted according to the user's emotional state.

[1225] A "terminal" is a device used by a user to input questions and receive answers, and includes smartphones and smart glasses.

[1226] This invention relates to a sales representative support system that combines a generative AI and an emotion engine, and will be described with a particular focus on its application in physical stores. Specific embodiments for carrying out this invention are described below.

[1227] Database and Server Configuration

[1228] The server periodically collects knowledge data such as technical documents and FAQs within the company, as well as detailed information about the products and services offered, and stores it in a database. This database provides the foundational data for generative AI to learn from.

[1229] Building Generative AI

[1230] The server provides the generative AI with knowledge data and service information stored in the database. The generative AI learns from this data and builds a knowledge base to generate appropriate answers to various user questions. It is important to recognize patterns for responding to questions using natural language processing techniques.

[1231] Introducing an emotional engine

[1232] The server is equipped with an emotion engine that analyzes voice and facial expression data from the user to recognize the user's emotional state. Specifically, when the user inputs a question through the terminal, the system uses voice and camera data to analyze their emotions in real time. The EmotionRecognizer library is used for emotion analysis.

[1233] Question reception and analysis

[1234] The device (e.g., a smartphone or smart glasses) has an interface that receives questions from the user. The user inputs the question in natural language, and the device sends the question to the server. It also collects the user's voice and camera footage in real time and sends it to the emotion engine.

[1235] Response generation and sentiment adjustment

[1236] The server receives questions from users and passes them to a generative AI for analysis. In parallel, the emotion engine analyzes audio and video data from the user and evaluates the user's emotional state. The obtained emotional information is fed back to the generative AI, and the tone and content of the response are adjusted according to the user's emotions.

[1237] Displayed as "Response submitted".

[1238] The generated response is sent from the server to the user's device. The response includes a tone and content that takes the user's emotions into consideration. The device receives the transmitted response and displays it to the user visually and audibly. This allows the user to receive an appropriate response that matches their emotions.

[1239] Specific example

[1240] For example, if a user asks "What is the warranty period for this product?" in a physical store, the smart glasses' camera might capture the user's facial expression and recognize that they are angry. Based on this information, a generative AI might generate a response such as "Excuse me, are you dissatisfied? The warranty period for this product is two years," and display it on the device in a tone appropriate to the user's emotions.

[1241] Example of a prompt

[1242] For example, here are some examples of prompt statements to input into a generative AI model:

[1243] Q: What is the warranty period for this product?

[1244] A:

[1245] Thus, the present invention provides a system that enables sales representatives to quickly and accurately acquire necessary knowledge, while also considering the user's feelings and providing appropriate responses. This can significantly improve the quality of customer service.

[1246] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1247] Step 1:

[1248] The user enters a question through a device (smartphone or smart glasses). The user enters the question in natural language and has it read aloud. At this time, the device's microphone captures the audio data.

[1249] Input: User's question (text format), user's voice data

[1250] Output: Question (text data), audio data

[1251] Step 2:

[1252] The device uses a speech recognition library to convert the captured audio data into text. The conversion from audio data to text data takes place.

[1253] Input: Audio data

[1254] Output: Question (text data)

[1255] Step 3:

[1256] The device captures the user's facial expressions with its camera and generates video data. This prepares it for real-time analysis of the user's emotional state.

[1257] Input: User's facial expression (video data)

[1258] Output: Video data

[1259] Step 4:

[1260] The device sends text data and video data of the question to the server. The server receives this data and passes the question data to the generative AI and the video data to the emotion engine.

[1261] Input: Question (text data), video data

[1262] Output: Sending question data and video data to the server.

[1263] Step 5:

[1264] The server inputs question data into a generative AI model and generates an appropriate answer. The generative AI generates the answer based on knowledge data and service information. This answer is provided in text format.

[1265] Input: Question (text data)

[1266] Output: Generated response (text data)

[1267] Step 6:

[1268] The server analyzes the video data using an emotion engine to recognize the user's emotional state. The emotion engine analyzes the video data and evaluates the user's emotions (e.g., anger, joy, dissatisfaction, etc.).

[1269] Input: Video data

[1270] Output: User's emotional state

[1271] Step 7:

[1272] The server adjusts the tone of the generated response based on emotional state information fed back from the emotion engine. The tone and content are adjusted according to the emotion.

[1273] Input: Generated response (text data), user's emotional state

[1274] Output: Emotionally balanced and adjusted responses (text data)

[1275] Step 8:

[1276] The server sends the adjusted response to the terminal. The terminal displays the received adjusted response to the user visually and audibly. The user can receive the appropriate response in an adjusted tone.

[1277] Input: Adjusted response (text data)

[1278] Output: Adjusted response displayed to the user

[1279] Specific actions:

[1280] For example, if a user asks, "What is the warranty period for this product?" and the device's camera captures an angry expression, the server uses generative AI to generate the answer, "The warranty period for this product is two years." After the emotion engine evaluates the anger, it adjusts the tone of the response to, "I'm sorry, are you dissatisfied? The warranty period for this product is two years."

[1281] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1282] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1283] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1284] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1285] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1286] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1287] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1288] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1289] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1290] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1291] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1292] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1293] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1294] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1295] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1296] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1297] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1298] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1299] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1300] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1301] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1302] The following is further disclosed regarding the embodiments described above.

[1303] (Claim 1)

[1304] A means for storing knowledge data and service information in a database,

[1305] A means for constructing a generative AI that learns knowledge data and service information,

[1306] A means of receiving questions from users,

[1307] A means of analyzing the received question and generating an answer using a generative AI,

[1308] A means of sending the generated response to the user,

[1309] A means of displaying submitted responses to the user,

[1310] A system that includes this.

[1311] (Claim 2)

[1312] The system according to claim 1, which includes internal technical documents and FAQ data as knowledge data.

[1313] (Claim 3)

[1314] The system according to claim 1, which uses a natural language processing model as a generative AI.

[1315]

[1316] "Example 1"

[1317] (Claim 1)

[1318] A means for storing knowledge data and product information in a database,

[1319] A means for constructing a generative AI model that learns knowledge data and product information,

[1320] A means of receiving user questions in natural language,

[1321] A means of analyzing the received questions and generating answers using a generative AI model,

[1322] A means of sending the generated response to the user's device,

[1323] A means of visually displaying the submitted response to the user,

[1324] A system that includes this.

[1325] (Claim 2)

[1326] The system according to claim 1, which includes internal technical documents and FAQ data as knowledge data.

[1327] (Claim 3)

[1328] The system according to claim 1, which uses a natural language processing model as a generative AI model.

[1329] "Application Example 1"

[1330] (Claim 1)

[1331] A means for storing knowledge data and service information in a database,

[1332] A means for constructing a generative AI that learns knowledge data and service information,

[1333] A means of receiving questions from users,

[1334] A means of analyzing the received question and generating an answer using a generative AI,

[1335] A means of sending the generated response to the user,

[1336] A means of displaying submitted responses to the user,

[1337] A means for receiving questions and displaying answers to robots operating in a factory,

[1338] A system that includes this.

[1339] (Claim 2)

[1340] The system according to claim 1, which includes technical documents and FAQ data as knowledge data.

[1341] (Claim 3)

[1342] The system according to claim 1, which uses a natural language processing model as a generative AI.

[1343] "Example 2 of combining an emotion engine"

[1344] (Claim 1)

[1345] A means for storing knowledge data and service information in a database,

[1346] A means for constructing a generative AI that learns knowledge data and service information,

[1347] A means of receiving questions from users,

[1348] A means of analyzing the received question and generating an answer using a generative AI,

[1349] A means of adjusting the generated response based on the user's emotions,

[1350] A means of sending the adjusted response to the user,

[1351] A means of displaying submitted responses to the user,

[1352] A system that includes this.

[1353] (Claim 2)

[1354] The system according to claim 1, which includes, as knowledge data, internal technical documents and FAQ data within a company.

[1355] (Claim 3)

[1356] The system according to claim 1, which uses a natural language processing model as a generative AI.

[1357] (Claim 4)

[1358] The system according to claim 1, comprising an emotion engine that analyzes the user's voice data and video data to recognize their emotional state.

[1359] (Claim 5)

[1360] The system according to claim 4, wherein a generative AI adjusts the tone and content of the response based on the user's emotional information.

[1361] "Application example 2 when combining with an emotional engine"

[1362] (Claim 1)

[1363] A means for storing knowledge data and service information in a database,

[1364] A means for constructing a generative AI that learns knowledge data and service information,

[1365] A means of receiving questions from users,

[1366] A means of analyzing the received question and generating an answer using a generative AI,

[1367] A means of sending the generated response to the user,

[1368] A means of displaying submitted responses to the user,

[1369] A means for analyzing user voice and video data to evaluate their emotional state,

[1370] A means to adjust the tone and content of the generated responses according to the user's emotions,

[1371] A system that includes this.

[1372] (Claim 2)

[1373] The system according to claim 1, which includes internal technical documents and FAQ data as knowledge data.

[1374] (Claim 3)

[1375] The system according to claim 1, which uses a natural language processing model as a generative AI. [Explanation of Symbols]

[1376] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for storing knowledge data and service information in a database, A means for constructing a generative AI that learns knowledge data and service information, A means of receiving questions from users, A means of analyzing the received question and generating an answer using a generative AI, A means of sending the generated response to the user, A means of displaying submitted responses to the user, A system that includes this.

2. The system according to claim 1, which includes internal technical documents and FAQ data as knowledge data.

3. The system according to claim 1, which uses a natural language processing model as a generative AI.

Images