system

Abstract

Provide a system. 【Solution means】 Means for inputting a question in a user terminal, In the server, means for receiving and analyzing the user's question, In the server, means for searching a database based on the question analysis result, In the server, means for generating an answer based on the search result, In the server, means for transmitting the generated answer to the user terminal, In the user terminal, means for displaying the transmitted answer, In the server, means for recording all questions and answers, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a conventional new employee training system, direct support from senior employees to new employees is required, which requires a great deal of man-hours and there are problems such as variations in the quality of the senior employees' support and the time taken for the support. For this reason, the quality and effect of new employee training are not unified, and new employees may not be able to quickly obtain the necessary information. Also, the same questions occur frequently, and it is inefficient to repeat the same response each time. In order to solve these problems, there is a need to provide an automatic response system using generative AI.

Means for Solving the Problems

[0005] The present invention solves the above problems by the following means: providing a system that includes means for inputting a question on a user terminal, means for receiving and analyzing the user's question on a server, means for searching a database on the server based on the question analysis results, means for generating an answer on the server based on the search results, means for transmitting the generated answer on the server to the user terminal, means for displaying the transmitted answer on the user terminal, and means for recording all questions and answers on the server. This enables the rapid provision of high-quality, consistent answers, thereby improving the efficiency and uniformity of new employee training. Furthermore, even when the same question is repeated, a quick response is possible by referring to past answers.

[0006] A "user terminal" is a device used by a user to access a system via an interface and input questions.

[0007] A "server" is a central computing system that performs tasks such as receiving, analyzing, searching, and generating answers for question data.

[0008] "Means for entering questions" refers to an interface on the user's terminal where the user enters questions in natural language.

[0009] "Means for receiving and analyzing questions" refers to the function by which the server analyzes questions received from user terminals using natural language processing technology.

[0010] "Means of searching the database" refers to the function by which the server searches the database for relevant information based on the results of query analysis.

[0011] "Means for generating answers" refers to the function by which a server generates answers based on search results, or from information that can be inferred if no search results are found.

[0012] "Means of sending responses" refers to the function by which the server sends the generated response to the user's terminal.

[0013] "Means for displaying responses" refers to an interface on the user's terminal that visualizes the responses sent from the server to the user.

[0014] "Means for recording all questions and answers" refers to a function where the server stores all user questions and their answers in a database, making them available for later reference. [Brief explanation of the drawing]

[0015] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0016] Hereinafter, an example of an embodiment of the system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0017] First, the language used in the following description will be explained.

[0018] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0019] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0020] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, etc.

[0021] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0023] [First Embodiment]

[0024] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0025] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0028] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0031] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0035] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0036] The system according to the present invention utilizes generative AI to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[0037] System Configuration

[0038] 1. User terminal

[0039] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0040] 2. Server

[0041] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, and generating answers. The server is equipped with a generative AI with natural language processing (NLP) capabilities, an FAQ database, and a database that records past questions and answers.

[0042] Flow of operations

[0043] 1. Enter and submit your question.

[0044] The user logs into their device and enters their question in natural language. For example, they might enter the question, "How do I start a new project?"

[0045] The terminal sends the entered question data to the server.

[0046] 2. Receiving and analyzing questions

[0047] The server receives the question data and analyzes the question content using generative AI. Natural language processing techniques are used for the analysis to extract key keywords from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[0048] 3. Database Search

[0049] Based on the analysis results, the server searches the FAQ database and the past question and answer database. For example, if "Steps to start a new project" is already registered in the FAQ database, that information will be retrieved.

[0050] 4. Generating the answer

[0051] The server generates an appropriate answer based on the search results. If a search result is found, it constructs an answer based on it; if no result is found, the AI ​​generates a predicted answer.

[0052] For example, it can generate a response like, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[0053] 5. Submitting and displaying responses

[0054] The server sends the generated response to the user's terminal.

[0055] The device displays the received responses to the user. The user can review the displayed responses and ask additional questions if necessary.

[0056] 6. Record of questions and answers

[0057] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0058] Specific example

[0059] For example, if new employee Tanaka enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[0060] 1. Enter and submit your question.

[0061] Mr. Tanaka (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[0062] 2. Reception and Analysis

[0063] The server receives the question and extracts the keywords "working hours" and "recording method."

[0064] 3. Database Search

[0065] The server searches the FAQ database for information on "how to record working hours."

[0066] 4. Answer generation

[0067] The system generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[0068] 5. Submit your response and display

[0069] The server generates an answer and sends it to Tanaka's terminal, which then displays the answer.

[0070] 6. Record

[0071] The server records this question and the generated answer in the database.

[0072] In this way, we can provide prompt and consistent answers to questions from new employees and improve the quality of their training.

[0073] The following describes the processing flow.

[0074] Step 1:

[0075] The user logs into their device and accesses the question input screen. The user enters the question in natural language and clicks the submit button.

[0076] Step 2:

[0077] The terminal sends the entered question data to the server. Specifically, the string entered by the user is sent to the server as a packet.

[0078] Step 3:

[0079] The server receives the question data. The server parses the received data and reads the string data.

[0080] Step 4:

[0081] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[0082] Step 5:

[0083] The server searches the database based on the analysis results. It refers to the FAQ database and past question and answer database to find relevant information.

[0084] Step 6:

[0085] The server evaluates the search results and generates an appropriate answer. If a search result is found, it creates an answer based on that data. If no search results are found, the AI ​​generates a predicted answer.

[0086] Step 7:

[0087] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[0088] Step 8:

[0089] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary.

[0090] Step 9:

[0091] The server records all questions and answers in a database. This ensures that if the same question arises again in the future, a quick response can be provided.

[0092] Through the processing steps described above, this system reduces the burden on senior employees and enables new graduates to quickly and consistently obtain the information they need.

[0093] (Example 1)

[0094] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0095] In the previous system, there was a problem in that it took a long time to resolve questions that new graduates had about daily tasks, attendance tracking, and how to use the company's internal systems. Furthermore, when specialized knowledge was required, the burden on the person in charge increased, which was another issue. This resulted in decreased work efficiency for new graduates and inconsistent quality of training.

[0096] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0097] In this invention, the server includes means for receiving user questions and analyzing them using natural language processing, means for searching a knowledge base based on the question analysis results, and means for generating estimated answers using a generative AI model. This makes it possible to automatically provide quick and appropriate answers to user-inputted questions, improve the work efficiency of new employees, and maintain a consistent quality of training.

[0098] A "user terminal" is a device used by a user to access the system and input questions.

[0099] A "server" is a central computing resource that receives, analyzes, searches databases, and generates answers for questions.

[0100] "Natural language processing" is a technology that analyzes data entered by users in natural language and extracts necessary keywords and information.

[0101] A "knowledge base" is a general term for databases that record FAQs and past questions and answers, and is a source of information that a system uses to generate answers to questions.

[0102] A "generative AI model" is an artificial intelligence model that generates appropriate answers based on received questions and search results, and examples include GPT-3 (registered trademark) and GPT-4 (registered trademark).

[0103] An "estimated answer" is an answer that a generative AI model generates based on inference when no search results are found.

[0104] "Keywords" are important words or phrases extracted from the user's question using natural language processing.

[0105] "Knowledge-based search" is the process of searching for relevant information by referring to FAQ databases and past question-and-answer databases based on extracted keywords.

[0106] "Means of recording" refers to the function of saving all questions and generated answers in a database for future use.

[0107] The system according to the present invention efficiently provides support for new graduates' work, attendance tracking, instruction on how to use internal systems, and answers questions related to their work. An embodiment of this system is described below.

[0108] User terminal

[0109] User terminals are devices used by new employees to access the system and input questions in natural language. Specifically, they include personal computers, smartphones, and tablets. Users use these terminals to input questions in natural language, such as "Please tell me the procedure for starting a new project."

[0110] server

[0111] Servers are the central computing resources of a system and perform the following main functions:

[0112] 1. Receiving and analyzing questions

[0113] The server receives question data submitted by the user. The received data is then analyzed using natural language processing with Python libraries such as SpaCy and NLTK. As a result of the analysis, key keywords are extracted from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[0114] 2. Knowledge base search

[0115] Based on the analysis results, the server searches the FAQ database and databases containing past questions and answers. SQL queries and NoSQL database queries are used for this search process. For example, a query like "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Starting Procedure%'" is executed.

[0116] 3. Generating answers using generative AI models

[0117] The server generates appropriate answers based on information retrieved from the database. If search results are found, the AI ​​creates an answer based on their content. If no search results are found, a generative AI model (such as GPT-3 or GPT-4) generates an appropriate answer based on inferences. Specifically, in response to the question, "Please tell me the steps to start a new project," the server generates the answer, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[0118] 4. Submitting and viewing the generated responses

[0119] The server sends the generated response to the user's terminal. A REST API is used for transmission, and the response data is transferred in JSON format. The user's terminal parses and displays the received response. The user can review the displayed response and ask additional questions if necessary.

[0120] 5. Record of questions and answers

[0121] The server records all questions and generated answers in a database. For example, using MongoDB or MySQL®, data is stored in the format "Question: 'How do I record my working hours?', Answer: 'Working hours are recorded using the following procedure...'". This allows for quick response when similar questions arise in the future.

[0122] Specific example

[0123] When a new employee types "Please tell me how to record my working hours" into a terminal and sends it to the server, it will look like this:

[0124] 1. Enter and submit your question.

[0125] The user enters "Please tell me how to record my working hours" into the terminal.

[0126] 2. Reception and Analysis

[0127] The server receives the question and uses natural language processing (NLP) techniques to extract keywords such as "working hours" and "recording method."

[0128] 3. Knowledge base search

[0129] The server searches the FAQ database for information on "how to record working hours."

[0130] 4. Answer generation

[0131] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[0132] 5. Submit your response and display

[0133] The server generates an answer and sends it to the user's terminal, which then displays the answer.

[0134] 6. Record

[0135] The server records this question and the generated answer in the database.

[0136] This system allows us to provide quick and consistent answers to questions from new employees, improving work efficiency and maintaining a consistent quality of training.

[0137] Example of a prompt

[0138] "Please tell me how to start a new project."

[0139] "Please tell me how to record working hours."

[0140] "I have a question about how to use the company's internal systems."

[0141] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0142] System processing flow

[0143] Step 1:

[0144] The user logs into the device and enters the question.

[0145] Input: The user enters "Please tell me how to start a new project."

[0146] Operation: The terminal retrieves the entered question data.

[0147] Step 2:

[0148] The terminal sends the entered question data to the server.

[0149] Input: Question data "Please tell me how to start a new project."

[0150] Operation: The terminal securely sends the question data to the server using the HTTPS protocol.

[0151] Output: Send the question data to the server.

[0152] Step 3:

[0153] The server receives the question data and begins analysis using natural language processing (NLP).

[0154] Input: Question data sent from the device

[0155] Operation: The server analyzes the question content using Python libraries (e.g., SpaCy or NLTK). It extracts key keywords.

[0156] Example: Identify keywords such as "new project" and "starting procedures".

[0157] Output: Extracted keywords

[0158] Step 4:

[0159] The server searches the knowledge base based on the analysis results.

[0160] Input: Extracted keywords "New project" and "Starting procedure"

[0161] Operation: The server uses SQL queries and NoSQL databases to search the FAQ database and past question and answer databases.

[0162] For example, in the case of an SQL database, execute the query "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Start Procedure%'".

[0163] Output: Search Results

[0164] Step 5:

[0165] The server generates an appropriate answer based on the search results. If no search results are found, a generative AI model is used to generate an estimated answer.

[0166] Input: Search Results

[0167] Operation: The server generates an answer based on the search results. If no search results are found, it estimates an answer using a generative AI model (e.g., GPT-3 or GPT-4).

[0168] Example: If there are search results, it will generate a response such as, "The steps to start a new project are as follows: 1. Fill out the project application form 2. Obtain approval from your supervisor 3. Form a project team 4. Prepare the necessary materials."

[0169] Output: Generated answer

[0170] Step 6:

[0171] The server sends the generated response to the user's terminal.

[0172] Input: Generated answer

[0173] Operation: The server sends response data in JSON format using a REST API.

[0174] Output: Send the response data to the user's terminal.

[0175] Step 7:

[0176] The device displays the received response to the user.

[0177] Input: Response data sent from the server

[0178] Operation: The device parses the JSON data and displays it on the screen. The user can then review the displayed response.

[0179] Output: Answer displayed to the user

[0180] Step 8:

[0181] The server records all questions and generated answers in a database.

[0182] Input: Question data and generated response data

[0183] Operation: The server records the questions and answers in a database (e.g., MongoDB or MySQL).

[0184] For example, record it in the format: "Question: 'Please tell me how to start a new project,' Answer: 'The steps to start a new project are as follows...'"

[0185] Output: Recorded data

[0186] (Application Example 1)

[0187] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0188] In a factory environment, a sophisticated training support system is necessary for new employees to receive appropriate guidance and quickly begin their work. However, the current system makes it difficult for new employees to get quick answers to their questions about operation methods and work procedures. Furthermore, there are concerns that the quality of training will decline because consistent answers to questions are not provided.

[0189] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0190] In this invention, the server includes means for receiving and analyzing user questions, means for searching a database based on the question analysis results, and means for generating answers based on the search results. This makes it possible to provide rapid and consistent answers using a generative AI model for new employees to answer questions from factory automation machines in a factory environment.

[0191] A "user terminal" is a device used to input questions and communicate with a server. Examples include personal computers, smartphones, and tablets.

[0192] A "server" is a central processing unit that analyzes questions received from users and generates appropriate answers. It performs tasks such as analyzing question data, searching databases, generating answers, and recording them.

[0193] A "generative AI model" is an artificial intelligence technology used to analyze a question and generate an appropriate answer. For example, natural language processing technologies such as OpenAI's GPT-3 fall into this category.

[0194] "Natural language processing" is a technology that understands questions entered by users in natural language and extracts appropriate keywords.

[0195] A "database" is an information storage system that stores past questions and answers, as well as FAQ information. The server searches this database to find answers to questions.

[0196] "Factory automation machinery" refers to machines that perform tasks in a factory automatically. Examples include robotic arms and automated transport systems.

[0197] "Means for entering questions" refers to an interface that allows users to enter questions in natural language and send them to the server.

[0198] The system according to the present invention provides a question-answering system using a generative AI model to efficiently support the training of new employees in a factory environment. This system includes a user terminal, a server, and a database.

[0199] User terminal

[0200] User terminals are devices used by new employees to input questions to factory automation machinery. Specifically, these include personal computers, tablets, and smartphones. Users input questions in natural language using these terminals, and these questions are sent to the server.

[0201] server

[0202] The server is the central computer of this system. The server receives questions submitted by users and analyzes them using a generative AI model (e.g., OpenAI GPT-3). Natural language processing techniques (e.g., spaCy) are used for the analysis to extract key keywords.

[0203] The server searches the database based on the analysis results. The database contains past questions and answers, as well as FAQ information. If search results exist, it generates an answer based on that information. Even if no search results are found, the AI ​​model automatically generates an estimated answer.

[0204] The generated responses are sent from the server to the user's terminal. The user's terminal displays these responses to the user, providing consistent educational support.

[0205] database

[0206] A database is a system that stores information for the server to perform searches. This includes past questions and answers, and FAQ information. The server uses this database to provide the most appropriate answer to the user's question.

[0207] Specific example

[0208] For example, if a new employee asks a factory automation machine, "How do I maintain this machine?", the process would proceed as follows: The question is entered into the user terminal and sent to the server. The server receives the question and extracts the keywords "machine" and "maintenance method". It searches the database to find information about maintenance procedures and generates an answer, such as "The maintenance method for this machine is as follows: 1. Turn off the power. 2. Open the panel. 3. Clean the filter." The generated answer is sent to the user terminal and displayed.

[0209] Example of a prompt

[0210] The prompt message when a new employee enters the question, "Please tell me how to operate this machine," is as follows:

[0211] New employee: Please explain how to operate this machine.

[0212] Robot AI:

[0213] In this way, the system of the present invention can respond quickly and accurately to questions from new employees and improve the quality of training.

[0214] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0215] Step 1:

[0216] The user enters a question in natural language on their device. The entered question is a specific text, such as "Please tell me how to maintain this machine." The user's device then sends this text data to the server.

[0217] Input: Question text in natural language

[0218] Output: Question data sent to the server

[0219] Step 2:

[0220] The server receives question data sent from the user's terminal. The server converts this data into a format for input into a generative AI model (OpenAI GPT-3). Specifically, it formats the question text into a prompt.

[0221] Input: Question data from the user terminal

[0222] Output: Prompt text input to the generative AI model

[0223] Step 3:

[0224] The server uses a generative AI model to analyze the content of the received question. In this process, natural language processing technology (such as spaCy) is used to extract key keywords from the question. For example, from the question "Please tell me how to maintain this machine," keywords such as "machine" and "maintenance method" are extracted.

[0225] Input: Prompt message

[0226] Output: Extracted keywords

[0227] Step 4:

[0228] The server searches the database based on the extracted keywords. This includes databases of past questions and answers and FAQ information. If relevant information exists, it is retrieved. For example, if detailed instructions regarding "maintenance methods" are stored in the database, that information will be retrieved as a search result.

[0229] Input: Extracted keywords

[0230] Output: Search results data

[0231] Step 5:

[0232] The server generates an appropriate answer based on the search results. If search results exist, it constructs the answer text based on that information. For example, it generates a specific answer such as, "The maintenance procedure for this machine is as follows: 1. Turn off the power 2. Open the panel 3. Clean the filter." If no search results are found, a generative AI model generates an estimated answer.

[0233] Input: Search result data

[0234] Output: Generated answer text

[0235] Step 6:

[0236] The server sends the generated response text to the user's terminal. The user's terminal displays the received response. This allows the new employee to review the displayed response and respond appropriately.

[0237] Input: Generated response text

[0238] Output: The answer displayed on the user's terminal.

[0239] Step 7:

[0240] The server records all questions and generated answers in a database. This allows for a quick response if the same question arises in the future.

[0241] Input: Question data and answer data

[0242] Output: Questions and answers recorded in the database

[0243] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0244] The system according to the present invention utilizes generative AI and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[0245] System Configuration

[0246] 1. User terminal

[0247] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0248] 2. Server

[0249] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching databases, sentiment analysis, and answer generation. The server is equipped with a generative AI with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database that records past questions and answers.

[0250] Flow of operations

[0251] 1. Enter and submit your question.

[0252] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[0253] 2. Receiving the questionnaire data

[0254] The terminal sends the entered question data to the server. The question entered by the user is transferred to the server.

[0255] 3. Analysis of Questionnaire Data

[0256] The server receives the question data and uses generative AI to analyze the question content. It extracts key keywords and phrases from the question and understands the intent.

[0257] 4. Emotion analysis

[0258] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0259] 5. Database Search

[0260] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[0261] 6. Generating the answer

[0262] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[0263] 7. Submitting and displaying responses

[0264] The server sends the generated response data to the user's terminal.

[0265] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[0266] 8. Record of questions and answers

[0267] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0268] Specific example

[0269] For example, if new employee Sato enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[0270] 1. Enter and submit your question.

[0271] Mr. Sato (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[0272] 2. Reception and Analysis

[0273] The server receives the question and extracts the keywords "working hours" and "recording method."

[0274] 3. Emotion analysis

[0275] The server uses an emotion engine to detect the "confusion" contained in Sato's question.

[0276] 4. Database Search

[0277] The server searches the FAQ database for information on "how to record working hours."

[0278] 5. Answer generation

[0279] Generate an answer that says "Working hours are recorded in the following procedure: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter the clock-in and clock-out times. 4. Click the save button." and add a supplementary explanation in a friendly tone because "confusion" is detected.

[0280] 6. Answer transmission and display

[0281] The server sends the answer generated to Mr. Sato's terminal, and the terminal displays the answer.

[0282] 7. Recording

[0283] The server records this question and the generated answer in the database.

[0284] In this way, by detecting the situations that new employees are troubled with and providing appropriate answers accordingly, the quality of education can be improved. (

[0285] The following explains the processing flow.

[0286] Step 1:

[0287] The user logs in to the terminal and accesses the question input screen. The user inputs a question in natural language and clicks the send button.

[0288] Step 2:

[0289] The terminal sends the input question data to the server. Specifically, the character string input by the user is sent to the server as a packet.

[0290] Step 3:

[0291] The server receives the question data. The server analyzes the received data and reads the character string data.

[0292] Step 4:

[0293] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[0294] Step 5:

[0295] The server uses an emotion engine to analyze the emotions contained in the user's question. The analysis employs techniques to classify emotions from words and phrases contained in the text. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0296] Step 6:

[0297] The server searches the database based on keyword analysis results and sentiment analysis results. It searches the FAQ database and past question and answer database to find relevant information.

[0298] Step 7:

[0299] The server evaluates the search results and generates an appropriate response. If a search result is found, it creates a response based on that data. Even if no search results are found, the AI ​​generates a predicted response. Furthermore, it adjusts the response to have an appropriate tone and content based on sentiment analysis results.

[0300] Step 8:

[0301] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[0302] Step 9:

[0303] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary. For example, to alleviate any confusion the user might feel, detailed instructions in a gentle tone are displayed.

[0304] Step 10:

[0305] The server records all questions and answers in the database, thus preparing for the ability to respond quickly when the same question arises in the future.

[0306] Through the above processing steps, this system reduces the burden on elderly employees and enables new employees to quickly and consistently obtain the necessary information with a high level of emotional understanding.

[0307] (Example 2)

[0308] Next, Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart device 14 is referred to as the "terminal".

[0309] In the modern corporate environment, it is important for new employees to quickly and efficiently adapt to their work. However, conventional FAQ systems and manuals are limited to providing static information and it is difficult to respond to specific questions and emotions of users. As a result, new employees may not receive appropriate guidance and support, and it may take time to adapt to their work. In addition, since appropriate answers to questions are not provided quickly, work efficiency may decrease.

[0310] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following respective means.

[0311] In this invention, the server includes means for performing question analysis using a generative artificial intelligence model, means for performing emotional analysis of questions, and means for generating answers based on search results and emotional analysis results. Thereby, it becomes possible to quickly provide appropriate answers to specific questions and emotions of users.

[0312] The "user terminal" is a device for a user to access the system and input questions, and includes a personal computer, a smartphone, a tablet, etc.

[0313] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[0314] A "generative artificial intelligence model" refers to an artificial intelligence technology that uses natural language processing (NLP) to generate answers to questions based on the generated data.

[0315] "Question analysis" refers to the process of analyzing a user's question, extracting key keywords and phrases, and understanding their intent.

[0316] "Sentiment analysis" refers to the process of detecting the emotions contained in a user's question (e.g., anxiety, confusion, excitement) and adjusting the tone of the response based on that.

[0317] A "database" refers to a storage device that stores information that a system can refer to, such as FAQ data and past question-and-answer data.

[0318] "Answer generation" refers to the process of constructing and generating appropriate answers based on search results and sentiment analysis results.

[0319] "Recording" refers to the process of saving all questions and their answers in a database so that they can be referenced later.

[0320] The system according to the present invention utilizes a generative artificial intelligence model and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new employees. This system consists of a user terminal, a server, a generative artificial intelligence model, an emotion engine, and a database.

[0321] System Configuration

[0322] 1. User terminal

[0323] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0324] 2. Server

[0325] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, sentiment analysis, and answer generation. The server is equipped with a generative artificial intelligence model with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database recording past questions and answers. Specifically, OpenAI's GPT-4 is used as the generative AI model. IBM Watson's Tone Analyzer is used as the sentiment engine.

[0326] Flow of operations

[0327] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[0328] The terminal sends the entered question data to the server.

[0329] The server receives the question data and analyzes the question content using a generative artificial intelligence model. It extracts key keywords and phrases from the question and understands the intent.

[0330] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0331] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[0332] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[0333] The server sends the generated response data to the user's terminal.

[0334] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[0335] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0336] Specific example

[0337] For example, if a new employee enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[0338] The user types "Please tell me how to record my working hours" into the terminal and clicks the send button.

[0339] The server receives the question and extracts the keywords "working hours" and "recording method."

[0340] The server uses an emotion engine to detect "confusion" contained in the question.

[0341] The server searches the FAQ database for information on "how to record working hours."

[0342] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system 2. Access the attendance management page 3. Enter your start and end times 4. Click the save button," and, sensing "confusion," adds supplementary explanations in a gentle tone.

[0343] The server generates an answer and sends it to the user's device, which then displays the answer on its screen.

[0344] The server records this question and the generated answer in the database.

[0345] In this way, the quality of training can be improved by sensing when new employees are having trouble and providing appropriate answers accordingly. Examples of specific prompts include: "Please tell me how to start a new project," "Please tell me how to connect to the company Wi-Fi," and "Please tell me how to apply for paid leave."

[0346] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0347] Step 1:

[0348] The user accesses the login screen on their device, enters their ID and password, and logs into the system. After successfully logging in, they enter their specific question or inquiry in natural language in the question input field. For example, they might enter, "Please tell me how to start a new project." Then, the user clicks the "Submit" button, and the question data is sent.

[0349] Input: User ID, password, and security question.

[0350] Output: Login authentication result, submitted question data

[0351] Step 2:

[0352] The terminal sends the user-entered question data to the server. Specifically, the question data is sent to the server using a communication protocol such as an HTTP request. The entered data is then converted to JSON format.

[0353] Input: User's question data

[0354] Output: Question data in JSON format sent to the server

[0355] Step 3:

[0356] The server receives the question data and analyzes the question content using a generative artificial intelligence model. The server analyzes the received JSON data and uses a generative AI model (e.g., OpenAI's GPT-4) to extract the main keywords and phrases of the question. For example, from the question "Please tell me the procedure for starting a new project," the keywords "project," "start," and "procedure" are extracted.

[0357] Input: Question data in JSON format

[0358] Output: Extracted keywords and analysis results

[0359] Step 4:

[0360] The server uses an emotion engine (e.g., IBM Watson's Tone Analyzer) to analyze the emotions contained in the user's questions. Specifically, it detects emotions such as "anxiety," "confusion," and "excitement" based on the received question data. In this step, an emotion score is generated, and the user's emotional state is evaluated.

[0361] Input: Question data

[0362] Output: Sentiment analysis results (emotion score and emotional state)

[0363] Step 5:

[0364] The server searches the FAQ database and past question-and-answer database based on keyword and sentiment analysis results. It uses the extracted keywords (e.g., "project start procedure") as search queries to find relevant information from the database. The search is performed using appropriate SQL queries and search algorithms.

[0365] Input: Keyword analysis results, sentiment analysis results

[0366] Output: Search results (related FAQs and past questions / answers)

[0367] Step 6:

[0368] The server evaluates the search results and generates an appropriate response. It uses a generative artificial intelligence model to generate responses based on the search results. For example, if detailed instructions on "how to start a new project" are found in the search results, the response will be constructed based on that information. If no results are found, the AI ​​will generate a speculative response. The response is then adjusted to an appropriate tone, taking sentiment analysis results into consideration.

[0369] Input: Search results, sentiment analysis results

[0370] Output: Generated response data

[0371] Step 7:

[0372] The server converts the generated response data into JSON format and sends it to the user's terminal. The generated response data is then transferred to the terminal as an HTTP response.

[0373] Input: Generated response data

[0374] Output: JSON format response data sent to the terminal

[0375] Step 8:

[0376] The device analyzes the received response data and displays it to the user. The response is displayed on the device's user interface (UI). The user can review the response on the screen and re-enter the question if necessary.

[0377] Input: Response data in JSON format

[0378] Output: Answer displayed to the user

[0379] Step 9:

[0380] The server records all questions and generated answers in a database. This allows for quick response when the same question arises in the future. The data is recorded along with the question content, the answer content, and their associated metadata.

[0381] Input: Question data, generated response data

[0382] Output: Records stored in the database

[0383] (Application Example 2)

[0384] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0385] In the factory, new graduates are unfamiliar with their work and require efficient support. However, the current system is inadequate in handling questions in natural language and fails to provide appropriate support that takes emotions into account. Furthermore, the lack of real-time speech recognition and response capabilities makes it difficult to provide immediate solutions. This is leading to decreased work efficiency for new graduates and a decline in the quality of their training. In this situation, there is a need for a system that allows new graduates to perform their duties smoothly without feeling confused or anxious.

[0386] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0387] In this invention, the server includes means for receiving and analyzing a user's question, means for searching a database based on the question analysis results, and means for analyzing the emotions contained in the user's question. This allows for the analysis of the user's emotions along with the question content, enabling the generation of an answer in the optimal tone and immediate voice response.

[0388] A "user terminal" is a device used by a user to access a system and input questions. Examples include personal computers, smartphones, and tablets.

[0389] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[0390] "Question analysis" is the process by which a server uses natural language processing to extract key keywords and phrases from a question received from a user and understand the user's intent.

[0391] "Database search" refers to the process where the server searches the FAQ database and past question / answer database based on the question analysis results to find relevant information.

[0392] "Answer generation" refers to the process by which a server evaluates the search results in a database, constructs an appropriate answer, and, in some cases, generates an estimated answer.

[0393] "Sending a response" refers to the process of sending the response data generated by the server to the user's terminal.

[0394] "Displaying responses" refers to the process of displaying the responses received by the user's terminal to the user on the screen.

[0395] "Question and answer recording" refers to the process where the server saves all questions and answers in a database. This makes it possible to provide a quick answer when the same question arises in the future.

[0396] "Emotional analysis" is the process by which a server uses an emotion engine to analyze the emotions contained in a user's question and detect emotions such as "anxiety," "confusion," and "excitement."

[0397] "Speech recognition" is the process by which a robot converts a user's voice into text. This process allows the user to input questions using their voice.

[0398] "Voice output" is the process by which the robot communicates its generated response to the user via voice. Voice output allows the user to receive the response intuitively.

[0399] This invention provides a system that enables new employees to use robots within a factory to provide real-time work support. This system utilizes generative AI and an emotion engine and operates in conjunction with user terminals such as smartphones and tablets.

[0400] System Configuration

[0401] 1. User terminal

[0402] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0403] 2. Robots

[0404] The robot is a device used by new employees within the factory and accepts voice input from users. The robot has the function of converting speech into text using speech recognition technology and outputting the generated response as voice.

[0405] 3. Server

[0406] A server is the central computing resource of a system. A server has the following functions:

[0407] Receiving and analyzing question data

[0408] Implementation of an emotion engine for performing emotion analysis

[0409] Searching Databases

[0410] Response generation and evaluation

[0411] Record of questions and answers

[0412] Flow of operations

[0413] 1. Enter your question

[0414] The user logs into the terminal, enters their question in natural language, and submits it. For example, they might enter, "Please tell me how to start a new project."

[0415] 2. Voice recognition and question submission

[0416] When a user inputs a question into the robot via voice, the robot converts the voice into text and sends it to the server.

[0417] 3. Analysis of Questionnaire Data

[0418] The server receives the question data and analyzes the question content using a generative AI model. It extracts key keywords and phrases from the question and understands the intent.

[0419] 4. Emotion analysis

[0420] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0421] 5. Database Search

[0422] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[0423] 6. Generating the answer

[0424] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs a response based on that data. If no search results are found, it generates an estimated response using a generative AI model. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[0425] 7. Submitting and displaying responses

[0426] The server sends the generated response data to the user's terminal, and the robot outputs the response verbally. The user can also check the response on the terminal's screen.

[0427] 8. Record of questions and answers

[0428] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0429] Hardware and software to use

[0430] Generative AI model: GPT-2 (using the transformers library)

[0431] Emotion Engine: Proprietary emotion analysis engine (sentiment_analysis)

[0432] User devices: PCs, smartphones, tablets

[0433] Robot: A device with voice recognition and voice output capabilities.

[0434] Specific example

[0435] For example, if a new employee types "Please tell me how to start a new project" into the terminal, the system will respond as follows:

[0436] 1. Enter and submit your question.

[0437] The user types "Please tell me how to start a new project" and submits it.

[0438] 2. Reception and Analysis

[0439] The server receives the question, extracts the keywords "project," "start," and "procedure," and understands the intent.

[0440] 3. Emotion analysis

[0441] The server uses an emotion engine to detect "confusion" contained in the user's question.

[0442] 4. Database Search

[0443] The server searches the FAQ database for information on "Project Start Procedures".

[0444] 5. Answer generation

[0445] The system generates a response stating, "The project initiation procedure is as follows: 1. Fill out the application form 2. Obtain approval 3. Select members 4. Set a schedule," and adds supplementary explanations in a gentle tone based on the sentiment analysis results.

[0446] 6. Submit your response and display

[0447] The server generates an answer, which is sent to the user's terminal, and the robot outputs it verbally.

[0448] Example of a prompt

[0449] "Please tell me how to start a new project."

[0450] "What should I do if the machine breaks down?"

[0451] "Please tell me how to record working hours."

[0452] This system provides an environment where new graduates can perform their duties smoothly without feeling confused or anxious.

[0453] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0454] Step 1:

[0455] The user logs into their device and enters their question in natural language.

[0456] Input: The user operates the terminal and enters the question in natural language (e.g., "Please tell me how to start a new project").

[0457] Output: Question text data is generated.

[0458] Step 2:

[0459] The robot receives voice input from the user, converts it to text using speech recognition technology, and sends it to the server.

[0460] Input: Voice input from the user.

[0461] Output: The audio data is converted to text data and sent to the server.

[0462] Step 3:

[0463] The server receives the question data and analyzes the question using a generative AI model.

[0464] Input: Question in text data format.

[0465] Data processing: A generative AI model (GPT-2) analyzes the question content and extracts key keywords and phrases.

[0466] Output: Analyzed keywords and phrases.

[0467] Step 4:

[0468] The server uses an emotion analysis engine to analyze the emotions contained in the user's questions.

[0469] Input: Question text.

[0470] Data processing: The emotion analysis engine analyzes the text and detects emotions such as "anxiety," "confusion," and "excitement."

[0471] Output: Detected emotion information.

[0472] Step 5:

[0473] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[0474] Input: Analyzed keywords and sentiment information.

[0475] Data Search: Searches for server-related information from the FAQ database and past question-and-answer database.

[0476] Output: Search results.

[0477] Step 6:

[0478] The server evaluates the search results and generates appropriate answers.

[0479] Input: Search results from a database.

[0480] Data processing: The server constructs an answer based on the search results and adjusts the answer to an appropriate tone according to the sentiment analysis results.

[0481] Output: Generated answer text.

[0482] Step 7:

[0483] The server generates response data and sends it to the user's terminal, and the robot outputs the response via voice.

[0484] Input: Generated response text.

[0485] Data transfer: The server sends the response data to the user's terminal.

[0486] Data output: The robot outputs the answer in voice.

[0487] Output: A voice response to the user.

[0488] Step 8:

[0489] The server records all questions and answers in a database.

[0490] Input: Question and generated answer data.

[0491] Data storage: The server stores questions and answers in a database.

[0492] Output: Database entries for quick responses to future questions.

[0493] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0494] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0495] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0496] [Second Embodiment]

[0497] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0498] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0499] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0500] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0501] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0502] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0503] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0504] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0505] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0506] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0507] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0508] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0509] The system according to the present invention utilizes generative AI to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[0510] System Configuration

[0511] 1. User terminal

[0512] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0513] 2. Server

[0514] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, and generating answers. The server is equipped with a generative AI with natural language processing (NLP) capabilities, an FAQ database, and a database that records past questions and answers.

[0515] Flow of operations

[0516] 1. Enter and submit your question.

[0517] The user logs into their device and enters their question in natural language. For example, they might enter the question, "How do I start a new project?"

[0518] The terminal sends the entered question data to the server.

[0519] 2. Receiving and analyzing questions

[0520] The server receives the question data and analyzes the question content using generative AI. Natural language processing techniques are used for the analysis to extract key keywords from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[0521] 3. Database Search

[0522] Based on the analysis results, the server searches the FAQ database and the past question and answer database. For example, if "Steps to start a new project" is already registered in the FAQ database, that information will be retrieved.

[0523] 4. Generating the answer

[0524] The server generates an appropriate answer based on the search results. If a search result is found, it constructs an answer based on it; if no result is found, the AI ​​generates a predicted answer.

[0525] For example, it can generate a response like, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[0526] 5. Submitting and displaying responses

[0527] The server sends the generated response to the user's terminal.

[0528] The device displays the received responses to the user. The user can review the displayed responses and ask additional questions if necessary.

[0529] 6. Record of questions and answers

[0530] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0531] Specific example

[0532] For example, if new employee Tanaka enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[0533] 1. Enter and submit your question.

[0534] Mr. Tanaka (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[0535] 2. Reception and Analysis

[0536] The server receives the question and extracts the keywords "working hours" and "recording method."

[0537] 3. Database Search

[0538] The server searches the FAQ database for information on "how to record working hours."

[0539] 4. Answer generation

[0540] The system generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[0541] 5. Submit your response and display

[0542] The server generates an answer and sends it to Tanaka's terminal, which then displays the answer.

[0543] 6. Record

[0544] The server records this question and the generated answer in the database.

[0545] In this way, we can provide prompt and consistent answers to questions from new employees and improve the quality of their training.

[0546] The following describes the processing flow.

[0547] Step 1:

[0548] The user logs into their device and accesses the question input screen. The user enters the question in natural language and clicks the submit button.

[0549] Step 2:

[0550] The terminal sends the entered question data to the server. Specifically, the string entered by the user is sent to the server as a packet.

[0551] Step 3:

[0552] The server receives the question data. The server parses the received data and reads the string data.

[0553] Step 4:

[0554] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[0555] Step 5:

[0556] The server searches the database based on the analysis results. It refers to the FAQ database and past question and answer database to find relevant information.

[0557] Step 6:

[0558] The server evaluates the search results and generates an appropriate answer. If a search result is found, it creates an answer based on that data. If no search results are found, the AI ​​generates a predicted answer.

[0559] Step 7:

[0560] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[0561] Step 8:

[0562] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary.

[0563] Step 9:

[0564] The server records all questions and answers in a database. This ensures that if the same question arises again in the future, a quick response can be provided.

[0565] Through the processing steps described above, this system reduces the burden on senior employees and enables new graduates to quickly and consistently obtain the information they need.

[0566] (Example 1)

[0567] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0568] In the previous system, there was a problem in that it took a long time to resolve questions that new graduates had about daily tasks, attendance tracking, and how to use the company's internal systems. Furthermore, when specialized knowledge was required, the burden on the person in charge increased, which was another issue. This resulted in decreased work efficiency for new graduates and inconsistent quality of training.

[0569] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0570] In this invention, the server includes means for receiving user questions and analyzing them using natural language processing, means for searching a knowledge base based on the question analysis results, and means for generating estimated answers using a generative AI model. This makes it possible to automatically provide quick and appropriate answers to user-inputted questions, improve the work efficiency of new employees, and maintain a consistent quality of training.

[0571] A "user terminal" is a device used by a user to access the system and input questions.

[0572] A "server" is a central computing resource that receives, analyzes, searches databases, and generates answers for questions.

[0573] "Natural language processing" is a technology that analyzes data entered by users in natural language and extracts necessary keywords and information.

[0574] A "knowledge base" is a general term for databases that record FAQs and past questions and answers, and is a source of information that a system uses to generate answers to questions.

[0575] A "generative AI model" is an artificial intelligence model that generates appropriate answers based on received questions and search results; examples include GPT-3 and GPT-4.

[0576] An "estimated answer" is an answer that a generative AI model generates based on inference when no search results are found.

[0577] "Keywords" are important words or phrases extracted from the user's question using natural language processing.

[0578] "Knowledge-based search" is the process of searching for relevant information by referring to FAQ databases and past question-and-answer databases based on extracted keywords.

[0579] "Means of recording" refers to the function of saving all questions and generated answers in a database for future use.

[0580] The system according to the present invention efficiently provides support for new graduates' work, attendance tracking, instruction on how to use internal systems, and answers questions related to their work. An embodiment of this system is described below.

[0581] User terminal

[0582] User terminals are devices used by new employees to access the system and input questions in natural language. Specifically, they include personal computers, smartphones, and tablets. Users use these terminals to input questions in natural language, such as "Please tell me the procedure for starting a new project."

[0583] server

[0584] Servers are the central computing resources of a system and perform the following main functions:

[0585] 1. Receiving and analyzing questions

[0586] The server receives question data submitted by the user. The received data is then analyzed using natural language processing with Python libraries such as SpaCy and NLTK. As a result of the analysis, key keywords are extracted from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[0587] 2. Knowledge base search

[0588] Based on the analysis results, the server searches the FAQ database and databases containing past questions and answers. SQL queries and NoSQL database queries are used for this search process. For example, a query like "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Starting Procedure%'" is executed.

[0589] 3. Generating answers using generative AI models

[0590] The server generates appropriate answers based on information retrieved from the database. If search results are found, the AI ​​creates an answer based on their content. If no search results are found, a generative AI model (such as GPT-3 or GPT-4) generates an appropriate answer based on inferences. Specifically, in response to the question, "Please tell me the steps to start a new project," the server generates the answer, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[0591] 4. Submitting and viewing the generated responses

[0592] The server sends the generated response to the user's terminal. A REST API is used for transmission, and the response data is transferred in JSON format. The user's terminal parses and displays the received response. The user can review the displayed response and ask additional questions if necessary.

[0593] 5. Record of questions and answers

[0594] The server records all questions and generated answers in a database. For example, using MongoDB or MySQL, data is stored in the format "Question: 'How do I record my working hours?', Answer: 'Working hours are recorded using the following steps...'". This allows for quick response when similar questions arise in the future.

[0595] Specific example

[0596] When a new employee types "Please tell me how to record my working hours" into a terminal and sends it to the server, it will look like this:

[0597] 1. Enter and submit your question.

[0598] The user enters "Please tell me how to record my working hours" into the terminal.

[0599] 2. Reception and Analysis

[0600] The server receives the question and uses natural language processing (NLP) techniques to extract keywords such as "working hours" and "recording method."

[0601] 3. Knowledge base search

[0602] The server searches the FAQ database for information on "how to record working hours."

[0603] 4. Answer generation

[0604] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[0605] 5. Submit your response and display

[0606] The server generates an answer and sends it to the user's terminal, which then displays the answer.

[0607] 6. Record

[0608] The server records this question and the generated answer in the database.

[0609] This system allows us to provide quick and consistent answers to questions from new employees, improving work efficiency and maintaining a consistent quality of training.

[0610] Example of a prompt

[0611] "Please tell me how to start a new project."

[0612] "Please tell me how to record working hours."

[0613] "I have a question about how to use the company's internal systems."

[0614] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0615] System processing flow

[0616] Step 1:

[0617] The user logs into the device and enters the question.

[0618] Input: The user enters "Please tell me how to start a new project."

[0619] Operation: The terminal retrieves the entered question data.

[0620] Step 2:

[0621] The terminal sends the entered question data to the server.

[0622] Input: Question data "Please tell me how to start a new project."

[0623] Operation: The terminal securely sends the question data to the server using the HTTPS protocol.

[0624] Output: Send the question data to the server.

[0625] Step 3:

[0626] The server receives the question data and begins analysis using natural language processing (NLP).

[0627] Input: Question data sent from the device

[0628] Operation: The server analyzes the question content using Python libraries (e.g., SpaCy or NLTK). It extracts key keywords.

[0629] Example: Identify keywords such as "new project" and "starting procedures".

[0630] Output: Extracted keywords

[0631] Step 4:

[0632] The server searches the knowledge base based on the analysis results.

[0633] Input: Extracted keywords "New project" and "Starting procedure"

[0634] Operation: The server uses SQL queries and NoSQL databases to search the FAQ database and past question and answer databases.

[0635] For example, in the case of an SQL database, execute the query "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Start Procedure%'".

[0636] Output: Search Results

[0637] Step 5:

[0638] The server generates an appropriate answer based on the search results. If no search results are found, a generative AI model is used to generate an estimated answer.

[0639] Input: Search Results

[0640] Operation: The server generates an answer based on the search results. If no search results are found, it estimates an answer using a generative AI model (e.g., GPT-3 or GPT-4).

[0641] Example: If there are search results, it will generate a response such as, "The steps to start a new project are as follows: 1. Fill out the project application form 2. Obtain approval from your supervisor 3. Form a project team 4. Prepare the necessary materials."

[0642] Output: Generated answer

[0643] Step 6:

[0644] The server sends the generated response to the user's terminal.

[0645] Input: Generated answer

[0646] Operation: The server sends response data in JSON format using a REST API.

[0647] Output: Send the response data to the user's terminal.

[0648] Step 7:

[0649] The device displays the received response to the user.

[0650] Input: Response data sent from the server

[0651] Operation: The device parses the JSON data and displays it on the screen. The user can then review the displayed response.

[0652] Output: Answer displayed to the user

[0653] Step 8:

[0654] The server records all questions and generated answers in a database.

[0655] Input: Question data and generated response data

[0656] Operation: The server records the questions and answers in a database (e.g., MongoDB or MySQL).

[0657] For example, record it in the format: "Question: 'Please tell me how to start a new project,' Answer: 'The steps to start a new project are as follows...'"

[0658] Output: Recorded data

[0659] (Application Example 1)

[0660] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0661] In a factory environment, a sophisticated training support system is necessary for new employees to receive appropriate guidance and quickly begin their work. However, the current system makes it difficult for new employees to get quick answers to their questions about operation methods and work procedures. Furthermore, there are concerns that the quality of training will decline because consistent answers to questions are not provided.

[0662] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0663] In this invention, the server includes means for receiving and analyzing user questions, means for searching a database based on the question analysis results, and means for generating answers based on the search results. This makes it possible to provide rapid and consistent answers using a generative AI model for new employees to answer questions from factory automation machines in a factory environment.

[0664] A "user terminal" is a device used to input questions and communicate with a server. Examples include personal computers, smartphones, and tablets.

[0665] A "server" is a central processing unit that analyzes questions received from users and generates appropriate answers. It performs tasks such as analyzing question data, searching databases, generating answers, and recording them.

[0666] A "generative AI model" is an artificial intelligence technology used to analyze a question and generate an appropriate answer. For example, natural language processing technologies such as OpenAI's GPT-3 fall into this category.

[0667] "Natural language processing" is a technology that understands questions entered by users in natural language and extracts appropriate keywords.

[0668] A "database" is an information storage system that stores past questions and answers, as well as FAQ information. The server searches this database to find answers to questions.

[0669] "Factory automation machinery" refers to machines that perform tasks in a factory automatically. Examples include robotic arms and automated transport systems.

[0670] "Means for entering questions" refers to an interface that allows users to enter questions in natural language and send them to the server.

[0671] The system according to the present invention provides a question-answering system using a generative AI model to efficiently support the training of new employees in a factory environment. This system includes a user terminal, a server, and a database.

[0672] User terminal

[0673] User terminals are devices used by new employees to input questions to factory automation machinery. Specifically, these include personal computers, tablets, and smartphones. Users input questions in natural language using these terminals, and these questions are sent to the server.

[0674] server

[0675] The server is the central computer of this system. The server receives questions submitted by users and analyzes them using a generative AI model (e.g., OpenAI GPT-3). Natural language processing techniques (e.g., spaCy) are used for the analysis to extract key keywords.

[0676] The server searches the database based on the analysis results. The database contains past questions and answers, as well as FAQ information. If search results exist, it generates an answer based on that information. Even if no search results are found, the AI ​​model automatically generates an estimated answer.

[0677] The generated responses are sent from the server to the user's terminal. The user's terminal displays these responses to the user, providing consistent educational support.

[0678] database

[0679] A database is a system that stores information for the server to perform searches. This includes past questions and answers, and FAQ information. The server uses this database to provide the most appropriate answer to the user's question.

[0680] Specific example

[0681] For example, if a new employee asks a factory automation machine, "How do I maintain this machine?", the process would proceed as follows: The question is entered into the user terminal and sent to the server. The server receives the question and extracts the keywords "machine" and "maintenance method". It searches the database to find information about maintenance procedures and generates an answer, such as "The maintenance method for this machine is as follows: 1. Turn off the power. 2. Open the panel. 3. Clean the filter." The generated answer is sent to the user terminal and displayed.

[0682] Example of a prompt

[0683] The prompt message when a new employee enters the question, "Please tell me how to operate this machine," is as follows:

[0684] New employee: Please explain how to operate this machine.

[0685] Robot AI:

[0686] In this way, the system of the present invention can respond quickly and accurately to questions from new employees and improve the quality of training.

[0687] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0688] Step 1:

[0689] The user enters a question in natural language on their device. The entered question is a specific text, such as "Please tell me how to maintain this machine." The user's device then sends this text data to the server.

[0690] Input: Question text in natural language

[0691] Output: Question data sent to the server

[0692] Step 2:

[0693] The server receives question data sent from the user's terminal. The server converts this data into a format for input into a generative AI model (OpenAI GPT-3). Specifically, it formats the question text into a prompt.

[0694] Input: Question data from the user terminal

[0695] Output: Prompt text input to the generative AI model

[0696] Step 3:

[0697] The server uses a generative AI model to analyze the content of the received question. In this process, natural language processing technology (such as spaCy) is used to extract key keywords from the question. For example, from the question "Please tell me how to maintain this machine," keywords such as "machine" and "maintenance method" are extracted.

[0698] Input: Prompt message

[0699] Output: Extracted keywords

[0700] Step 4:

[0701] The server searches the database based on the extracted keywords. This includes databases of past questions and answers and FAQ information. If relevant information exists, it is retrieved. For example, if detailed instructions regarding "maintenance methods" are stored in the database, that information will be retrieved as a search result.

[0702] Input: Extracted keywords

[0703] Output: Search results data

[0704] Step 5:

[0705] The server generates an appropriate answer based on the search results. If search results exist, it constructs the answer text based on that information. For example, it generates a specific answer such as, "The maintenance procedure for this machine is as follows: 1. Turn off the power 2. Open the panel 3. Clean the filter." If no search results are found, a generative AI model generates an estimated answer.

[0706] Input: Search result data

[0707] Output: Generated answer text

[0708] Step 6:

[0709] The server sends the generated response text to the user's terminal. The user's terminal displays the received response. This allows the new employee to review the displayed response and respond appropriately.

[0710] Input: Generated response text

[0711] Output: The answer displayed on the user's terminal.

[0712] Step 7:

[0713] The server records all questions and generated answers in a database. This allows for a quick response if the same question arises in the future.

[0714] Input: Question data and answer data

[0715] Output: Questions and answers recorded in the database

[0716] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0717] The system according to the present invention utilizes generative AI and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[0718] System Configuration

[0719] 1. User terminal

[0720] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0721] 2. Server

[0722] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching databases, sentiment analysis, and answer generation. The server is equipped with a generative AI with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database that records past questions and answers.

[0723] Flow of operations

[0724] 1. Enter and submit your question.

[0725] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[0726] 2. Receiving the questionnaire data

[0727] The terminal sends the entered question data to the server. The question entered by the user is transferred to the server.

[0728] 3. Analysis of Questionnaire Data

[0729] The server receives the question data and uses generative AI to analyze the question content. It extracts key keywords and phrases from the question and understands the intent.

[0730] 4. Emotion analysis

[0731] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0732] 5. Database Search

[0733] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[0734] 6. Generating the answer

[0735] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[0736] 7. Submitting and displaying responses

[0737] The server sends the generated response data to the user's terminal.

[0738] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[0739] 8. Record of questions and answers

[0740] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0741] Specific example

[0742] For example, if new employee Sato enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[0743] 1. Enter and submit your question.

[0744] Mr. Sato (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[0745] 2. Reception and Analysis

[0746] The server receives the question and extracts the keywords "working hours" and "recording method."

[0747] 3. Emotion analysis

[0748] The server uses an emotion engine to detect the "confusion" contained in Sato's question.

[0749] 4. Database Search

[0750] The server searches the FAQ database for information on "how to record working hours."

[0751] 5. Answer generation

[0752] The system generates the following response: "Working hours are recorded using the following steps: 1. Log in to the company system 2. Access the attendance management page 3. Enter your start and end times 4. Click the save button." Sensing confusion, it also adds supplementary explanations in a gentle tone.

[0753] 6. Submit your response and display

[0754] The server sends the generated response to Mr. Sato's terminal, and the terminal displays the response.

[0755] 7. Record

[0756] The server records this question and the generated answer in the database.

[0757] In this way, by recognizing the difficulties new employees are facing and providing appropriate answers, the quality of their training can be improved.

[0758] The following describes the processing flow.

[0759] Step 1:

[0760] The user logs into their device and accesses the question input screen. The user enters the question in natural language and clicks the submit button.

[0761] Step 2:

[0762] The terminal sends the entered question data to the server. Specifically, the string entered by the user is sent to the server as a packet.

[0763] Step 3:

[0764] The server receives the question data. The server parses the received data and reads the string data.

[0765] Step 4:

[0766] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[0767] Step 5:

[0768] The server uses an emotion engine to analyze the emotions contained in the user's question. The analysis employs techniques to classify emotions from words and phrases contained in the text. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0769] Step 6:

[0770] The server searches the database based on keyword analysis results and sentiment analysis results. It searches the FAQ database and past question and answer database to find relevant information.

[0771] Step 7:

[0772] The server evaluates the search results and generates an appropriate response. If a search result is found, it creates a response based on that data. Even if no search results are found, the AI ​​generates a predicted response. Furthermore, it adjusts the response to have an appropriate tone and content based on sentiment analysis results.

[0773] Step 8:

[0774] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[0775] Step 9:

[0776] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary. For example, to alleviate any confusion the user might feel, detailed instructions in a gentle tone are displayed.

[0777] Step 10:

[0778] The server records all questions and answers in a database. This ensures that if the same question arises again in the future, a quick response will be possible.

[0779] Through the processing steps described above, this system reduces the burden on senior employees and enables new employees to quickly and consistently obtain necessary information with a high level of emotional understanding.

[0780] (Example 2)

[0781] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0782] In today's business environment, it is crucial for new graduates to adapt to their work quickly and efficiently. However, traditional FAQ systems and manuals are limited to providing static information and struggle to address users' specific questions and concerns. As a result, new graduates may not receive adequate guidance and support, which can delay their adaptation to their work. Furthermore, the lack of timely and appropriate answers to questions can lead to decreased work efficiency.

[0783] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0784] In this invention, the server includes means for performing question analysis using a generative artificial intelligence model, means for performing sentiment analysis of the question, and means for generating an answer based on the search results and sentiment analysis results. This makes it possible to quickly provide appropriate answers to the user's specific questions and emotions.

[0785] A "user terminal" is a device used by a user to access the system and enter questions, and includes personal computers, smartphones, tablets, and other similar devices.

[0786] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[0787] A "generative artificial intelligence model" refers to an artificial intelligence technology that uses natural language processing (NLP) to generate answers to questions based on the generated data.

[0788] "Question analysis" refers to the process of analyzing a user's question, extracting key keywords and phrases, and understanding their intent.

[0789] "Sentiment analysis" refers to the process of detecting the emotions contained in a user's question (e.g., anxiety, confusion, excitement) and adjusting the tone of the response based on that.

[0790] A "database" refers to a storage device that stores information that a system can refer to, such as FAQ data and past question-and-answer data.

[0791] "Answer generation" refers to the process of constructing and generating appropriate answers based on search results and sentiment analysis results.

[0792] "Recording" refers to the process of saving all questions and their answers in a database so that they can be referenced later.

[0793] The system according to the present invention utilizes a generative artificial intelligence model and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new employees. This system consists of a user terminal, a server, a generative artificial intelligence model, an emotion engine, and a database.

[0794] System Configuration

[0795] 1. User terminal

[0796] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0797] 2. Server

[0798] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, sentiment analysis, and answer generation. The server is equipped with a generative artificial intelligence model with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database recording past questions and answers. Specifically, OpenAI's GPT-4 is used as the generative AI model, and IBM Watson's Tone Analyzer is used as the sentiment engine.

[0799] Flow of operations

[0800] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[0801] The terminal sends the entered question data to the server.

[0802] The server receives the question data and analyzes the question content using a generative artificial intelligence model. It extracts key keywords and phrases from the question and understands the intent.

[0803] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0804] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[0805] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[0806] The server sends the generated response data to the user's terminal.

[0807] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[0808] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0809] Specific example

[0810] For example, if a new employee enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[0811] The user types "Please tell me how to record my working hours" into the terminal and clicks the send button.

[0812] The server receives the question and extracts the keywords "working hours" and "recording method."

[0813] The server uses an emotion engine to detect "confusion" contained in the question.

[0814] The server searches the FAQ database for information on "how to record working hours."

[0815] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system 2. Access the attendance management page 3. Enter your start and end times 4. Click the save button," and, sensing "confusion," adds supplementary explanations in a gentle tone.

[0816] The server generates an answer and sends it to the user's device, which then displays the answer on its screen.

[0817] The server records this question and the generated answer in the database.

[0818] In this way, the quality of training can be improved by sensing when new employees are having trouble and providing appropriate answers accordingly. Examples of specific prompts include: "Please tell me how to start a new project," "Please tell me how to connect to the company Wi-Fi," and "Please tell me how to apply for paid leave."

[0819] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0820] Step 1:

[0821] The user accesses the login screen on their device, enters their ID and password, and logs into the system. After successfully logging in, they enter their specific question or inquiry in natural language in the question input field. For example, they might enter, "Please tell me how to start a new project." Then, the user clicks the "Submit" button, and the question data is sent.

[0822] Input: User ID, password, and security question.

[0823] Output: Login authentication result, submitted question data

[0824] Step 2:

[0825] The terminal sends the user-entered question data to the server. Specifically, the question data is sent to the server using a communication protocol such as an HTTP request. The entered data is then converted to JSON format.

[0826] Input: User's question data

[0827] Output: Question data in JSON format sent to the server

[0828] Step 3:

[0829] The server receives the question data and analyzes the question content using a generative artificial intelligence model. The server analyzes the received JSON data and uses a generative AI model (e.g., OpenAI's GPT-4) to extract the main keywords and phrases of the question. For example, from the question "Please tell me the procedure for starting a new project," the keywords "project," "start," and "procedure" are extracted.

[0830] Input: Question data in JSON format

[0831] Output: Extracted keywords and analysis results

[0832] Step 4:

[0833] The server uses an emotion engine (e.g., IBM Watson's Tone Analyzer) to analyze the emotions contained in the user's questions. Specifically, it detects emotions such as "anxiety," "confusion," and "excitement" based on the received question data. In this step, an emotion score is generated, and the user's emotional state is evaluated.

[0834] Input: Question data

[0835] Output: Sentiment analysis results (emotion score and emotional state)

[0836] Step 5:

[0837] The server searches the FAQ database and past question-and-answer database based on keyword and sentiment analysis results. It uses the extracted keywords (e.g., "project start procedure") as search queries to find relevant information from the database. The search is performed using appropriate SQL queries and search algorithms.

[0838] Input: Keyword analysis results, sentiment analysis results

[0839] Output: Search results (related FAQs and past questions / answers)

[0840] Step 6:

[0841] The server evaluates the search results and generates an appropriate response. It uses a generative artificial intelligence model to generate responses based on the search results. For example, if detailed instructions on "how to start a new project" are found in the search results, the response will be constructed based on that information. If no results are found, the AI ​​will generate a speculative response. The response is then adjusted to an appropriate tone, taking sentiment analysis results into consideration.

[0842] Input: Search results, sentiment analysis results

[0843] Output: Generated response data

[0844] Step 7:

[0845] The server converts the generated response data into JSON format and sends it to the user's terminal. The generated response data is then transferred to the terminal as an HTTP response.

[0846] Input: Generated response data

[0847] Output: JSON format response data sent to the terminal

[0848] Step 8:

[0849] The device analyzes the received response data and displays it to the user. The response is displayed on the device's user interface (UI). The user can review the response on the screen and re-enter the question if necessary.

[0850] Input: Response data in JSON format

[0851] Output: Answer displayed to the user

[0852] Step 9:

[0853] The server records all questions and generated answers in a database. This allows for quick response when the same question arises in the future. The data is recorded along with the question content, the answer content, and their associated metadata.

[0854] Input: Question data, generated response data

[0855] Output: Records stored in the database

[0856] (Application Example 2)

[0857] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0858] In the factory, new graduates are unfamiliar with their work and require efficient support. However, the current system is inadequate in handling questions in natural language and fails to provide appropriate support that takes emotions into account. Furthermore, the lack of real-time speech recognition and response capabilities makes it difficult to provide immediate solutions. This is leading to decreased work efficiency for new graduates and a decline in the quality of their training. In this situation, there is a need for a system that allows new graduates to perform their duties smoothly without feeling confused or anxious.

[0859] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0860] In this invention, the server includes means for receiving and analyzing a user's question, means for searching a database based on the question analysis results, and means for analyzing the emotions contained in the user's question. This allows for the analysis of the user's emotions along with the question content, enabling the generation of an answer in the optimal tone and immediate voice response.

[0861] A "user terminal" is a device used by a user to access a system and input questions. Examples include personal computers, smartphones, and tablets.

[0862] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[0863] "Question analysis" is the process by which a server uses natural language processing to extract key keywords and phrases from a question received from a user and understand the user's intent.

[0864] "Database search" refers to the process where the server searches the FAQ database and past question / answer database based on the question analysis results to find relevant information.

[0865] "Answer generation" refers to the process by which a server evaluates the search results in a database, constructs an appropriate answer, and, in some cases, generates an estimated answer.

[0866] "Sending a response" refers to the process of sending the response data generated by the server to the user's terminal.

[0867] "Displaying responses" refers to the process of displaying the responses received by the user's terminal to the user on the screen.

[0868] "Question and answer recording" refers to the process where the server saves all questions and answers in a database. This makes it possible to provide a quick answer when the same question arises in the future.

[0869] "Emotional analysis" is the process by which a server uses an emotion engine to analyze the emotions contained in a user's question and detect emotions such as "anxiety," "confusion," and "excitement."

[0870] "Speech recognition" is the process by which a robot converts a user's voice into text. This process allows the user to input questions using their voice.

[0871] "Voice output" is the process by which the robot communicates its generated response to the user via voice. Voice output allows the user to receive the response intuitively.

[0872] This invention provides a system that enables new employees to use robots within a factory to provide real-time work support. This system utilizes generative AI and an emotion engine and operates in conjunction with user terminals such as smartphones and tablets.

[0873] System Configuration

[0874] 1. User terminal

[0875] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0876] 2. Robots

[0877] The robot is a device used by new employees within the factory and accepts voice input from users. The robot has the function of converting speech into text using speech recognition technology and outputting the generated response as voice.

[0878] 3. Server

[0879] A server is the central computing resource of a system. A server has the following functions:

[0880] Receiving and analyzing question data

[0881] Implementation of an emotion engine for performing emotion analysis

[0882] Searching Databases

[0883] Response generation and evaluation

[0884] Record of questions and answers

[0885] Flow of operations

[0886] 1. Enter your question

[0887] The user logs into the terminal, enters their question in natural language, and submits it. For example, they might enter, "Please tell me how to start a new project."

[0888] 2. Voice recognition and question submission

[0889] When a user inputs a question into the robot via voice, the robot converts the voice into text and sends it to the server.

[0890] 3. Analysis of Questionnaire Data

[0891] The server receives the question data and analyzes the question content using a generative AI model. It extracts key keywords and phrases from the question and understands the intent.

[0892] 4. Emotion analysis

[0893] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[0894] 5. Database Search

[0895] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[0896] 6. Generating the answer

[0897] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs a response based on that data. If no search results are found, it generates an estimated response using a generative AI model. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[0898] 7. Submitting and displaying responses

[0899] The server sends the generated response data to the user's terminal, and the robot outputs the response verbally. The user can also check the response on the terminal's screen.

[0900] 8. Record of questions and answers

[0901] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[0902] Hardware and software to use

[0903] Generative AI model: GPT-2 (using the transformers library)

[0904] Emotion Engine: Proprietary emotion analysis engine (sentiment_analysis)

[0905] User devices: PCs, smartphones, tablets

[0906] Robot: A device with voice recognition and voice output capabilities.

[0907] Specific example

[0908] For example, if a new employee types "Please tell me how to start a new project" into the terminal, the system will respond as follows:

[0909] 1. Enter and submit your question.

[0910] The user types "Please tell me how to start a new project" and submits it.

[0911] 2. Reception and Analysis

[0912] The server receives the question, extracts the keywords "project," "start," and "procedure," and understands the intent.

[0913] 3. Emotion analysis

[0914] The server uses an emotion engine to detect "confusion" contained in the user's question.

[0915] 4. Database Search

[0916] The server searches the FAQ database for information on "Project Start Procedures".

[0917] 5. Answer generation

[0918] The system generates a response stating, "The project initiation procedure is as follows: 1. Fill out the application form 2. Obtain approval 3. Select members 4. Set a schedule," and adds supplementary explanations in a gentle tone based on the sentiment analysis results.

[0919] 6. Submit your response and display

[0920] The server generates an answer, which is sent to the user's terminal, and the robot outputs it verbally.

[0921] Example of a prompt

[0922] "Please tell me how to start a new project."

[0923] "What should I do if the machine breaks down?"

[0924] "Please tell me how to record working hours."

[0925] This system provides an environment where new graduates can perform their duties smoothly without feeling confused or anxious.

[0926] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0927] Step 1:

[0928] The user logs into their device and enters their question in natural language.

[0929] Input: The user operates the terminal and enters the question in natural language (e.g., "Please tell me how to start a new project").

[0930] Output: Question text data is generated.

[0931] Step 2:

[0932] The robot receives voice input from the user, converts it to text using speech recognition technology, and sends it to the server.

[0933] Input: Voice input from the user.

[0934] Output: The audio data is converted to text data and sent to the server.

[0935] Step 3:

[0936] The server receives the question data and analyzes the question using a generative AI model.

[0937] Input: Question in text data format.

[0938] Data processing: A generative AI model (GPT-2) analyzes the question content and extracts key keywords and phrases.

[0939] Output: Analyzed keywords and phrases.

[0940] Step 4:

[0941] The server uses an emotion analysis engine to analyze the emotions contained in the user's questions.

[0942] Input: Question text.

[0943] Data processing: The emotion analysis engine analyzes the text and detects emotions such as "anxiety," "confusion," and "excitement."

[0944] Output: Detected emotion information.

[0945] Step 5:

[0946] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[0947] Input: Analyzed keywords and sentiment information.

[0948] Data Search: Searches for server-related information from the FAQ database and past question-and-answer database.

[0949] Output: Search results.

[0950] Step 6:

[0951] The server evaluates the search results and generates appropriate answers.

[0952] Input: Search results from a database.

[0953] Data processing: The server constructs an answer based on the search results and adjusts the answer to an appropriate tone according to the sentiment analysis results.

[0954] Output: Generated answer text.

[0955] Step 7:

[0956] The server generates response data and sends it to the user's terminal, and the robot outputs the response via voice.

[0957] Input: Generated response text.

[0958] Data transfer: The server sends the response data to the user's terminal.

[0959] Data output: The robot outputs the answer in voice.

[0960] Output: A voice response to the user.

[0961] Step 8:

[0962] The server records all questions and answers in a database.

[0963] Input: Question and generated answer data.

[0964] Data storage: The server stores questions and answers in a database.

[0965] Output: Database entries for quick responses to future questions.

[0966] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0967] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0968] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0969] [Third Embodiment]

[0970] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0971] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0972] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0973] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0974] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0975] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0976] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0977] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0978] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0979] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0980] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0981] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0982] The system according to the present invention utilizes generative AI to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[0983] System Configuration

[0984] 1. User terminal

[0985] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[0986] 2. Server

[0987] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, and generating answers. The server is equipped with a generative AI with natural language processing (NLP) capabilities, an FAQ database, and a database that records past questions and answers.

[0988] Flow of operations

[0989] 1. Enter and submit your question.

[0990] The user logs into their device and enters their question in natural language. For example, they might enter the question, "How do I start a new project?"

[0991] The terminal sends the entered question data to the server.

[0992] 2. Receiving and analyzing questions

[0993] The server receives the question data and analyzes the question content using generative AI. Natural language processing techniques are used for the analysis to extract key keywords from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[0994] 3. Database Search

[0995] Based on the analysis results, the server searches the FAQ database and the past question and answer database. For example, if "Steps to start a new project" is already registered in the FAQ database, that information will be retrieved.

[0996] 4. Generating the answer

[0997] The server generates an appropriate answer based on the search results. If a search result is found, it constructs an answer based on it; if no result is found, the AI ​​generates a predicted answer.

[0998] For example, it can generate a response like, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[0999] 5. Submitting and displaying responses

[1000] The server sends the generated response to the user's terminal.

[1001] The device displays the received responses to the user. The user can review the displayed responses and ask additional questions if necessary.

[1002] 6. Record of questions and answers

[1003] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1004] Specific example

[1005] For example, if new employee Tanaka enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[1006] 1. Enter and submit your question.

[1007] Mr. Tanaka (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[1008] 2. Reception and Analysis

[1009] The server receives the question and extracts the keywords "working hours" and "recording method."

[1010] 3. Database Search

[1011] The server searches the FAQ database for information on "how to record working hours."

[1012] 4. Answer generation

[1013] The system generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[1014] 5. Submit your response and display

[1015] The server generates an answer and sends it to Tanaka's terminal, which then displays the answer.

[1016] 6. Record

[1017] The server records this question and the generated answer in the database.

[1018] In this way, we can provide prompt and consistent answers to questions from new employees and improve the quality of their training.

[1019] The following describes the processing flow.

[1020] Step 1:

[1021] The user logs into their device and accesses the question input screen. The user enters the question in natural language and clicks the submit button.

[1022] Step 2:

[1023] The terminal sends the entered question data to the server. Specifically, the string entered by the user is sent to the server as a packet.

[1024] Step 3:

[1025] The server receives the question data. The server parses the received data and reads the string data.

[1026] Step 4:

[1027] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[1028] Step 5:

[1029] The server searches the database based on the analysis results. It refers to the FAQ database and past question and answer database to find relevant information.

[1030] Step 6:

[1031] The server evaluates the search results and generates an appropriate answer. If a search result is found, it creates an answer based on that data. If no search results are found, the AI ​​generates a predicted answer.

[1032] Step 7:

[1033] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[1034] Step 8:

[1035] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary.

[1036] Step 9:

[1037] The server records all questions and answers in a database. This ensures that if the same question arises again in the future, a quick response can be provided.

[1038] Through the processing steps described above, this system reduces the burden on senior employees and enables new graduates to quickly and consistently obtain the information they need.

[1039] (Example 1)

[1040] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1041] In the previous system, there was a problem in that it took a long time to resolve questions that new graduates had about daily tasks, attendance tracking, and how to use the company's internal systems. Furthermore, when specialized knowledge was required, the burden on the person in charge increased, which was another issue. This resulted in decreased work efficiency for new graduates and inconsistent quality of training.

[1042] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1043] In this invention, the server includes means for receiving user questions and analyzing them using natural language processing, means for searching a knowledge base based on the question analysis results, and means for generating estimated answers using a generative AI model. This makes it possible to automatically provide quick and appropriate answers to user-inputted questions, improve the work efficiency of new employees, and maintain a consistent quality of training.

[1044] A "user terminal" is a device used by a user to access the system and input questions.

[1045] A "server" is a central computing resource that receives, analyzes, searches databases, and generates answers for questions.

[1046] "Natural language processing" is a technology that analyzes data entered by users in natural language and extracts necessary keywords and information.

[1047] A "knowledge base" is a general term for databases that record FAQs and past questions and answers, and is a source of information that a system uses to generate answers to questions.

[1048] A "generative AI model" is an artificial intelligence model that generates appropriate answers based on received questions and search results; examples include GPT-3 and GPT-4.

[1049] An "estimated answer" is an answer that a generative AI model generates based on inference when no search results are found.

[1050] "Keywords" are important words or phrases extracted from the user's question using natural language processing.

[1051] "Knowledge-based search" is the process of searching for relevant information by referring to FAQ databases and past question-and-answer databases based on extracted keywords.

[1052] "Means of recording" refers to the function of saving all questions and generated answers in a database for future use.

[1053] The system according to the present invention efficiently provides support for new graduates' work, attendance tracking, instruction on how to use internal systems, and answers questions related to their work. An embodiment of this system is described below.

[1054] User terminal

[1055] User terminals are devices used by new employees to access the system and input questions in natural language. Specifically, they include personal computers, smartphones, and tablets. Users use these terminals to input questions in natural language, such as "Please tell me the procedure for starting a new project."

[1056] server

[1057] Servers are the central computing resources of a system and perform the following main functions:

[1058] 1. Receiving and analyzing questions

[1059] The server receives question data submitted by the user. The received data is then analyzed using natural language processing with Python libraries such as SpaCy and NLTK. As a result of the analysis, key keywords are extracted from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[1060] 2. Knowledge base search

[1061] Based on the analysis results, the server searches the FAQ database and databases containing past questions and answers. SQL queries and NoSQL database queries are used for this search process. For example, a query like "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Starting Procedure%'" is executed.

[1062] 3. Generating answers using generative AI models

[1063] The server generates appropriate answers based on information retrieved from the database. If search results are found, the AI ​​creates an answer based on their content. If no search results are found, a generative AI model (such as GPT-3 or GPT-4) generates an appropriate answer based on inferences. Specifically, in response to the question, "Please tell me the steps to start a new project," the server generates the answer, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[1064] 4. Submitting and viewing the generated responses

[1065] The server sends the generated response to the user's terminal. A REST API is used for transmission, and the response data is transferred in JSON format. The user's terminal parses and displays the received response. The user can review the displayed response and ask additional questions if necessary.

[1066] 5. Record of questions and answers

[1067] The server records all questions and generated answers in a database. For example, using MongoDB or MySQL, data is stored in the format "Question: 'How do I record my working hours?', Answer: 'Working hours are recorded using the following steps...'". This allows for quick response when similar questions arise in the future.

[1068] Specific example

[1069] When a new employee types "Please tell me how to record my working hours" into a terminal and sends it to the server, it will look like this:

[1070] 1. Enter and submit your question.

[1071] The user enters "Please tell me how to record my working hours" into the terminal.

[1072] 2. Reception and Analysis

[1073] The server receives the question and uses natural language processing (NLP) techniques to extract keywords such as "working hours" and "recording method."

[1074] 3. Knowledge base search

[1075] The server searches the FAQ database for information on "how to record working hours."

[1076] 4. Answer generation

[1077] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[1078] 5. Submit your response and display

[1079] The server generates an answer and sends it to the user's terminal, which then displays the answer.

[1080] 6. Record

[1081] The server records this question and the generated answer in the database.

[1082] This system allows us to provide quick and consistent answers to questions from new employees, improving work efficiency and maintaining a consistent quality of training.

[1083] Example of a prompt

[1084] "Please tell me how to start a new project."

[1085] "Please tell me how to record working hours."

[1086] "I have a question about how to use the company's internal systems."

[1087] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1088] System processing flow

[1089] Step 1:

[1090] The user logs into the device and enters the question.

[1091] Input: The user enters "Please tell me how to start a new project."

[1092] Operation: The terminal retrieves the entered question data.

[1093] Step 2:

[1094] The terminal sends the entered question data to the server.

[1095] Input: Question data "Please tell me how to start a new project."

[1096] Operation: The terminal securely sends the question data to the server using the HTTPS protocol.

[1097] Output: Send the question data to the server.

[1098] Step 3:

[1099] The server receives the question data and begins analysis using natural language processing (NLP).

[1100] Input: Question data sent from the device

[1101] Operation: The server analyzes the question content using Python libraries (e.g., SpaCy or NLTK). It extracts key keywords.

[1102] Example: Identify keywords such as "new project" and "starting procedures".

[1103] Output: Extracted keywords

[1104] Step 4:

[1105] The server searches the knowledge base based on the analysis results.

[1106] Input: Extracted keywords "New project" and "Starting procedure"

[1107] Operation: The server uses SQL queries and NoSQL databases to search the FAQ database and past question and answer databases.

[1108] For example, in the case of an SQL database, execute the query "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Start Procedure%'".

[1109] Output: Search Results

[1110] Step 5:

[1111] The server generates an appropriate answer based on the search results. If no search results are found, a generative AI model is used to generate an estimated answer.

[1112] Input: Search Results

[1113] Operation: The server generates an answer based on the search results. If no search results are found, it estimates an answer using a generative AI model (e.g., GPT-3 or GPT-4).

[1114] Example: If there are search results, it will generate a response such as, "The steps to start a new project are as follows: 1. Fill out the project application form 2. Obtain approval from your supervisor 3. Form a project team 4. Prepare the necessary materials."

[1115] Output: Generated answer

[1116] Step 6:

[1117] The server sends the generated response to the user's terminal.

[1118] Input: Generated answer

[1119] Operation: The server sends response data in JSON format using a REST API.

[1120] Output: Send the response data to the user's terminal.

[1121] Step 7:

[1122] The device displays the received response to the user.

[1123] Input: Response data sent from the server

[1124] Operation: The device parses the JSON data and displays it on the screen. The user can then review the displayed response.

[1125] Output: Answer displayed to the user

[1126] Step 8:

[1127] The server records all questions and generated answers in a database.

[1128] Input: Question data and generated response data

[1129] Operation: The server records the questions and answers in a database (e.g., MongoDB or MySQL).

[1130] For example, record it in the format: "Question: 'Please tell me how to start a new project,' Answer: 'The steps to start a new project are as follows...'"

[1131] Output: Recorded data

[1132] (Application Example 1)

[1133] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1134] In a factory environment, a sophisticated training support system is necessary for new employees to receive appropriate guidance and quickly begin their work. However, the current system makes it difficult for new employees to get quick answers to their questions about operation methods and work procedures. Furthermore, there are concerns that the quality of training will decline because consistent answers to questions are not provided.

[1135] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1136] In this invention, the server includes means for receiving and analyzing user questions, means for searching a database based on the question analysis results, and means for generating answers based on the search results. This makes it possible to provide rapid and consistent answers using a generative AI model for new employees to answer questions from factory automation machines in a factory environment.

[1137] A "user terminal" is a device used to input questions and communicate with a server. Examples include personal computers, smartphones, and tablets.

[1138] A "server" is a central processing unit that analyzes questions received from users and generates appropriate answers. It performs tasks such as analyzing question data, searching databases, generating answers, and recording them.

[1139] A "generative AI model" is an artificial intelligence technology used to analyze a question and generate an appropriate answer. For example, natural language processing technologies such as OpenAI's GPT-3 fall into this category.

[1140] "Natural language processing" is a technology that understands questions entered by users in natural language and extracts appropriate keywords.

[1141] A "database" is an information storage system that stores past questions and answers, as well as FAQ information. The server searches this database to find answers to questions.

[1142] "Factory automation machinery" refers to machines that perform tasks in a factory automatically. Examples include robotic arms and automated transport systems.

[1143] "Means for entering questions" refers to an interface that allows users to enter questions in natural language and send them to the server.

[1144] The system according to the present invention provides a question-answering system using a generative AI model to efficiently support the training of new employees in a factory environment. This system includes a user terminal, a server, and a database.

[1145] User terminal

[1146] User terminals are devices used by new employees to input questions to factory automation machinery. Specifically, these include personal computers, tablets, and smartphones. Users input questions in natural language using these terminals, and these questions are sent to the server.

[1147] server

[1148] The server is the central computer of this system. The server receives questions submitted by users and analyzes them using a generative AI model (e.g., OpenAI GPT-3). Natural language processing techniques (e.g., spaCy) are used for the analysis to extract key keywords.

[1149] The server searches the database based on the analysis results. The database contains past questions and answers, as well as FAQ information. If search results exist, it generates an answer based on that information. Even if no search results are found, the AI ​​model automatically generates an estimated answer.

[1150] The generated responses are sent from the server to the user's terminal. The user's terminal displays these responses to the user, providing consistent educational support.

[1151] database

[1152] A database is a system that stores information for the server to perform searches. This includes past questions and answers, and FAQ information. The server uses this database to provide the most appropriate answer to the user's question.

[1153] Specific example

[1154] For example, if a new employee asks a factory automation machine, "How do I maintain this machine?", the process would proceed as follows: The question is entered into the user terminal and sent to the server. The server receives the question and extracts the keywords "machine" and "maintenance method". It searches the database to find information about maintenance procedures and generates an answer, such as "The maintenance method for this machine is as follows: 1. Turn off the power. 2. Open the panel. 3. Clean the filter." The generated answer is sent to the user terminal and displayed.

[1155] Example of a prompt

[1156] The prompt message when a new employee enters the question, "Please tell me how to operate this machine," is as follows:

[1157] New employee: Please explain how to operate this machine.

[1158] Robot AI:

[1159] In this way, the system of the present invention can respond quickly and accurately to questions from new employees and improve the quality of training.

[1160] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1161] Step 1:

[1162] The user enters a question in natural language on their device. The entered question is a specific text, such as "Please tell me how to maintain this machine." The user's device then sends this text data to the server.

[1163] Input: Question text in natural language

[1164] Output: Question data sent to the server

[1165] Step 2:

[1166] The server receives question data sent from the user's terminal. The server converts this data into a format for input into a generative AI model (OpenAI GPT-3). Specifically, it formats the question text into a prompt.

[1167] Input: Question data from the user terminal

[1168] Output: Prompt text input to the generative AI model

[1169] Step 3:

[1170] The server uses a generative AI model to analyze the content of the received question. In this process, natural language processing technology (such as spaCy) is used to extract key keywords from the question. For example, from the question "Please tell me how to maintain this machine," keywords such as "machine" and "maintenance method" are extracted.

[1171] Input: Prompt message

[1172] Output: Extracted keywords

[1173] Step 4:

[1174] The server searches the database based on the extracted keywords. This includes databases of past questions and answers and FAQ information. If relevant information exists, it is retrieved. For example, if detailed instructions regarding "maintenance methods" are stored in the database, that information will be retrieved as a search result.

[1175] Input: Extracted keywords

[1176] Output: Search results data

[1177] Step 5:

[1178] The server generates an appropriate answer based on the search results. If search results exist, it constructs the answer text based on that information. For example, it generates a specific answer such as, "The maintenance procedure for this machine is as follows: 1. Turn off the power 2. Open the panel 3. Clean the filter." If no search results are found, a generative AI model generates an estimated answer.

[1179] Input: Search result data

[1180] Output: Generated answer text

[1181] Step 6:

[1182] The server sends the generated response text to the user's terminal. The user's terminal displays the received response. This allows the new employee to review the displayed response and respond appropriately.

[1183] Input: Generated response text

[1184] Output: The answer displayed on the user's terminal.

[1185] Step 7:

[1186] The server records all questions and generated answers in a database. This allows for a quick response if the same question arises in the future.

[1187] Input: Question data and answer data

[1188] Output: Questions and answers recorded in the database

[1189] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1190] The system according to the present invention utilizes generative AI and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[1191] System Configuration

[1192] 1. User terminal

[1193] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[1194] 2. Server

[1195] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching databases, sentiment analysis, and answer generation. The server is equipped with a generative AI with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database that records past questions and answers.

[1196] Flow of operations

[1197] 1. Enter and submit your question.

[1198] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[1199] 2. Receiving the questionnaire data

[1200] The terminal sends the entered question data to the server. The question entered by the user is transferred to the server.

[1201] 3. Analysis of Questionnaire Data

[1202] The server receives the question data and uses generative AI to analyze the question content. It extracts key keywords and phrases from the question and understands the intent.

[1203] 4. Emotion analysis

[1204] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1205] 5. Database Search

[1206] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[1207] 6. Generating the answer

[1208] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[1209] 7. Submitting and displaying responses

[1210] The server sends the generated response data to the user's terminal.

[1211] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[1212] 8. Record of questions and answers

[1213] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1214] Specific example

[1215] For example, if new employee Sato enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[1216] 1. Enter and submit your question.

[1217] Mr. Sato (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[1218] 2. Reception and Analysis

[1219] The server receives the question and extracts the keywords "working hours" and "recording method."

[1220] 3. Emotion analysis

[1221] The server uses an emotion engine to detect the "confusion" contained in Sato's question.

[1222] 4. Database Search

[1223] The server searches the FAQ database for information on "how to record working hours."

[1224] 5. Answer generation

[1225] The system generates the following response: "Working hours are recorded using the following steps: 1. Log in to the company system 2. Access the attendance management page 3. Enter your start and end times 4. Click the save button." Sensing confusion, it also adds supplementary explanations in a gentle tone.

[1226] 6. Submit your response and display

[1227] The server sends the generated response to Mr. Sato's terminal, and the terminal displays the response.

[1228] 7. Record

[1229] The server records this question and the generated answer in the database.

[1230] In this way, by recognizing the difficulties new employees are facing and providing appropriate answers, the quality of their training can be improved.

[1231] The following describes the processing flow.

[1232] Step 1:

[1233] The user logs into their device and accesses the question input screen. The user enters the question in natural language and clicks the submit button.

[1234] Step 2:

[1235] The terminal sends the entered question data to the server. Specifically, the string entered by the user is sent to the server as a packet.

[1236] Step 3:

[1237] The server receives the question data. The server parses the received data and reads the string data.

[1238] Step 4:

[1239] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[1240] Step 5:

[1241] The server uses an emotion engine to analyze the emotions contained in the user's question. The analysis employs techniques to classify emotions from words and phrases contained in the text. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1242] Step 6:

[1243] The server searches the database based on keyword analysis results and sentiment analysis results. It searches the FAQ database and past question and answer database to find relevant information.

[1244] Step 7:

[1245] The server evaluates the search results and generates an appropriate response. If a search result is found, it creates a response based on that data. Even if no search results are found, the AI ​​generates a predicted response. Furthermore, it adjusts the response to have an appropriate tone and content based on sentiment analysis results.

[1246] Step 8:

[1247] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[1248] Step 9:

[1249] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary. For example, to alleviate any confusion the user might feel, detailed instructions in a gentle tone are displayed.

[1250] Step 10:

[1251] The server records all questions and answers in a database. This ensures that if the same question arises again in the future, a quick response will be possible.

[1252] Through the processing steps described above, this system reduces the burden on senior employees and enables new employees to quickly and consistently obtain necessary information with a high level of emotional understanding.

[1253] (Example 2)

[1254] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1255] In today's business environment, it is crucial for new graduates to adapt to their work quickly and efficiently. However, traditional FAQ systems and manuals are limited to providing static information and struggle to address users' specific questions and concerns. As a result, new graduates may not receive adequate guidance and support, which can delay their adaptation to their work. Furthermore, the lack of timely and appropriate answers to questions can lead to decreased work efficiency.

[1256] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1257] In this invention, the server includes means for performing question analysis using a generative artificial intelligence model, means for performing sentiment analysis of the question, and means for generating an answer based on the search results and sentiment analysis results. This makes it possible to quickly provide appropriate answers to the user's specific questions and emotions.

[1258] A "user terminal" is a device used by a user to access the system and enter questions, and includes personal computers, smartphones, tablets, and other similar devices.

[1259] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[1260] A "generative artificial intelligence model" refers to an artificial intelligence technology that uses natural language processing (NLP) to generate answers to questions based on the generated data.

[1261] "Question analysis" refers to the process of analyzing a user's question, extracting key keywords and phrases, and understanding their intent.

[1262] "Sentiment analysis" refers to the process of detecting the emotions contained in a user's question (e.g., anxiety, confusion, excitement) and adjusting the tone of the response based on that.

[1263] A "database" refers to a storage device that stores information that a system can refer to, such as FAQ data and past question-and-answer data.

[1264] "Answer generation" refers to the process of constructing and generating appropriate answers based on search results and sentiment analysis results.

[1265] "Recording" refers to the process of saving all questions and their answers in a database so that they can be referenced later.

[1266] The system according to the present invention utilizes a generative artificial intelligence model and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new employees. This system consists of a user terminal, a server, a generative artificial intelligence model, an emotion engine, and a database.

[1267] System Configuration

[1268] 1. User terminal

[1269] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[1270] 2. Server

[1271] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, sentiment analysis, and answer generation. The server is equipped with a generative artificial intelligence model with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database recording past questions and answers. Specifically, OpenAI's GPT-4 is used as the generative AI model, and IBM Watson's Tone Analyzer is used as the sentiment engine.

[1272] Flow of operations

[1273] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[1274] The terminal sends the entered question data to the server.

[1275] The server receives the question data and analyzes the question content using a generative artificial intelligence model. It extracts key keywords and phrases from the question and understands the intent.

[1276] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1277] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[1278] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[1279] The server sends the generated response data to the user's terminal.

[1280] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[1281] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1282] Specific example

[1283] For example, if a new employee enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[1284] The user types "Please tell me how to record my working hours" into the terminal and clicks the send button.

[1285] The server receives the question and extracts the keywords "working hours" and "recording method."

[1286] The server uses an emotion engine to detect "confusion" contained in the question.

[1287] The server searches the FAQ database for information on "how to record working hours."

[1288] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system 2. Access the attendance management page 3. Enter your start and end times 4. Click the save button," and, sensing "confusion," adds supplementary explanations in a gentle tone.

[1289] The server generates an answer and sends it to the user's device, which then displays the answer on its screen.

[1290] The server records this question and the generated answer in the database.

[1291] In this way, the quality of training can be improved by sensing when new employees are having trouble and providing appropriate answers accordingly. Examples of specific prompts include: "Please tell me how to start a new project," "Please tell me how to connect to the company Wi-Fi," and "Please tell me how to apply for paid leave."

[1292] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1293] Step 1:

[1294] The user accesses the login screen on their device, enters their ID and password, and logs into the system. After successfully logging in, they enter their specific question or inquiry in natural language in the question input field. For example, they might enter, "Please tell me how to start a new project." Then, the user clicks the "Submit" button, and the question data is sent.

[1295] Input: User ID, password, and security question.

[1296] Output: Login authentication result, submitted question data

[1297] Step 2:

[1298] The terminal sends the user-entered question data to the server. Specifically, the question data is sent to the server using a communication protocol such as an HTTP request. The entered data is then converted to JSON format.

[1299] Input: User's question data

[1300] Output: Question data in JSON format sent to the server

[1301] Step 3:

[1302] The server receives the question data and analyzes the question content using a generative artificial intelligence model. The server analyzes the received JSON data and uses a generative AI model (e.g., OpenAI's GPT-4) to extract the main keywords and phrases of the question. For example, from the question "Please tell me the procedure for starting a new project," the keywords "project," "start," and "procedure" are extracted.

[1303] Input: Question data in JSON format

[1304] Output: Extracted keywords and analysis results

[1305] Step 4:

[1306] The server uses an emotion engine (e.g., IBM Watson's Tone Analyzer) to analyze the emotions contained in the user's questions. Specifically, it detects emotions such as "anxiety," "confusion," and "excitement" based on the received question data. In this step, an emotion score is generated, and the user's emotional state is evaluated.

[1307] Input: Question data

[1308] Output: Sentiment analysis results (emotion score and emotional state)

[1309] Step 5:

[1310] The server searches the FAQ database and past question-and-answer database based on keyword and sentiment analysis results. It uses the extracted keywords (e.g., "project start procedure") as search queries to find relevant information from the database. The search is performed using appropriate SQL queries and search algorithms.

[1311] Input: Keyword analysis results, sentiment analysis results

[1312] Output: Search results (related FAQs and past questions / answers)

[1313] Step 6:

[1314] The server evaluates the search results and generates an appropriate response. It uses a generative artificial intelligence model to generate responses based on the search results. For example, if detailed instructions on "how to start a new project" are found in the search results, the response will be constructed based on that information. If no results are found, the AI ​​will generate a speculative response. The response is then adjusted to an appropriate tone, taking sentiment analysis results into consideration.

[1315] Input: Search results, sentiment analysis results

[1316] Output: Generated response data

[1317] Step 7:

[1318] The server converts the generated response data into JSON format and sends it to the user's terminal. The generated response data is then transferred to the terminal as an HTTP response.

[1319] Input: Generated response data

[1320] Output: JSON format response data sent to the terminal

[1321] Step 8:

[1322] The device analyzes the received response data and displays it to the user. The response is displayed on the device's user interface (UI). The user can review the response on the screen and re-enter the question if necessary.

[1323] Input: Response data in JSON format

[1324] Output: Answer displayed to the user

[1325] Step 9:

[1326] The server records all questions and generated answers in a database. This allows for quick response when the same question arises in the future. The data is recorded along with the question content, the answer content, and their associated metadata.

[1327] Input: Question data, generated response data

[1328] Output: Records stored in the database

[1329] (Application Example 2)

[1330] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1331] In the factory, new graduates are unfamiliar with their work and require efficient support. However, the current system is inadequate in handling questions in natural language and fails to provide appropriate support that takes emotions into account. Furthermore, the lack of real-time speech recognition and response capabilities makes it difficult to provide immediate solutions. This is leading to decreased work efficiency for new graduates and a decline in the quality of their training. In this situation, there is a need for a system that allows new graduates to perform their duties smoothly without feeling confused or anxious.

[1332] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1333] In this invention, the server includes means for receiving and analyzing a user's question, means for searching a database based on the question analysis results, and means for analyzing the emotions contained in the user's question. This allows for the analysis of the user's emotions along with the question content, enabling the generation of an answer in the optimal tone and immediate voice response.

[1334] A "user terminal" is a device used by a user to access a system and input questions. Examples include personal computers, smartphones, and tablets.

[1335] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[1336] "Question analysis" is the process by which a server uses natural language processing to extract key keywords and phrases from a question received from a user and understand the user's intent.

[1337] "Database search" refers to the process where the server searches the FAQ database and past question / answer database based on the question analysis results to find relevant information.

[1338] "Answer generation" refers to the process by which a server evaluates the search results in a database, constructs an appropriate answer, and, in some cases, generates an estimated answer.

[1339] "Sending a response" refers to the process of sending the response data generated by the server to the user's terminal.

[1340] "Displaying responses" refers to the process of displaying the responses received by the user's terminal to the user on the screen.

[1341] "Question and answer recording" refers to the process where the server saves all questions and answers in a database. This makes it possible to provide a quick answer when the same question arises in the future.

[1342] "Emotional analysis" is the process by which a server uses an emotion engine to analyze the emotions contained in a user's question and detect emotions such as "anxiety," "confusion," and "excitement."

[1343] "Speech recognition" is the process by which a robot converts a user's voice into text. This process allows the user to input questions using their voice.

[1344] "Voice output" is the process by which the robot communicates its generated response to the user via voice. Voice output allows the user to receive the response intuitively.

[1345] This invention provides a system that enables new employees to use robots within a factory to provide real-time work support. This system utilizes generative AI and an emotion engine and operates in conjunction with user terminals such as smartphones and tablets.

[1346] System Configuration

[1347] 1. User terminal

[1348] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[1349] 2. Robots

[1350] The robot is a device used by new employees within the factory and accepts voice input from users. The robot has the function of converting speech into text using speech recognition technology and outputting the generated response as voice.

[1351] 3. Server

[1352] A server is the central computing resource of a system. A server has the following functions:

[1353] Receiving and analyzing question data

[1354] Implementation of an emotion engine for performing emotion analysis

[1355] Searching Databases

[1356] Response generation and evaluation

[1357] Record of questions and answers

[1358] Flow of operations

[1359] 1. Enter your question

[1360] The user logs into the terminal, enters their question in natural language, and submits it. For example, they might enter, "Please tell me how to start a new project."

[1361] 2. Voice recognition and question submission

[1362] When a user inputs a question into the robot via voice, the robot converts the voice into text and sends it to the server.

[1363] 3. Analysis of Questionnaire Data

[1364] The server receives the question data and analyzes the question content using a generative AI model. It extracts key keywords and phrases from the question and understands the intent.

[1365] 4. Emotion analysis

[1366] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1367] 5. Database Search

[1368] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[1369] 6. Generating the answer

[1370] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs a response based on that data. If no search results are found, it generates an estimated response using a generative AI model. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[1371] 7. Submitting and displaying responses

[1372] The server sends the generated response data to the user's terminal, and the robot outputs the response verbally. The user can also check the response on the terminal's screen.

[1373] 8. Record of questions and answers

[1374] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1375] Hardware and software to use

[1376] Generative AI model: GPT-2 (using the transformers library)

[1377] Emotion Engine: Proprietary emotion analysis engine (sentiment_analysis)

[1378] User devices: PCs, smartphones, tablets

[1379] Robot: A device with voice recognition and voice output capabilities.

[1380] Specific example

[1381] For example, if a new employee types "Please tell me how to start a new project" into the terminal, the system will respond as follows:

[1382] 1. Enter and submit your question.

[1383] The user types "Please tell me how to start a new project" and submits it.

[1384] 2. Reception and Analysis

[1385] The server receives the question, extracts the keywords "project," "start," and "procedure," and understands the intent.

[1386] 3. Emotion analysis

[1387] The server uses an emotion engine to detect "confusion" contained in the user's question.

[1388] 4. Database Search

[1389] The server searches the FAQ database for information on "Project Start Procedures".

[1390] 5. Answer generation

[1391] The system generates a response stating, "The project initiation procedure is as follows: 1. Fill out the application form 2. Obtain approval 3. Select members 4. Set a schedule," and adds supplementary explanations in a gentle tone based on the sentiment analysis results.

[1392] 6. Submit your response and display

[1393] The server generates an answer, which is sent to the user's terminal, and the robot outputs it verbally.

[1394] Example of a prompt

[1395] "Please tell me how to start a new project."

[1396] "What should I do if the machine breaks down?"

[1397] "Please tell me how to record working hours."

[1398] This system provides an environment where new graduates can perform their duties smoothly without feeling confused or anxious.

[1399] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1400] Step 1:

[1401] The user logs into their device and enters their question in natural language.

[1402] Input: The user operates the terminal and enters the question in natural language (e.g., "Please tell me how to start a new project").

[1403] Output: Question text data is generated.

[1404] Step 2:

[1405] The robot receives voice input from the user, converts it to text using speech recognition technology, and sends it to the server.

[1406] Input: Voice input from the user.

[1407] Output: The audio data is converted to text data and sent to the server.

[1408] Step 3:

[1409] The server receives the question data and analyzes the question using a generative AI model.

[1410] Input: Question in text data format.

[1411] Data processing: A generative AI model (GPT-2) analyzes the question content and extracts key keywords and phrases.

[1412] Output: Analyzed keywords and phrases.

[1413] Step 4:

[1414] The server uses an emotion analysis engine to analyze the emotions contained in the user's questions.

[1415] Input: Question text.

[1416] Data processing: The emotion analysis engine analyzes the text and detects emotions such as "anxiety," "confusion," and "excitement."

[1417] Output: Detected emotion information.

[1418] Step 5:

[1419] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[1420] Input: Analyzed keywords and sentiment information.

[1421] Data Search: Searches for server-related information from the FAQ database and past question-and-answer database.

[1422] Output: Search results.

[1423] Step 6:

[1424] The server evaluates the search results and generates appropriate answers.

[1425] Input: Search results from a database.

[1426] Data processing: The server constructs an answer based on the search results and adjusts the answer to an appropriate tone according to the sentiment analysis results.

[1427] Output: Generated answer text.

[1428] Step 7:

[1429] The server generates response data and sends it to the user's terminal, and the robot outputs the response via voice.

[1430] Input: Generated response text.

[1431] Data transfer: The server sends the response data to the user's terminal.

[1432] Data output: The robot outputs the answer in voice.

[1433] Output: A voice response to the user.

[1434] Step 8:

[1435] The server records all questions and answers in a database.

[1436] Input: Question and generated answer data.

[1437] Data storage: The server stores questions and answers in a database.

[1438] Output: Database entries for quick responses to future questions.

[1439] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1440] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1441] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1442] [Fourth Embodiment]

[1443] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1444] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1445] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1446] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1447] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1448] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1449] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1450] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1451] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1452] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1453] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1454] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1455] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1456] The system according to the present invention utilizes generative AI to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[1457] System Configuration

[1458] 1. User terminal

[1459] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[1460] 2. Server

[1461] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, and generating answers. The server is equipped with a generative AI with natural language processing (NLP) capabilities, an FAQ database, and a database that records past questions and answers.

[1462] Flow of operations

[1463] 1. Enter and submit your question.

[1464] The user logs into their device and enters their question in natural language. For example, they might enter the question, "How do I start a new project?"

[1465] The terminal sends the entered question data to the server.

[1466] 2. Receiving and analyzing questions

[1467] The server receives the question data and analyzes the question content using generative AI. Natural language processing techniques are used for the analysis to extract key keywords from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[1468] 3. Database Search

[1469] Based on the analysis results, the server searches the FAQ database and the past question and answer database. For example, if "Steps to start a new project" is already registered in the FAQ database, that information will be retrieved.

[1470] 4. Generating the answer

[1471] The server generates an appropriate answer based on the search results. If a search result is found, it constructs an answer based on it; if no result is found, the AI ​​generates a predicted answer.

[1472] For example, it can generate a response like, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[1473] 5. Submitting and displaying responses

[1474] The server sends the generated response to the user's terminal.

[1475] The device displays the received responses to the user. The user can review the displayed responses and ask additional questions if necessary.

[1476] 6. Record of questions and answers

[1477] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1478] Specific example

[1479] For example, if new employee Tanaka enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[1480] 1. Enter and submit your question.

[1481] Mr. Tanaka (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[1482] 2. Reception and Analysis

[1483] The server receives the question and extracts the keywords "working hours" and "recording method."

[1484] 3. Database Search

[1485] The server searches the FAQ database for information on "how to record working hours."

[1486] 4. Answer generation

[1487] The system generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[1488] 5. Submit your response and display

[1489] The server generates an answer and sends it to Tanaka's terminal, which then displays the answer.

[1490] 6. Record

[1491] The server records this question and the generated answer in the database.

[1492] In this way, we can provide prompt and consistent answers to questions from new employees and improve the quality of their training.

[1493] The following describes the processing flow.

[1494] Step 1:

[1495] The user logs into their device and accesses the question input screen. The user enters the question in natural language and clicks the submit button.

[1496] Step 2:

[1497] The terminal sends the entered question data to the server. Specifically, the string entered by the user is sent to the server as a packet.

[1498] Step 3:

[1499] The server receives the question data. The server parses the received data and reads the string data.

[1500] Step 4:

[1501] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[1502] Step 5:

[1503] The server searches the database based on the analysis results. It refers to the FAQ database and past question and answer database to find relevant information.

[1504] Step 6:

[1505] The server evaluates the search results and generates an appropriate answer. If a search result is found, it creates an answer based on that data. If no search results are found, the AI ​​generates a predicted answer.

[1506] Step 7:

[1507] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[1508] Step 8:

[1509] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary.

[1510] Step 9:

[1511] The server records all questions and answers in a database. This ensures that if the same question arises again in the future, a quick response can be provided.

[1512] Through the processing steps described above, this system reduces the burden on senior employees and enables new graduates to quickly and consistently obtain the information they need.

[1513] (Example 1)

[1514] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1515] In the previous system, there was a problem in that it took a long time to resolve questions that new graduates had about daily tasks, attendance tracking, and how to use the company's internal systems. Furthermore, when specialized knowledge was required, the burden on the person in charge increased, which was another issue. This resulted in decreased work efficiency for new graduates and inconsistent quality of training.

[1516] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1517] In this invention, the server includes means for receiving user questions and analyzing them using natural language processing, means for searching a knowledge base based on the question analysis results, and means for generating estimated answers using a generative AI model. This makes it possible to automatically provide quick and appropriate answers to user-inputted questions, improve the work efficiency of new employees, and maintain a consistent quality of training.

[1518] A "user terminal" is a device used by a user to access the system and input questions.

[1519] A "server" is a central computing resource that receives, analyzes, searches databases, and generates answers for questions.

[1520] "Natural language processing" is a technology that analyzes data entered by users in natural language and extracts necessary keywords and information.

[1521] A "knowledge base" is a general term for databases that record FAQs and past questions and answers, and is a source of information that a system uses to generate answers to questions.

[1522] A "generative AI model" is an artificial intelligence model that generates appropriate answers based on received questions and search results; examples include GPT-3 and GPT-4.

[1523] An "estimated answer" is an answer that a generative AI model generates based on inference when no search results are found.

[1524] "Keywords" are important words or phrases extracted from the user's question using natural language processing.

[1525] "Knowledge-based search" is the process of searching for relevant information by referring to FAQ databases and past question-and-answer databases based on extracted keywords.

[1526] "Means of recording" refers to the function of saving all questions and generated answers in a database for future use.

[1527] The system according to the present invention efficiently provides support for new graduates' work, attendance tracking, instruction on how to use internal systems, and answers questions related to their work. An embodiment of this system is described below.

[1528] User terminal

[1529] User terminals are devices used by new employees to access the system and input questions in natural language. Specifically, they include personal computers, smartphones, and tablets. Users use these terminals to input questions in natural language, such as "Please tell me the procedure for starting a new project."

[1530] server

[1531] Servers are the central computing resources of a system and perform the following main functions:

[1532] 1. Receiving and analyzing questions

[1533] The server receives question data submitted by the user. The received data is then analyzed using natural language processing with Python libraries such as SpaCy and NLTK. As a result of the analysis, key keywords are extracted from the question. For example, keywords such as "new project" and "starting procedure" are identified.

[1534] 2. Knowledge base search

[1535] Based on the analysis results, the server searches the FAQ database and databases containing past questions and answers. SQL queries and NoSQL database queries are used for this search process. For example, a query like "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Starting Procedure%'" is executed.

[1536] 3. Generating answers using generative AI models

[1537] The server generates appropriate answers based on information retrieved from the database. If search results are found, the AI ​​creates an answer based on their content. If no search results are found, a generative AI model (such as GPT-3 or GPT-4) generates an appropriate answer based on inferences. Specifically, in response to the question, "Please tell me the steps to start a new project," the server generates the answer, "The steps to start a new project are as follows: 1. Fill out the project application form. 2. Obtain approval from your supervisor. 3. Form a project team. 4. Prepare the necessary materials."

[1538] 4. Submitting and viewing the generated responses

[1539] The server sends the generated response to the user's terminal. A REST API is used for transmission, and the response data is transferred in JSON format. The user's terminal parses and displays the received response. The user can review the displayed response and ask additional questions if necessary.

[1540] 5. Record of questions and answers

[1541] The server records all questions and generated answers in a database. For example, using MongoDB or MySQL, data is stored in the format "Question: 'How do I record my working hours?', Answer: 'Working hours are recorded using the following steps...'". This allows for quick response when similar questions arise in the future.

[1542] Specific example

[1543] When a new employee types "Please tell me how to record my working hours" into a terminal and sends it to the server, it will look like this:

[1544] 1. Enter and submit your question.

[1545] The user enters "Please tell me how to record my working hours" into the terminal.

[1546] 2. Reception and Analysis

[1547] The server receives the question and uses natural language processing (NLP) techniques to extract keywords such as "working hours" and "recording method."

[1548] 3. Knowledge base search

[1549] The server searches the FAQ database for information on "how to record working hours."

[1550] 4. Answer generation

[1551] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system. 2. Access the attendance management page. 3. Enter your start and end times. 4. Click the save button."

[1552] 5. Submit your response and display

[1553] The server generates an answer and sends it to the user's terminal, which then displays the answer.

[1554] 6. Record

[1555] The server records this question and the generated answer in the database.

[1556] This system allows us to provide quick and consistent answers to questions from new employees, improving work efficiency and maintaining a consistent quality of training.

[1557] Example of a prompt

[1558] "Please tell me how to start a new project."

[1559] "Please tell me how to record working hours."

[1560] "I have a question about how to use the company's internal systems."

[1561] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1562] System processing flow

[1563] Step 1:

[1564] The user logs into the device and enters the question.

[1565] Input: The user enters "Please tell me how to start a new project."

[1566] Operation: The terminal retrieves the entered question data.

[1567] Step 2:

[1568] The terminal sends the entered question data to the server.

[1569] Input: Question data "Please tell me how to start a new project."

[1570] Operation: The terminal securely sends the question data to the server using the HTTPS protocol.

[1571] Output: Send the question data to the server.

[1572] Step 3:

[1573] The server receives the question data and begins analysis using natural language processing (NLP).

[1574] Input: Question data sent from the device

[1575] Operation: The server analyzes the question content using Python libraries (e.g., SpaCy or NLTK). It extracts key keywords.

[1576] Example: Identify keywords such as "new project" and "starting procedures".

[1577] Output: Extracted keywords

[1578] Step 4:

[1579] The server searches the knowledge base based on the analysis results.

[1580] Input: Extracted keywords "New project" and "Starting procedure"

[1581] Operation: The server uses SQL queries and NoSQL databases to search the FAQ database and past question and answer databases.

[1582] For example, in the case of an SQL database, execute the query "SELECT FROM faq WHERE keywords LIKE '%New Project%' AND keywords LIKE '%Start Procedure%'".

[1583] Output: Search Results

[1584] Step 5:

[1585] The server generates an appropriate answer based on the search results. If no search results are found, a generative AI model is used to generate an estimated answer.

[1586] Input: Search Results

[1587] Operation: The server generates an answer based on the search results. If no search results are found, it estimates an answer using a generative AI model (e.g., GPT-3 or GPT-4).

[1588] Example: If there are search results, it will generate a response such as, "The steps to start a new project are as follows: 1. Fill out the project application form 2. Obtain approval from your supervisor 3. Form a project team 4. Prepare the necessary materials."

[1589] Output: Generated answer

[1590] Step 6:

[1591] The server sends the generated response to the user's terminal.

[1592] Input: Generated answer

[1593] Operation: The server sends response data in JSON format using a REST API.

[1594] Output: Send the response data to the user's terminal.

[1595] Step 7:

[1596] The device displays the received response to the user.

[1597] Input: Response data sent from the server

[1598] Operation: The device parses the JSON data and displays it on the screen. The user can then review the displayed response.

[1599] Output: Answer displayed to the user

[1600] Step 8:

[1601] The server records all questions and generated answers in a database.

[1602] Input: Question data and generated response data

[1603] Operation: The server records the questions and answers in a database (e.g., MongoDB or MySQL).

[1604] For example, record it in the format: "Question: 'Please tell me how to start a new project,' Answer: 'The steps to start a new project are as follows...'"

[1605] Output: Recorded data

[1606] (Application Example 1)

[1607] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1608] In a factory environment, a sophisticated training support system is necessary for new employees to receive appropriate guidance and quickly begin their work. However, the current system makes it difficult for new employees to get quick answers to their questions about operation methods and work procedures. Furthermore, there are concerns that the quality of training will decline because consistent answers to questions are not provided.

[1609] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1610] In this invention, the server includes means for receiving and analyzing user questions, means for searching a database based on the question analysis results, and means for generating answers based on the search results. This makes it possible to provide rapid and consistent answers using a generative AI model for new employees to answer questions from factory automation machines in a factory environment.

[1611] A "user terminal" is a device used to input questions and communicate with a server. Examples include personal computers, smartphones, and tablets.

[1612] A "server" is a central processing unit that analyzes questions received from users and generates appropriate answers. It performs tasks such as analyzing question data, searching databases, generating answers, and recording them.

[1613] A "generative AI model" is an artificial intelligence technology used to analyze a question and generate an appropriate answer. For example, natural language processing technologies such as OpenAI's GPT-3 fall into this category.

[1614] "Natural language processing" is a technology that understands questions entered by users in natural language and extracts appropriate keywords.

[1615] A "database" is an information storage system that stores past questions and answers, as well as FAQ information. The server searches this database to find answers to questions.

[1616] "Factory automation machinery" refers to machines that perform tasks in a factory automatically. Examples include robotic arms and automated transport systems.

[1617] "Means for entering questions" refers to an interface that allows users to enter questions in natural language and send them to the server.

[1618] The system according to the present invention provides a question-answering system using a generative AI model to efficiently support the training of new employees in a factory environment. This system includes a user terminal, a server, and a database.

[1619] User terminal

[1620] User terminals are devices used by new employees to input questions to factory automation machinery. Specifically, these include personal computers, tablets, and smartphones. Users input questions in natural language using these terminals, and these questions are sent to the server.

[1621] server

[1622] The server is the central computer of this system. The server receives questions submitted by users and analyzes them using a generative AI model (e.g., OpenAI GPT-3). Natural language processing techniques (e.g., spaCy) are used for the analysis to extract key keywords.

[1623] The server searches the database based on the analysis results. The database contains past questions and answers, as well as FAQ information. If search results exist, it generates an answer based on that information. Even if no search results are found, the AI ​​model automatically generates an estimated answer.

[1624] The generated responses are sent from the server to the user's terminal. The user's terminal displays these responses to the user, providing consistent educational support.

[1625] database

[1626] A database is a system that stores information for the server to perform searches. This includes past questions and answers, and FAQ information. The server uses this database to provide the most appropriate answer to the user's question.

[1627] Specific example

[1628] For example, if a new employee asks a factory automation machine, "How do I maintain this machine?", the process would proceed as follows: The question is entered into the user terminal and sent to the server. The server receives the question and extracts the keywords "machine" and "maintenance method". It searches the database to find information about maintenance procedures and generates an answer, such as "The maintenance method for this machine is as follows: 1. Turn off the power. 2. Open the panel. 3. Clean the filter." The generated answer is sent to the user terminal and displayed.

[1629] Example of a prompt

[1630] The prompt message when a new employee enters the question, "Please tell me how to operate this machine," is as follows:

[1631] New employee: Please explain how to operate this machine.

[1632] Robot AI:

[1633] In this way, the system of the present invention can respond quickly and accurately to questions from new employees and improve the quality of training.

[1634] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1635] Step 1:

[1636] The user enters a question in natural language on their device. The entered question is a specific text, such as "Please tell me how to maintain this machine." The user's device then sends this text data to the server.

[1637] Input: Question text in natural language

[1638] Output: Question data sent to the server

[1639] Step 2:

[1640] The server receives question data sent from the user's terminal. The server converts this data into a format for input into a generative AI model (OpenAI GPT-3). Specifically, it formats the question text into a prompt.

[1641] Input: Question data from the user terminal

[1642] Output: Prompt text input to the generative AI model

[1643] Step 3:

[1644] The server uses a generative AI model to analyze the content of the received question. In this process, natural language processing technology (such as spaCy) is used to extract key keywords from the question. For example, from the question "Please tell me how to maintain this machine," keywords such as "machine" and "maintenance method" are extracted.

[1645] Input: Prompt message

[1646] Output: Extracted keywords

[1647] Step 4:

[1648] The server searches the database based on the extracted keywords. This includes databases of past questions and answers and FAQ information. If relevant information exists, it is retrieved. For example, if detailed instructions regarding "maintenance methods" are stored in the database, that information will be retrieved as a search result.

[1649] Input: Extracted keywords

[1650] Output: Search results data

[1651] Step 5:

[1652] The server generates an appropriate answer based on the search results. If search results exist, it constructs the answer text based on that information. For example, it generates a specific answer such as, "The maintenance procedure for this machine is as follows: 1. Turn off the power 2. Open the panel 3. Clean the filter." If no search results are found, a generative AI model generates an estimated answer.

[1653] Input: Search result data

[1654] Output: Generated answer text

[1655] Step 6:

[1656] The server sends the generated response text to the user's terminal. The user's terminal displays the received response. This allows the new employee to review the displayed response and respond appropriately.

[1657] Input: Generated response text

[1658] Output: The answer displayed on the user's terminal.

[1659] Step 7:

[1660] The server records all questions and generated answers in a database. This allows for a quick response if the same question arises in the future.

[1661] Input: Question data and answer data

[1662] Output: Questions and answers recorded in the database

[1663] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1664] The system according to the present invention utilizes generative AI and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new graduates. A specific embodiment of this system is described below in natural language.

[1665] System Configuration

[1666] 1. User terminal

[1667] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[1668] 2. Server

[1669] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching databases, sentiment analysis, and answer generation. The server is equipped with a generative AI with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database that records past questions and answers.

[1670] Flow of operations

[1671] 1. Enter and submit your question.

[1672] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[1673] 2. Receiving the questionnaire data

[1674] The terminal sends the entered question data to the server. The question entered by the user is transferred to the server.

[1675] 3. Analysis of Questionnaire Data

[1676] The server receives the question data and uses generative AI to analyze the question content. It extracts key keywords and phrases from the question and understands the intent.

[1677] 4. Emotion analysis

[1678] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1679] 5. Database Search

[1680] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[1681] 6. Generating the answer

[1682] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[1683] 7. Submitting and displaying responses

[1684] The server sends the generated response data to the user's terminal.

[1685] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[1686] 8. Record of questions and answers

[1687] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1688] Specific example

[1689] For example, if new employee Sato enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[1690] 1. Enter and submit your question.

[1691] Mr. Sato (the user) enters "Please tell me how to record my working hours" into the terminal and sends it to the server.

[1692] 2. Reception and Analysis

[1693] The server receives the question and extracts the keywords "working hours" and "recording method."

[1694] 3. Emotion analysis

[1695] The server uses an emotion engine to detect the "confusion" contained in Sato's question.

[1696] 4. Database Search

[1697] The server searches the FAQ database for information on "how to record working hours."

[1698] 5. Answer generation

[1699] The system generates the following response: "Working hours are recorded using the following steps: 1. Log in to the company system 2. Access the attendance management page 3. Enter your start and end times 4. Click the save button." Sensing confusion, it also adds supplementary explanations in a gentle tone.

[1700] 6. Submit your response and display

[1701] The server sends the generated response to Mr. Sato's terminal, and the terminal displays the response.

[1702] 7. Record

[1703] The server records this question and the generated answer in the database.

[1704] In this way, by recognizing the difficulties new employees are facing and providing appropriate answers, the quality of their training can be improved.

[1705] The following describes the processing flow.

[1706] Step 1:

[1707] The user logs into their device and accesses the question input screen. The user enters the question in natural language and clicks the submit button.

[1708] Step 2:

[1709] The terminal sends the entered question data to the server. Specifically, the string entered by the user is sent to the server as a packet.

[1710] Step 3:

[1711] The server receives the question data. The server parses the received data and reads the string data.

[1712] Step 4:

[1713] The server passes the received data to a generative AI, which uses natural language processing (NLP) techniques to analyze the question. Specifically, it extracts key keywords and phrases to understand the intent of the question.

[1714] Step 5:

[1715] The server uses an emotion engine to analyze the emotions contained in the user's question. The analysis employs techniques to classify emotions from words and phrases contained in the text. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1716] Step 6:

[1717] The server searches the database based on keyword analysis results and sentiment analysis results. It searches the FAQ database and past question and answer database to find relevant information.

[1718] Step 7:

[1719] The server evaluates the search results and generates an appropriate response. If a search result is found, it creates a response based on that data. Even if no search results are found, the AI ​​generates a predicted response. Furthermore, it adjusts the response to have an appropriate tone and content based on sentiment analysis results.

[1720] Step 8:

[1721] The server sends the generated response data to the terminal. Specifically, the created response is sent to the user's terminal as a packet.

[1722] Step 9:

[1723] The device displays the received response data to the user. The user can review the response on the device screen and re-enter the question if necessary. For example, to alleviate any confusion the user might feel, detailed instructions in a gentle tone are displayed.

[1724] Step 10:

[1725] The server records all questions and answers in a database. This ensures that if the same question arises again in the future, a quick response will be possible.

[1726] Through the processing steps described above, this system reduces the burden on senior employees and enables new employees to quickly and consistently obtain necessary information with a high level of emotional understanding.

[1727] (Example 2)

[1728] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1729] In today's business environment, it is crucial for new graduates to adapt to their work quickly and efficiently. However, traditional FAQ systems and manuals are limited to providing static information and struggle to address users' specific questions and concerns. As a result, new graduates may not receive adequate guidance and support, which can delay their adaptation to their work. Furthermore, the lack of timely and appropriate answers to questions can lead to decreased work efficiency.

[1730] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1731] In this invention, the server includes means for performing question analysis using a generative artificial intelligence model, means for performing sentiment analysis of the question, and means for generating an answer based on the search results and sentiment analysis results. This makes it possible to quickly provide appropriate answers to the user's specific questions and emotions.

[1732] A "user terminal" is a device used by a user to access the system and enter questions, and includes personal computers, smartphones, tablets, and other similar devices.

[1733] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[1734] A "generative artificial intelligence model" refers to an artificial intelligence technology that uses natural language processing (NLP) to generate answers to questions based on the generated data.

[1735] "Question analysis" refers to the process of analyzing a user's question, extracting key keywords and phrases, and understanding their intent.

[1736] "Sentiment analysis" refers to the process of detecting the emotions contained in a user's question (e.g., anxiety, confusion, excitement) and adjusting the tone of the response based on that.

[1737] A "database" refers to a storage device that stores information that a system can refer to, such as FAQ data and past question-and-answer data.

[1738] "Answer generation" refers to the process of constructing and generating appropriate answers based on search results and sentiment analysis results.

[1739] "Recording" refers to the process of saving all questions and their answers in a database so that they can be referenced later.

[1740] The system according to the present invention utilizes a generative artificial intelligence model and an emotion engine to efficiently provide work support, attendance tracking, instruction on how to use internal systems, and answer work-related questions for new employees. This system consists of a user terminal, a server, a generative artificial intelligence model, an emotion engine, and a database.

[1741] System Configuration

[1742] 1. User terminal

[1743] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[1744] 2. Server

[1745] The server is the central computing resource of the system, handling tasks such as receiving and analyzing question data, searching the database, sentiment analysis, and answer generation. The server is equipped with a generative artificial intelligence model with natural language processing (NLP) capabilities, a sentiment engine, an FAQ database, and a database recording past questions and answers. Specifically, OpenAI's GPT-4 is used as the generative AI model, and IBM Watson's Tone Analyzer is used as the sentiment engine.

[1746] Flow of operations

[1747] The user logs into their device, enters their question in natural language, and clicks the submit button. For example, they might enter the question, "Please tell me how to start a new project."

[1748] The terminal sends the entered question data to the server.

[1749] The server receives the question data and analyzes the question content using a generative artificial intelligence model. It extracts key keywords and phrases from the question and understands the intent.

[1750] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1751] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results. The search finds relevant information.

[1752] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs the response based on that data. If no search result is found, the AI ​​generates a predicted response. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[1753] The server sends the generated response data to the user's terminal.

[1754] The device displays the received response to the user. The user can review the response on the device screen and re-enter the question if necessary.

[1755] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1756] Specific example

[1757] For example, if a new employee enters "Please tell me how to record my working hours" into the terminal, it will look like this:

[1758] The user types "Please tell me how to record my working hours" into the terminal and clicks the send button.

[1759] The server receives the question and extracts the keywords "working hours" and "recording method."

[1760] The server uses an emotion engine to detect "confusion" contained in the question.

[1761] The server searches the FAQ database for information on "how to record working hours."

[1762] The server generates the following response: "Work hours are recorded using the following steps: 1. Log in to the company system 2. Access the attendance management page 3. Enter your start and end times 4. Click the save button," and, sensing "confusion," adds supplementary explanations in a gentle tone.

[1763] The server generates an answer and sends it to the user's device, which then displays the answer on its screen.

[1764] The server records this question and the generated answer in the database.

[1765] In this way, the quality of training can be improved by sensing when new employees are having trouble and providing appropriate answers accordingly. Examples of specific prompts include: "Please tell me how to start a new project," "Please tell me how to connect to the company Wi-Fi," and "Please tell me how to apply for paid leave."

[1766] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1767] Step 1:

[1768] The user accesses the login screen on their device, enters their ID and password, and logs into the system. After successfully logging in, they enter their specific question or inquiry in natural language in the question input field. For example, they might enter, "Please tell me how to start a new project." Then, the user clicks the "Submit" button, and the question data is sent.

[1769] Input: User ID, password, and security question.

[1770] Output: Login authentication result, submitted question data

[1771] Step 2:

[1772] The terminal sends the user-entered question data to the server. Specifically, the question data is sent to the server using a communication protocol such as an HTTP request. The entered data is then converted to JSON format.

[1773] Input: User's question data

[1774] Output: Question data in JSON format sent to the server

[1775] Step 3:

[1776] The server receives the question data and analyzes the question content using a generative artificial intelligence model. The server analyzes the received JSON data and uses a generative AI model (e.g., OpenAI's GPT-4) to extract the main keywords and phrases of the question. For example, from the question "Please tell me the procedure for starting a new project," the keywords "project," "start," and "procedure" are extracted.

[1777] Input: Question data in JSON format

[1778] Output: Extracted keywords and analysis results

[1779] Step 4:

[1780] The server uses an emotion engine (e.g., IBM Watson's Tone Analyzer) to analyze the emotions contained in the user's questions. Specifically, it detects emotions such as "anxiety," "confusion," and "excitement" based on the received question data. In this step, an emotion score is generated, and the user's emotional state is evaluated.

[1781] Input: Question data

[1782] Output: Sentiment analysis results (emotion score and emotional state)

[1783] Step 5:

[1784] The server searches the FAQ database and past question-and-answer database based on keyword and sentiment analysis results. It uses the extracted keywords (e.g., "project start procedure") as search queries to find relevant information from the database. The search is performed using appropriate SQL queries and search algorithms.

[1785] Input: Keyword analysis results, sentiment analysis results

[1786] Output: Search results (related FAQs and past questions / answers)

[1787] Step 6:

[1788] The server evaluates the search results and generates an appropriate response. It uses a generative artificial intelligence model to generate responses based on the search results. For example, if detailed instructions on "how to start a new project" are found in the search results, the response will be constructed based on that information. If no results are found, the AI ​​will generate a speculative response. The response is then adjusted to an appropriate tone, taking sentiment analysis results into consideration.

[1789] Input: Search results, sentiment analysis results

[1790] Output: Generated response data

[1791] Step 7:

[1792] The server converts the generated response data into JSON format and sends it to the user's terminal. The generated response data is then transferred to the terminal as an HTTP response.

[1793] Input: Generated response data

[1794] Output: JSON format response data sent to the terminal

[1795] Step 8:

[1796] The device analyzes the received response data and displays it to the user. The response is displayed on the device's user interface (UI). The user can review the response on the screen and re-enter the question if necessary.

[1797] Input: Response data in JSON format

[1798] Output: Answer displayed to the user

[1799] Step 9:

[1800] The server records all questions and generated answers in a database. This allows for quick response when the same question arises in the future. The data is recorded along with the question content, the answer content, and their associated metadata.

[1801] Input: Question data, generated response data

[1802] Output: Records stored in the database

[1803] (Application Example 2)

[1804] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1805] In the factory, new graduates are unfamiliar with their work and require efficient support. However, the current system is inadequate in handling questions in natural language and fails to provide appropriate support that takes emotions into account. Furthermore, the lack of real-time speech recognition and response capabilities makes it difficult to provide immediate solutions. This is leading to decreased work efficiency for new graduates and a decline in the quality of their training. In this situation, there is a need for a system that allows new graduates to perform their duties smoothly without feeling confused or anxious.

[1806] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[1807] In this invention, the server includes means for receiving and analyzing a user's question, means for searching a database based on the question analysis results, and means for analyzing the emotions contained in the user's question. This allows for the analysis of the user's emotions along with the question content, enabling the generation of an answer in the optimal tone and immediate voice response.

[1808] A "user terminal" is a device used by a user to access a system and input questions. Examples include personal computers, smartphones, and tablets.

[1809] A "server" is the central computing resource of a system, and is a device that receives and analyzes question data, searches databases, performs sentiment analysis, and generates answers.

[1810] "Question analysis" is the process by which a server uses natural language processing to extract key keywords and phrases from a question received from a user and understand the user's intent.

[1811] "Database search" refers to the process where the server searches the FAQ database and past question / answer database based on the question analysis results to find relevant information.

[1812] "Answer generation" refers to the process by which a server evaluates the search results in a database, constructs an appropriate answer, and, in some cases, generates an estimated answer.

[1813] "Sending a response" refers to the process of sending the response data generated by the server to the user's terminal.

[1814] "Displaying responses" refers to the process of displaying the responses received by the user's terminal to the user on the screen.

[1815] "Question and answer recording" refers to the process where the server saves all questions and answers in a database. This makes it possible to provide a quick answer when the same question arises in the future.

[1816] "Emotional analysis" is the process by which a server uses an emotion engine to analyze the emotions contained in a user's question and detect emotions such as "anxiety," "confusion," and "excitement."

[1817] "Speech recognition" is the process by which a robot converts a user's voice into text. This process allows the user to input questions using their voice.

[1818] "Voice output" is the process by which the robot communicates its generated response to the user via voice. Voice output allows the user to receive the response intuitively.

[1819] This invention provides a system that enables new employees to use robots within a factory to provide real-time work support. This system utilizes generative AI and an emotion engine and operates in conjunction with user terminals such as smartphones and tablets.

[1820] System Configuration

[1821] 1. User terminal

[1822] User terminals are devices used by new employees to access the system and input questions. Examples include personal computers, smartphones, and tablets. Users can input questions using natural language via these devices.

[1823] 2. Robots

[1824] The robot is a device used by new employees within the factory and accepts voice input from users. The robot has the function of converting speech into text using speech recognition technology and outputting the generated response as voice.

[1825] 3. Server

[1826] A server is the central computing resource of a system. A server has the following functions:

[1827] Receiving and analyzing question data

[1828] Implementation of an emotion engine for performing emotion analysis

[1829] Searching Databases

[1830] Response generation and evaluation

[1831] Record of questions and answers

[1832] Flow of operations

[1833] 1. Enter your question

[1834] The user logs into the terminal, enters their question in natural language, and submits it. For example, they might enter, "Please tell me how to start a new project."

[1835] 2. Voice recognition and question submission

[1836] When a user inputs a question into the robot via voice, the robot converts the voice into text and sends it to the server.

[1837] 3. Analysis of Questionnaire Data

[1838] The server receives the question data and analyzes the question content using a generative AI model. It extracts key keywords and phrases from the question and understands the intent.

[1839] 4. Emotion analysis

[1840] The server uses an emotion engine to analyze the emotions contained in the user's questions. For example, emotions such as "anxiety," "confusion," and "excitement" may be detected.

[1841] 5. Database Search

[1842] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[1843] 6. Generating the answer

[1844] The server evaluates the search results and generates an appropriate response. If a search result is found, it constructs a response based on that data. If no search results are found, it generates an estimated response using a generative AI model. Based on sentiment analysis results, it also adjusts the response to have an appropriate tone and content.

[1845] 7. Submitting and displaying responses

[1846] The server sends the generated response data to the user's terminal, and the robot outputs the response verbally. The user can also check the response on the terminal's screen.

[1847] 8. Record of questions and answers

[1848] The server records all questions and answers in a database. This allows for quick answers to similar questions that arise in the future.

[1849] Hardware and software to use

[1850] Generative AI model: GPT-2 (using the transformers library)

[1851] Emotion Engine: Proprietary emotion analysis engine (sentiment_analysis)

[1852] User devices: PCs, smartphones, tablets

[1853] Robot: A device with voice recognition and voice output capabilities.

[1854] Specific example

[1855] For example, if a new employee types "Please tell me how to start a new project" into the terminal, the system will respond as follows:

[1856] 1. Enter and submit your question.

[1857] The user types "Please tell me how to start a new project" and submits it.

[1858] 2. Reception and Analysis

[1859] The server receives the question, extracts the keywords "project," "start," and "procedure," and understands the intent.

[1860] 3. Emotion analysis

[1861] The server uses an emotion engine to detect "confusion" contained in the user's question.

[1862] 4. Database Search

[1863] The server searches the FAQ database for information on "Project Start Procedures".

[1864] 5. Answer generation

[1865] The system generates a response stating, "The project initiation procedure is as follows: 1. Fill out the application form 2. Obtain approval 3. Select members 4. Set a schedule," and adds supplementary explanations in a gentle tone based on the sentiment analysis results.

[1866] 6. Submit your response and display

[1867] The server generates an answer, which is sent to the user's terminal, and the robot outputs it verbally.

[1868] Example of a prompt

[1869] "Please tell me how to start a new project."

[1870] "What should I do if the machine breaks down?"

[1871] "Please tell me how to record working hours."

[1872] This system provides an environment where new graduates can perform their duties smoothly without feeling confused or anxious.

[1873] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1874] Step 1:

[1875] The user logs into their device and enters their question in natural language.

[1876] Input: The user operates the terminal and enters the question in natural language (e.g., "Please tell me how to start a new project").

[1877] Output: Question text data is generated.

[1878] Step 2:

[1879] The robot receives voice input from the user, converts it to text using speech recognition technology, and sends it to the server.

[1880] Input: Voice input from the user.

[1881] Output: The audio data is converted to text data and sent to the server.

[1882] Step 3:

[1883] The server receives the question data and analyzes the question using a generative AI model.

[1884] Input: Question in text data format.

[1885] Data processing: A generative AI model (GPT-2) analyzes the question content and extracts key keywords and phrases.

[1886] Output: Analyzed keywords and phrases.

[1887] Step 4:

[1888] The server uses an emotion analysis engine to analyze the emotions contained in the user's questions.

[1889] Input: Question text.

[1890] Data processing: The emotion analysis engine analyzes the text and detects emotions such as "anxiety," "confusion," and "excitement."

[1891] Output: Detected emotion information.

[1892] Step 5:

[1893] The server searches the FAQ database and past question-and-answer database based on keyword analysis and sentiment analysis results.

[1894] Input: Analyzed keywords and sentiment information.

[1895] Data Search: Searches for server-related information from the FAQ database and past question-and-answer database.

[1896] Output: Search results.

[1897] Step 6:

[1898] The server evaluates the search results and generates appropriate answers.

[1899] Input: Search results from a database.

[1900] Data processing: The server constructs an answer based on the search results and adjusts the answer to an appropriate tone according to the sentiment analysis results.

[1901] Output: Generated answer text.

[1902] Step 7:

[1903] The server generates response data and sends it to the user's terminal, and the robot outputs the response via voice.

[1904] Input: Generated response text.

[1905] Data transfer: The server sends the response data to the user's terminal.

[1906] Data output: The robot outputs the answer in voice.

[1907] Output: A voice response to the user.

[1908] Step 8:

[1909] The server records all questions and answers in a database.

[1910] Input: Question and generated answer data.

[1911] Data storage: The server stores questions and answers in a database.

[1912] Output: Database entries for quick responses to future questions.

[1913] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1914] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1915] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1916] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1917] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1918] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1919] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1920] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1921] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1922] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1923] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1924] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1925] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1926] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1927] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1928] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1929] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1930] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1931] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1932] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1933] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1934] The following is further disclosed regarding the embodiments described above.

[1935] (Claim 1)

[1936] A means of entering a question on the user terminal,

[1937] The server has a means for receiving and analyzing user questions,

[1938] On the server, there is a means for searching the database based on the results of the query analysis,

[1939] A server provides a means for generating answers based on search results,

[1940] The server provides a means for sending the generated response to the user's terminal,

[1941] A means for displaying the submitted response on the user terminal,

[1942] The server has a means of recording all questions and answers,

[1943] A system that includes this.

[1944] (Claim 2)

[1945] The system according to claim 1, wherein the means for analyzing user questions is a means for extracting keywords using natural language processing.

[1946] (Claim 3)

[1947] The system according to claim 1, wherein the means for generating an answer is a means for generating an estimated answer when no search results are found.

[1948] "Example 1"

[1949] (Claim 1)

[1950] A means of entering a question on the user terminal,

[1951] On the server, a means for receiving user questions and analyzing them using natural language processing,

[1952] On the server, a means for searching a knowledge base based on the question analysis results,

[1953] A server provides a means for generating answers based on search results,

[1954] A server provides a means for generating an estimated answer using a generative AI model,

[1955] The server provides a means for sending the generated response to the user's terminal,

[1956] A means for displaying the submitted response on the user terminal,

[1957] The server has a means of recording all questions and answers,

[1958] A system that includes this.

[1959] (Claim 2)

[1960] The system according to claim 1, comprising means for analyzing user questions using means for extracting keywords by natural language processing.

[1961] (Claim 3)

[1962] The system according to claim 1, wherein the means for generating an answer is a means for generating an estimated answer using a generative AI model when no search results are found.

[1963] "Application Example 1"

[1964] (Claim 1)

[1965] A means of entering a question on the user terminal,

[1966] The server has a means for receiving and analyzing user questions,

[1967] On the server, there is a means for searching the database based on the results of the query analysis,

[1968] A server provides a means for generating answers based on search results,

[1969] The server provides a means for sending the generated response to the user's terminal,

[1970] A means for displaying the submitted response on the user terminal,

[1971] The server has a means of recording all questions and answers,

[1972] To provide training support for new employees in a factory, a method is developed to receive questions from factory automation machines and generate answers using a generative AI model.

[1973] A system that includes this.

[1974] (Claim 2)

[1975] The system according to claim 1, wherein the means for analyzing user questions is a means for extracting keywords using natural language processing.

[1976] (Claim 3)

[1977] The system according to claim 1, wherein the means for generating an answer is a means for generating an estimated answer when no search results are found.

[1978] "Example 2 of combining an emotion engine"

[1979] (Claim 1)

[1980] A means of entering a question on the user terminal,

[1981] The server has a means for receiving and analyzing user questions,

[1982] On the server, there is a means for searching the database based on the results of the query analysis,

[1983] A means for performing question analysis using a generative artificial intelligence model on the server,

[1984] A means for performing sentiment analysis of questions on the server,

[1985] The server includes means for generating responses based on search results and sentiment analysis results,

[1986] The server provides a means for sending the generated response to the user's terminal,

[1987] A means for displaying the submitted response on the user terminal,

[1988] The server has a means of recording all questions and answers,

[1989] A system that includes this.

[1990] (Claim 2)

[1991] The system according to claim 1, wherein the means for analyzing user questions is a means for extracting keywords using natural language processing.

[1992] (Claim 3)

[1993] The system according to claim 1, wherein the means for generating an answer is a means for generating an estimated answer when no search results are found.

[1994] "Application example 2 when combining with an emotional engine"

[1995] (Claim 1)

[1996] A means of entering a question on the user terminal,

[1997] The server has a means for receiving and analyzing user questions,

[1998] On the server, there is a means for searching the database based on the results of the query analysis,

[1999] A server provides a means for generating answers based on search results,

[2000] The server provides a means for sending the generated response to the user's terminal,

[2001] A means for displaying the submitted response on the user terminal,

[2002] The server has a means of recording all questions and answers,

[2003] On the server, a means for analyzing the emotions contained in the user's question,

[2004] In robots, a means of converting user speech into text,

[2005] In a robot, a means for outputting the generated answer as voice,

[2006] A system that includes this.

[2007] (Claim 2)

[2008] The system according to claim 1, wherein the means for analyzing user questions is a means for extracting keywords using natural language processing.

[2009] (Claim 3)

[2010] The system according to claim 1, wherein the means for generating an answer is a means for generating an estimated answer when no search results are found. [Explanation of Symbols]

[2011] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of entering a question on the user terminal, The server has a means for receiving and analyzing user questions, On the server, there is a means for searching the database based on the results of the query analysis, A server provides a means for generating answers based on search results, The server provides a means for sending the generated response to the user's terminal, A means for displaying the submitted response on the user terminal, The server has a means of recording all questions and answers, A system that includes this.

2. The system according to claim 1, wherein the means for analyzing user questions is a means for extracting keywords using natural language processing.

3. The system according to claim 1, wherein the means for generating an answer is a means for generating an estimated answer when no search results are found.

Images