system

Abstract

Provide a system. [Solution] means for receiving question data from a user terminal; A means for transmitting the received question data to a generative artificial intelligence; A generative AI analyzes the content of the question and understands its intent. a means of retrieving relevant technical documentation; means for extracting the most relevant information from the search results and generating a response sentence; means for transmitting the generated response sentence to a user terminal; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional search systems for technical documents and manuals have the problem of being unable to quickly and accurately provide users with the specific operating procedures and troubleshooting information they require. Furthermore, these systems lack the ability to fully analyze users' questions and provide direct answers. As a result, users must manually search for the information they need, which requires time and effort. The purpose of this invention is to solve these problems and provide a system that allows users to efficiently obtain the information they need. [Means for solving the problem]

[0005] The present invention provides a system including means for receiving question data from a user's terminal, means for transmitting the received question data to a generative artificial intelligence, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, and means for transmitting the generated response sentence to the user's terminal. This system is able to understand the user's intent with high accuracy by using natural language processing when the generative artificial intelligence analyzes the question content. In addition, in the case of a question about a malfunction, the system also includes a function for collecting related log data, analyzing the cause of the malfunction, and generating a response sentence with a solution. This allows the user to quickly and accurately obtain the necessary operating instructions and solution information.

[0006] A "user" is a person who utilizes the system to input and submit questions about specific features or methods of operation.

[0007] A "terminal" is a computing device that a user uses to enter and send a question to a server.

[0008] "Question data" refers to information including the content of a question entered by a user regarding a specific function or operation method.

[0009] A "server" is a computer system that receives question data sent from a user's terminal and sends the question content to the generative artificial intelligence.

[0010] "Generative AI" is a machine learning model that uses natural language processing technology to analyze questions, search for relevant information, and respond to users.

[0011] "Natural language processing" is a technology used by generative artificial intelligence to understand and generate natural language when analyzing question content.

[0012] "Technical documentation" means documents or manuals that explain the functionality and operation of a particular software or system.

[0013] A "response sentence" is a response sentence that is created by the generative artificial intelligence based on the analysis results and provided to the user.

[0014] "Log data" refers to data that records system usage history and error information.

[0015] A "failure" is a problem that prevents a system or application from functioning properly. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[0038] overview

[0039] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data into JSON format and sends it to the server as an HTTPS request. The server receives the question data and passes it to a generative AI. The generative AI analyzes the question and searches for related technical documents. It then extracts the most relevant information from the search results, generates a response, and sends it to the user's device.

[0040] Enter and submit your question

[0041] User

[0042] A user fills out a web form with a question about a specific feature or procedure (e.g., "How do I create a chart in Excel?") and clicks the submit button.

[0043] Terminal

[0044] The user's device converts the entered question into JSON format and sends it to the server as an HTTPS request, for example, in the following format:

[0045] {

[0046] "question": "How do I create a graph in Excel?"

[0047] }

[0048] Receiving and parsing questions

[0049] server

[0050] The server receives question data sent from the user's device. The received question data is then sent to the generative AI. The generative AI analyzes the question using natural language processing and extracts keywords. For example, keywords such as "Excel," "graph," "create," and "method" are extracted.

[0051] Related document search and information extraction

[0052] server

[0053] The server runs a database query to find relevant technical documents, generates a list of hit documents, and a generative AI evaluates this list, extracts the most relevant information, and generates a response.

[0054] Generate and send a response

[0055] Generative Artificial Intelligence

[0056] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0057] server

[0058] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[0059] {

[0060] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[0061] }

[0062] Terminal

[0063] The user's terminal receives the response from the server and displays it on the screen.

[0064] Questions about disabilities and solutions

[0065] User

[0066] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0067] Terminal

[0068] The user's terminal transmits the question data to the server.

[0069] server

[0070] The server receives the question and passes it to the generative AI, which analyzes the question and collects relevant log data. For example, it analyzes system logs and error logs to identify the cause of the problem, which is "insufficient memory."

[0071] Generative Artificial Intelligence

[0072] AI analyzes the cause of the failure and countermeasures based on the log data, and generates a response message with the countermeasures. For example, the response message generated might read, "The cause of the application crash is likely a lack of memory. To resolve this, close any unnecessary applications or consider increasing the memory."

[0073] server

[0074] The generated response sentence is sent to the user's terminal.

[0075] Terminal

[0076] The user's terminal displays the response.

[0077] In this way, the system of the present invention is designed to enable users to efficiently obtain the information they need, and provides specific operating procedures and troubleshooting tips quickly and accurately.

[0078] The processing flow will be explained below.

[0079] Step 1:

[0080] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[0081] Step 2:

[0082] The device converts the question data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[0083] {

[0084] "question": "How do I create a graph in Excel?"

[0085] }

[0086] Step 3:

[0087] The server receives the question data sent from the user's terminal.

[0088] Step 4:

[0089] The server passes the received question data to the generative artificial intelligence.

[0090] Step 5:

[0091] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[0092] Step 6:

[0093] The server queries the database to find relevant technical documents, and generates a list of documents that are hit by the search.

[0094] Step 7:

[0095] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[0096] Step 8:

[0097] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0098] Step 9:

[0099] The server converts the generated response into JSON format and sends it to the user's device. For example, it will be sent in the following format:

[0100] {

[0101] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and select the chart type you want from the 'Charts' group."

[0102] }

[0103] Step 10:

[0104] The user's terminal receives the response from the server and displays it on the screen.

[0105] Step 11:

[0106] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0107] Step 12:

[0108] The device converts the question data into JSON format and sends it to the server.

[0109] Step 13:

[0110] The server receives question data about the problem sent from the user's terminal.

[0111] Step 14:

[0112] The server passes the received question data to the generative artificial intelligence.

[0113] Step 15:

[0114] Generative AI uses natural language processing to analyze the question and collect relevant log data, such as analyzing patterns in system logs and error logs.

[0115] Step 16:

[0116] Based on the log data collected by the generative AI, the cause of the failure is analyzed and a likely solution is generated. For example, the cause of the failure can be identified as "insufficient memory."

[0117] Step 17:

[0118] The generative AI generates a response with a solution, such as, "The application crash may be caused by insufficient memory. To solve this problem, close any unnecessary applications or consider increasing the memory."

[0119] Step 18:

[0120] The server sends the generated response to the user's terminal.

[0121] Step 19:

[0122] The user's terminal receives the response from the server and displays it on the screen.

[0123] The specific processing steps of the entire system have been explained above. At each step, the aim is to provide information to the user efficiently and accurately.

[0124] Example 1

[0125] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] Conventional information search systems have the problem that it is difficult for users to quickly obtain appropriate information on specific operation methods or troubleshooting. It is particularly difficult to provide appropriate information immediately for questions that require specialized knowledge. Furthermore, when a malfunction occurs, it is not possible to identify the cause and provide appropriate troubleshooting. This often results in a waste of time and effort on the part of the user.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0128] In this invention, the server includes a means for receiving question data from a user's device, a means for transmitting the received question data to a generative artificial intelligence, and a means for the generative artificial intelligence to analyze the question content and understand the intent of the question. This enables users to quickly obtain high-quality information. Furthermore, the server includes a means for searching for related technical documents, a means for extracting the most relevant information from the search results and generating a response, a means for transmitting the generated response to the user's device, a means for converting the question into JSON format on the user's device and transmitting it to the server, and a means for displaying the received response on the user's device. This enables users to quickly and accurately resolve their questions and improve convenience. Furthermore, when the system receives a question about a failure, the server includes a means for collecting related log data, analyzing the cause of the failure, and generating a response that addresses the failure. This enables quick identification of the cause and presentation of a solution when a failure occurs, improving system reliability and user satisfaction.

[0129] A "user" is someone who uses the system to enter questions and obtain information.

[0130] A "terminal" is an electronic device, such as a computer or smartphone, that a user uses to input a question.

[0131] "Question data" refers to data including the question entered by the user via the terminal and sent to the server.

[0132] A "server" is a device that processes question data received from a user's terminal and passes it on to the generative artificial intelligence.

[0133] "Generative AI" is an AI model that analyzes received question data, understands the intent of the question, and generates appropriate information.

[0134] "Natural language processing" is a technology used by generative artificial intelligence to understand and process text data when analyzing question content.

[0135] "Technical documentation" means documentation that contains information about a particular technology.

[0136] A "response sentence" is an answer to a user's question generated by generative artificial intelligence.

[0137] "Log data" refers to data that records the system's operating status and error information.

[0138] A "failure" is an error or problem that prevents the system from functioning properly.

[0139] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[0140] Hardware and software configuration

[0141] User

[0142] Users use devices such as personal computers, tablets, and smartphones. These devices are connected to the Internet and display a web form for submitting questions through a web browser. Users then enter their questions and click the submit button.

[0143] Terminal

[0144] The device receives questions entered by users, converts the received questions into JSON format, and sends them to the server as an HTTPS request. The device operates using a browser or a dedicated application.

[0145] server

[0146] The server receives question data sent from the user's device. It then passes the received question data to a generative AI for analysis. This generative AI includes a model capable of natural language processing (e.g., GPT-3 (registered trademark) from OpenAI (registered trademark)). The server converts the response received from the generative AI into JSON format and sends it to the user's device.

[0147] Data processing and calculation

[0148] Generative Artificial Intelligence

[0149] Generative AI analyzes the received question data. It uses natural language processing technology to understand the intent of the question and extract the necessary keywords. Based on the extracted keywords, it searches for related technical documents from a database. It extracts the most relevant information and generates an appropriate response based on that. For example, it uses a prompt such as, "Please tell me how to create a graph in Excel."

[0150] Specific examples

[0151] Entering and processing questions

[0152] For example, a user enters the question "How do I create a graph in Excel?" into a web form and submits it. The device converts this question into JSON format and sends it to the server, like this:

[0153] json

[0154] {

[0155] "question": "How do I create a graph in Excel?"

[0156] }

[0157] The server receives this data and passes it to the generative AI, which analyzes the question and generates a response like this:

[0158] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[0159] The generated response is returned to the server, which converts it to JSON format and sends it to the user's device, which receives it and displays it to the user.

[0160] Questions and handling of obstacles

[0161] For example, suppose a user posts a question such as "What causes my application to crash?" This question is also sent by the device to the server, where it is analyzed by the generative AI. Relevant log data is collected and it is determined that the cause is insufficient memory. A response is generated stating, "Insufficient memory is likely the cause of the application crash. To resolve this, please close unnecessary applications or consider increasing memory," and is provided to the user using the same procedure.

[0162] This system allows users to efficiently obtain the necessary information and quickly receive fault analysis and countermeasures.

[0163] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0164] Processing flow and specific operations at each step

[0165] Step 1:

[0166] User

[0167] A user fills out a web form with a question about a particular feature or operation and clicks the submit button. The input is in the form of text, such as "How do I create a chart in Excel?"

[0168] Step 2:

[0169] Terminal

[0170] The device receives the question entered by the user and converts it into JSON format. For example, it is converted as follows:

[0171] json

[0172] {

[0173] "question": "How do I create a graph in Excel?"

[0174] }

[0175] This JSON data is sent to the server as an HTTPS request.

[0176] Step 3:

[0177] server

[0178] The server receives question data sent from the device. It prepares the received JSON data to be passed to the generative AI. The input is the question data in JSON format, and the output is a prompt text for the generative AI. As a concrete example of the process, the data is converted as follows:

[0179] "Prompt: How do I create a graph in Excel?"

[0180] Step 4:

[0181] Generative Artificial Intelligence

[0182] A generative artificial intelligence analyzes the question based on the received prompt. The technology used is natural language processing. As a result of the analysis, keywords are extracted. For example, keywords such as "Excel," "graph," "create," and "how to" are extracted. Technical documents are searched based on these keywords. The input is the prompt and a database of technical documents, and the output is a set of related technical documents and information.

[0183] Step 5:

[0184] Generative Artificial Intelligence

[0185] It evaluates relevant technical documentation, extracts the most relevant information, and generates a response based on that information in a user-friendly format. For example, it might generate a response like this:

[0186] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[0187] The input is the relevant technical document and the output is the generated response sentence.

[0188] Step 6:

[0189] server

[0190] The generated response is received and converted to JSON format, for example:

[0191] json

[0192] {

[0193] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[0194] }

[0195] This JSON data is sent to the terminal as an HTTP response. The input is the generated response text, and the output is the JSON formatted response data.

[0196] Step 7:

[0197] Terminal

[0198] The device receives the response sent from the server. It parses the received JSON data and displays it on the screen in a format that is easy for the user to understand. For example, it might look like this:

[0199] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[0200] The input is the response data in JSON format, and the output is the answer that is displayed to the user.

[0201] At each step, the user, terminal, server, and generative artificial intelligence work together to efficiently provide appropriate information in response to the user's question.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] In modern manufacturing, the introduction of factory robots is increasing, but when complex settings and operations or unexpected failures occur, quick and accurate troubleshooting is required. With current systems, it is difficult for operators to obtain immediate solutions when they encounter problems, which can have a negative impact on efficiency and productivity. To solve this problem, a system that provides accurate information in real time and enables rapid response is needed.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes means for receiving question data from a user's information processing device, means for transmitting the question data to a generative artificial intelligence, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for transmitting the generated response sentence to the user's information processing device, means for displaying the response sentence on the user's information processing device, means for the generative artificial intelligence to use natural language processing when analyzing the question content, means for collecting related log data and analyzing the cause of the failure when the system receives a question about a failure, means for generating a response sentence proposing countermeasures for the failure, and means for transmitting the generated response sentence to the user's information processing device and displaying it. This allows an operator to obtain information for solving problems in real time, enabling fast and accurate troubleshooting.

[0207] A "user's information processing device" is a device that allows a user to input questions and check responses, and includes a smartphone, smart glasses, a tablet, a personal computer, etc.

[0208] "Question data" refers to data on the content of a question entered by a user via an information processing device, and is sent to the server in a data format such as JSON.

[0209] "Generative AI" is AI that analyzes input questions and generates appropriate responses, and includes, for example, models that use natural language processing technology.

[0210] "Natural language processing" is a general term for technologies that allow computers to understand, analyze, and generate human language, and is a technology that enables the analysis of text data and understanding of intent.

[0211] A "response sentence" is a response sentence generated by a generative artificial intelligence in response to a user's question, and is text data used to provide the user with appropriate information and countermeasures.

[0212] "Technical documentation" means documentation that contains information about a specific technology or operating method, including manuals, technical reports, online help, etc.

[0213] "Log data" is data that records the operating status and error information of systems and applications, and is used to analyze the cause of failures.

[0214] A "failure" refers to a state in which a system or application does not function properly, and is a problem that requires a solution based on a question from a user.

[0215] "Troubleshooting" refers to a set of steps for identifying, analyzing, and correcting problems with a system or application.

[0216] The present invention relates to a system that allows factory robot operators to post questions in real time about problems or malfunctions they are facing, and uses generative artificial intelligence to instantly analyze the questions and provide appropriate solutions. This system enables operators to respond quickly on-site, particularly by using information processing devices such as smartphones and smart glasses.

[0217] Hardware and Software Overview

[0218] Hardware:

[0219] Information processing devices: smartphones, smart glasses, tablets, PCs, etc.

[0220] server

[0221] software:

[0222] Programming language: Python

[0223] Communication library: requests

[0224] Generative AI: For example, OpenAI's GPT-4 (registered trademark)

[0225] Natural language processing technology

[0226] System operation explanation

[0227] 1. Enter and submit your question:

[0228] An operator inputs questions about the operation of a factory robot or problems into an information processing device (e.g., a smartphone). For example, the following questions are input:

[0229] My robot suddenly stopped working, what should I do?

[0230] When the user clicks the submit button, the question data is converted to JSON format and sent to the server as an HTTPS request.

[0231] 2. Server query reception and analysis:

[0232] The server receives the question data sent from the information processing device. The received data is passed to generative AI, which uses natural language processing to analyze the question and understand its intent. This analysis extracts relevant keywords and context, and searches for appropriate technical documents.

[0233] 3. Search for and extract information from relevant technical documents:

[0234] The server runs a database query to find relevant technical documents. Generative AI evaluates the search results and extracts the most relevant information. For example, if a robot stopped working because of a power supply problem and the appropriate solution is to check the power cable, a response based on that information is generated.

[0235] 4. Generate and send the response:

[0236] Based on the extracted information, the generative AI generates a response in a format that is easy for the user to understand. The generated response is then converted back to JSON format and sent to the information processing device. For example, the following response may be generated:

[0237] My robot suddenly stopped working, what should I do?

[0238] This may be due to a power supply problem. Check the power cable.

[0239] 5. Displaying the response sentence on the information processing device:

[0240] The information processing device displays the received response to the user, allowing the operator to check the corrective action in real time and quickly resolve the problem.

[0241] As a specific example, if a user inputs the question "My robot suddenly stopped, what should I do?", the server analyzes the question and, if it determines that the problem is due to a power supply problem, generates a response saying "Check the power cable" and displays it to the user.

[0242] This allows operators to respond quickly on-site and minimizes downtime for factory robots, contributing to improved efficiency and productivity in the manufacturing industry.

[0243] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0244] Step 1:

[0245] A user inputs a question using an information processing device. The input question is converted into JSON format data. For example, if the question "My robot suddenly stopped, what should I do?" is input, the resulting JSON data will be as follows:

[0246] Input: [user question]

[0247] Output: {"question": "My robot suddenly stopped, what should I do?"}

[0248] Specific actions: The user uses a smartphone or smart glasses to enter a question and click the send button.

[0249] Step 2:

[0250] The device sends the question data to the server as an HTTPS request. The question data is sent in JSON format.

[0251] Input: {"question": "My robot suddenly stopped, what should I do?"}

[0252] Output: HTTPS request

[0253] Specific operation: By clicking the send button, the terminal sends the question data to the server.

[0254] Step 3:

[0255] The server receives the question data and passes it to the generative AI, which analyzes the question and understands its intent.

[0256] Input: HTTPS request

[0257] Output: Analysis results that understand the intent of the question (e.g., keywords such as "power supply," "stop," and "measures")

[0258] Specific operation: The server passes the question data to the analysis module, and a generative artificial intelligence (such as GPT-4) analyzes it and extracts keywords.

[0259] Step 4:

[0260] The server searches for relevant technical documents, and executes a database query based on the extracted keywords to find relevant documents.

[0261] Input: Keywords of analysis results

[0262] Output: List of related technical documents

[0263] Specific operation: The server searches the database for appropriate technical documents and generates a list of relevant documents.

[0264] Step 5:

[0265] Generative artificial intelligence extracts relevant technical information from the search results and generates a response.

[0266] Input: List of relevant technical documents

[0267] Output: Response (e.g. "This may be due to a problem with the power supply. Please check the power cable.")

[0268] Specific operation: The generative AI evaluates relevant technical documents and generates a response to the user based on the most appropriate information.

[0269] Step 6:

[0270] The server converts the generated response text into JSON format and sends it to the terminal.

[0271] Input: Response

[0272] Output: {"response": "This may be due to a power supply problem. Please check the power cable."}

[0273] Specific operation: The server converts the response text into JSON format and sends it to the terminal as an HTTPS request.

[0274] Step 7:

[0275] The terminal receives the response from the server and displays it to the user.

[0276] Input: {"response": "This may be due to a power supply problem. Please check the power cable."}

[0277] Output: What is displayed to the user

[0278] Specific operation: The terminal displays the received response on the screen, and the user can check it and take prompt action.

[0279] Using generative AI models and prompts, these steps work together to provide real-time, highly accurate information.

[0280] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0281] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[0282] overview

[0283] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[0284] Enter and submit questions and emotion data

[0285] User

[0286] A user enters a question about a specific function or operation method (e.g., "How do I create a graph in Excel?") into a web form and clicks the submit button. At the same time, the system analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data.

[0287] Terminal

[0288] The user's device converts the entered question and emotion data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[0289] {

[0290] "question": "How to create a graph in Excel?",

[0291] "emotion": "curious"

[0292] }

[0293] Receiving and analyzing questions and emotion data

[0294] server

[0295] The server receives the question data and emotion data sent from the user's terminal.

[0296] server

[0297] The server passes the received question data to the generative artificial intelligence and passes the emotion data to the emotion engine.

[0298] Generative Artificial Intelligence

[0299] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[0300] Emotion Engine

[0301] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[0302] Related document search and information extraction

[0303] server

[0304] The server runs a database query to find relevant technical documents, generating a list of hit documents that are then evaluated by a generative artificial intelligence.

[0305] Generative Artificial Intelligence

[0306] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[0307] Response generation and emotional reflection

[0308] Generative Artificial Intelligence

[0309] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0310] Emotion Engine

[0311] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[0312] Sending and displaying responses

[0313] server

[0314] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[0315] {

[0316] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[0317] }

[0318] Terminal

[0319] The user's terminal receives the response from the server and displays it on the screen.

[0320] Questions about disabilities and solutions

[0321] User

[0322] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0323] Terminal

[0324] The user's device sends question data and emotion data in JSON format to the server.

[0325] server

[0326] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[0327] Generative Artificial Intelligence

[0328] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[0329] Emotion Engine

[0330] The emotion engine analyzes the emotion data and identifies the user's emotion.

[0331] Generative Artificial Intelligence

[0332] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[0333] Emotion Engine

[0334] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[0335] server

[0336] The generated response sentence is sent to the user's terminal.

[0337] Terminal

[0338] The user's terminal receives the response and displays it on the screen.

[0339] The above is an embodiment of the present invention. This system not only allows users to efficiently obtain necessary information, but also provides emotional support.

[0340] The processing flow will be explained below.

[0341] Step 1:

[0342] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[0343] Step 2:

[0344] The user's device converts the entered question, along with emotional data obtained from the user's facial expressions and tone of voice, into JSON format and sends it to the server as an HTTPS request. For example, it will look like this:

[0345] {

[0346] "question": "How to create a graph in Excel?",

[0347] "emotion": "curious"

[0348] }

[0349] Step 3:

[0350] The server receives the question data and emotion data sent from the user's terminal.

[0351] Step 4:

[0352] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[0353] Step 5:

[0354] Generative AI uses natural language processing to analyze the question and extract key keywords, such as "Excel," "graph," "create," and "how."

[0355] Step 6:

[0356] The emotion engine analyzes the emotion data to identify the user's emotional state, for example, the emotion "curious."

[0357] Step 7:

[0358] The server queries the database to find relevant technical documents and generates a list of documents that are found as search results.

[0359] Step 8:

[0360] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[0361] Step 9:

[0362] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0363] Step 10:

[0364] The emotion engine adjusts the response depending on the user's emotional state, for example adding a sentence that reflects the emotion "Interesting!"

[0365] Step 11:

[0366] The server converts the generated response into JSON format and sends it to the user's device, for example, in the following format:

[0367] {

[0368] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[0369] }

[0370] Step 12:

[0371] The user's terminal receives the response from the server and displays it on the screen.

[0372] Step 13:

[0373] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0374] Step 14:

[0375] The user's device converts the question data and emotion data into JSON format and sends it to the server.

[0376] Step 15:

[0377] The server receives question data and emotion data about the problem transmitted from the user's terminal.

[0378] Step 16:

[0379] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[0380] Step 17:

[0381] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[0382] Step 18:

[0383] The emotion engine analyzes the emotion data and identifies the user's emotion.

[0384] Step 19:

[0385] The generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[0386] Step 20:

[0387] The emotion engine adjusts the response sentence depending on the user's emotional state, for example adding a sentence that reflects the emotion, "Don't worry, it's okay!"

[0388] Step 21:

[0389] The server sends the generated response to the user's terminal.

[0390] Step 22:

[0391] The user's terminal receives the response from the server and displays it on the screen.

[0392] Example 2

[0393] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0394] Current question-answering systems often only provide information without considering the user's feelings. This leaves the user experience unsatisfactory, making it difficult to alleviate user anxiety and stress, especially when dealing with problems. Furthermore, conventional systems often fail to provide accurate responses because they do not perform detailed analysis of questions or problems.

[0395] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0396] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for the emotion engine to analyze the emotion data and identify the user's emotion, means for searching relevant databases and technical documents, means for extracting the most relevant information from the search results and for the generative artificial intelligence to generate a response sentence, means for the emotion engine to reflect the user's emotion in the response sentence, and means for transmitting the generated response sentence to the user's terminal. This enables an appropriate and friendly response that takes the user's emotion into consideration.

[0397] A "user's terminal" is a computer device that allows a user to input and send a question, and includes a personal computer, a smartphone, or the like.

[0398] "Question data" is information relating to the content of an inquiry that a user inputs and sends via a terminal.

[0399] "Emotion data" is information about emotions extracted from the user's facial expressions, voice, etc.

[0400] "Generative AI" is an AI system that has the ability to analyze input data and generate answers.

[0401] An "emotion engine" is a system that analyzes emotional data collected from users and recognizes specific emotions.

[0402] "Natural language processing" is a technique used by generative artificial intelligence to understand the content of questions, and is a technology that analyzes natural language and extracts meaning.

[0403] A "database" is an information repository where technical documents and related information are stored.

[0404] A "technical document" is a document that contains detailed information about a particular technology and is used to answer user questions.

[0405] A "response sentence" is an answer generated by generative artificial intelligence and provided to the user.

[0406] "Log data" is data that records the operation history of a system or application.

[0407] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and suggest countermeasures. An embodiment of the present invention is described in detail below.

[0408] overview

[0409] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[0410] Hardware and software used

[0411] User

[0412] Users use devices such as PCs and smartphones that have a web browser installed, allowing them to enter questions via a web form.

[0413] Terminal

[0414] The terminal receives input from the user, converts question data and emotion data into JSON format, and sends it to the server.

[0415] server

[0416] The server utilizes cloud infrastructure, such as an AWS® EC2 instance, and uses generative artificial intelligence (e.g., OpenAI GPT-4) and emotion engines (e.g., Microsoft® Azure® Face API) to analyze and process the incoming data.

[0417] Generative Artificial Intelligence

[0418] Generative AI uses natural language processing to analyze the input question and generate a response based on related technical documentation.

[0419] Emotion Engine

[0420] The emotion engine acquires and analyzes emotion data from the user's facial expressions and tone of voice.

[0421] Enter and submit questions and emotion data

[0422] User

[0423] A user enters a question into a web form, such as "How do I create a graph in Excel?", and clicks the submit button. At this time, the system analyzes the user's facial expressions and tone of voice to obtain emotional data (e.g., "curious").

[0424] Receiving and analyzing questions and emotion data

[0425] server

[0426] The server receives the question data and emotion data sent from the user's device, passes the received question data to the generative AI, and passes the emotion data to the emotion engine.

[0427] Generative Artificial Intelligence

[0428] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[0429] Emotion Engine

[0430] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[0431] Related document search and information extraction

[0432] server

[0433] The server queries the database to find relevant technical documents, and the resulting list of documents is passed to the generative AI.

[0434] Generative Artificial Intelligence

[0435] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[0436] Response generation and emotional reflection

[0437] Generative Artificial Intelligence

[0438] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0439] Emotion Engine

[0440] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[0441] Sending and displaying responses

[0442] server

[0443] The generated response is converted to JSON format and sent to the user's device. For example, it is sent in the following format:

[0444] json

[0445] {

[0446] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[0447] }

[0448] Terminal

[0449] The terminal receives the response from the server and displays it on the screen.

[0450] Questions about disabilities and solutions

[0451] User

[0452] A user posts a question like, "What causes my application to crash?"

[0453] Terminal

[0454] The device sends question data and emotion data in JSON format to the server.

[0455] server

[0456] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[0457] Generative Artificial Intelligence

[0458] Generative artificial intelligence analyzes the question content and collects and analyzes related log data (system logs and error logs).

[0459] Emotion Engine

[0460] The emotion engine analyzes the emotion data and identifies the user's emotion.

[0461] Generative Artificial Intelligence

[0462] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, if the cause of the problem is identified as "insufficient memory," it generates a response such as "Please close unnecessary applications or consider increasing memory."

[0463] Emotion Engine

[0464] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[0465] server

[0466] The generated response sentence is sent to the user's terminal.

[0467] Terminal

[0468] The terminal receives the response and displays it on the screen.

[0469] As described above, the system of the present invention not only allows users to efficiently obtain the information they need, but also provides emotional support. For example, in response to the question, "How do I create a graph in Excel?", the system provides the following response: "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the type of graph you want from the 'Chart' group. Interesting!" In this way, the system provides users with prompt and appropriate information tailored to their needs.

[0470] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0471] Processing flow

[0472] Step 1:

[0473] A user enters a question about a specific function or operation method into a web form and clicks the submit button. For example, the input might be, "How do I create a graph in Excel?" At this time, the user's facial expression and tone of voice are sent to the device as emotional data.

[0474] Step 2:

[0475] The device converts the input question and emotion data into JSON format and sends it to the server as an HTTPS request. The input data is in the following format:

[0476] json

[0477] {

[0478] "question": "How to create a graph in Excel?",

[0479] "emotion": "curious"

[0480] }

[0481] Step 3:

[0482] The server receives question data and emotion data sent from the user's device, processes the received data, and passes it to the generative AI and emotion engine in an appropriate format.

[0483] Step 4:

[0484] Generative AI analyzes question data. It uses natural language processing (NLP) to understand the question and extract key keywords. The input data is the question data, and the output data is the extracted keywords (e.g., "Excel," "graph," "create," "how to").

[0485] Step 5:

[0486] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "interesting").

[0487] Step 6:

[0488] The server executes a database query to search for relevant technical documents, with the input data being the extracted keywords and the output data being a list of hit technical documents.

[0489] Step 7:

[0490] Generative AI evaluates search results and extracts the most relevant information. The input data is a list of technical documents, and the output data is the extracted technical information.

[0491] Step 8:

[0492] Based on the information extracted by the generative artificial intelligence, a response is generated in a format that is easy for the user to understand. Specifically, the response generated is, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group." The input data is the extracted technical information, and the output data is the generated response.

[0493] Step 9:

[0494] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Interesting!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[0495] Step 10:

[0496] The server converts the generated response into JSON format and sends it to the user's device, specifically in the following format:

[0497] json

[0498] {

[0499] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[0500] }

[0501] Step 11:

[0502] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[0503] Inquiry about disabilities and provision of solutions

[0504] Step 12:

[0505] A user posts a question about a specific problem or error. For example, "What causes my application to crash?"

[0506] Step 13:

[0507] The device sends question data and emotion data in JSON format to the server. The input data is in the following format:

[0508] json

[0509] {

[0510] "question": "What causes my application to crash?",

[0511] "emotion": "worried"

[0512] }

[0513] Step 14:

[0514] The server receives the question data and emotion data and passes them to the generative AI and emotion engine. The server processes the received data and sends it in the appropriate format.

[0515] Step 15:

[0516] The generative AI analyzes the question content and collects and analyzes related log data (e.g., system logs, error logs). The input data is the question data, and the output data is the analyzed log data and the extracted cause of the failure.

[0517] Step 16:

[0518] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "worry").

[0519] Step 17:

[0520] The generative AI analyzes the cause of the failure and countermeasures, and generates a response. Specifically, it identifies the cause of the failure as "insufficient memory," and generates a response saying, "Please close unnecessary applications or consider increasing memory." The input data is the analyzed log data and the extracted cause of the failure, and the output data is the generated response.

[0521] Step 18:

[0522] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Don't worry, everything is fine!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[0523] Step 19:

[0524] The server converts the generated response text into JSON format and sends it to the user's device. The input data is the generated response text, and the output data is the JSON format data sent to the user's device.

[0525] Step 20:

[0526] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[0527] This is the specific processing flow of this system, which allows users to efficiently obtain information while also receiving emotional support.

[0528] (Application example 2)

[0529] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0530] Many users today are facing serious security-related problems. In these circumstances, conventional automated response systems must not only provide accurate information in response to users' questions and problems, but also provide emotional support. However, current systems lack emotional consideration, making it difficult for users who are anxious or nervous to feel at ease. Furthermore, there is a need for systems that can respond to users' emotions rather than simply providing technical information. A system that can appropriately address these issues is needed.

[0531] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0532] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for emotionally adjusting the response sentence based on the emotion data analyzed by the emotion engine, and means for transmitting the generated response sentence to the user's terminal. This makes it possible to provide not only accurate information in response to a user's questions or problems, but also emotional support.

[0533] "Question data" is text information about questions or problems that users input to the system.

[0534] "Emotion data" refers to emotion information extracted from the user's facial expression, tone of voice, and text content.

[0535] "Generative AI" is an AI system that uses natural language processing technology to analyze the content of a user's question, understand their intent, and generate an appropriate response.

[0536] An "emotion engine" is a system that analyzes a user's emotional data and generates a response that takes those emotions into consideration.

[0537] A "user's terminal" is an information device such as a smartphone or a personal computer used by a user.

[0538] The "server" is an information processing device that relays between the user's device and the generative AI and emotion engine, receiving and sending data and generating responses.

[0539] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[0540] "Technical documents" are documents that contain relevant technical information and are the subject of searches by the server.

[0541] "Log data" refers to data that records the operation history and error information of systems and applications.

[0542] A "response sentence" is an answer to a user's question generated by a generative artificial intelligence, and is a sentence adjusted by an emotion engine.

[0543] The present invention relates to a system that receives question data and emotion data from a user's device and uses generative artificial intelligence and an emotion engine to provide appropriate and emotionally sensitive responses. This system is particularly effective in responding to security-related questions and problems.

[0544] System Program Overview

[0545] 1. Receiving user questions and emotion data

[0546] The user enters a question in text and sends it from the terminal.

[0547] Emotional data is obtained from the user's facial expressions and tone of voice, and transmitted from the device.

[0548] 2. Question and Sentiment Data Analysis

[0549] A server receives the question data and the emotion data.

[0550] Generative AI analyzes the question using natural language processing and extracts important keywords.

[0551] The emotion engine analyzes the emotion data and identifies the user's emotion.

[0552] 3. Technical Document Search and Response Generation

[0553] The server searches the database for relevant technical documents and retrieves the most relevant information.

[0554] Generative artificial intelligence generates appropriate responses from technical documents.

[0555] The emotion engine adjusts the response sentence according to the user's emotion.

[0556] 4. Sending a response

[0557] The server sends the final response to the terminal and displays it to the user.

[0558] Implementation hardware and software

[0559] Hardware

[0560] User's device: Information devices such as smartphones and PCs

[0561] Server: A high-performance server for data processing and analysis

[0562] software

[0563] Natural language processing technology: Google (registered trademark) Cloud Natural Language API, IBM Watson (registered trademark), etc.

[0564] Sentiment analysis technology: Microsoft Azure Emotion API, Amazon Rekognition, etc.

[0565] Database: Relational database such as MySQL (registered trademark), PostgreSQL, etc.

[0566] Generative AI: GPT-4 and similar generative AI models

[0567] Example of system operation

[0568] Entering questions and sentiment data

[0569] The user types the question "Is this network safe?" and sends it from the device. At the same time, emotional data such as "anxiety" is obtained from the user's facial expression and tone of voice.

[0570] analysis

[0571] The server receives this data, and the generative AI extracts keywords such as "network" and "safety," while the emotion engine identifies the emotion "anxiety."

[0572] Response generation and adjustment

[0573] The generative AI generates a technical response such as "The current network is encrypted," while the emotion engine adds an emotional element: "Don't worry, we'll follow up."

[0574] Sending a response

[0575] The final response, "The current network is encrypted. Don't worry, we will follow up, so don't worry," is sent to the user's device and displayed.

[0576] Prompt Sentence Examples

[0577] "Is my network secure?"

[0578] "I think I might have the virus, what should I do?"

[0579] "How can I reduce the risk of my password being hacked?"

[0580] This can alleviate the user's doubts and anxieties and provide a sense of security.

[0581] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0582] Step 1:

[0583] The user inputs and sends question data and emotion data from the terminal.

[0584] Input: The user enters the question "Is this network safe?" and emotional data of "anxiety" is obtained from facial expressions and tone of voice.

[0585] Processing: The device converts this data into JSON format and sends it to the server as an HTTPS request.

[0586] Output: Question data and emotion data are sent to the server.

[0587] Step 2:

[0588] A server receives the question data and the emotion data.

[0589] Input: Question data and emotion data sent from the device

[0590] Processing: The server passes the received data to the generative artificial intelligence and emotion engine for analysis.

[0591] Output: Question data passed to the generative AI, emotion data passed to the emotion engine

[0592] Step 3:

[0593] Generative AI analyzes the content of the question and understands its intent.

[0594] Input: Query data passed from the server

[0595] Processing: Using natural language processing technology (e.g., Google Cloud Natural Language API), important keywords are extracted from the question.

[0596] Data processing and calculation: For example, extracting keywords such as "network" and "safety."

[0597] Output: Parsed question content and extracted keywords

[0598] Step 4:

[0599] The emotion engine analyzes the emotion data and identifies the user's emotion.

[0600] Input: Emotion data passed from the server

[0601] Processing: Analyze the emotion data using emotion analysis technology (e.g., Microsoft Azure Emotion API).

[0602] Data processing and calculation: Identify the user's emotion as "anxiety."

[0603] Output: Analyzed user emotion information

[0604] Step 5:

[0605] The server searches the database for relevant technical documents and retrieves the most relevant information.

[0606] Input: Keywords passed from the generative AI

[0607] Processing: Executes database queries and retrieves technical documents.

[0608] Data processing and calculation: Narrow down relevant documents based on keywords.

[0609] Output: Search results for a list of technical documents

[0610] Step 6:

[0611] Generative artificial intelligence generates appropriate responses from technical documents.

[0612] Input: List of technical documents and questions

[0613] Processing: Generate the most appropriate response sentence based on the technical documentation.

[0614] Data processing and calculation: For example, generating a technical response such as "The current network is encrypted."

[0615] Output: Response

[0616] Step 7:

[0617] The emotion engine adjusts the response sentence according to the user's emotion.

[0618] Input: Response sentences from generative AI and analyzed emotional information

[0619] Processing: Emotionally adjusting the response sentence.

[0620] Data processing and calculation: For example, adding an emotional element such as "Don't worry, we will follow up, so don't worry."

[0621] Output: Emotionally tailored response sentence

[0622] Step 8:

[0623] The server sends the final response to the terminal and displays it to the user.

[0624] Input: Emotionally tailored response sentences

[0625] Processing: Convert the response into JSON format and send it to the terminal.

[0626] Output: The final response sent to the terminal.

[0627] These processing steps allow users to receive technically accurate and emotionally sensitive responses to their questions and concerns in real time.

[0628] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0629] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0630] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0631] [Second embodiment]

[0632] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0633] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0634] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0635] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0636] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0637] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0638] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0639] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0640] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0641] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0642] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0643] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0644] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[0645] overview

[0646] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data into JSON format and sends it to the server as an HTTPS request. The server receives the question data and passes it to a generative AI. The generative AI analyzes the question and searches for related technical documents. It then extracts the most relevant information from the search results, generates a response, and sends it to the user's device.

[0647] Enter and submit your question

[0648] User

[0649] A user fills out a web form with a question about a specific feature or procedure (e.g., "How do I create a chart in Excel?") and clicks the submit button.

[0650] Terminal

[0651] The user's device converts the entered question into JSON format and sends it to the server as an HTTPS request, for example, in the following format:

[0652] {

[0653] "question": "How do I create a graph in Excel?"

[0654] }

[0655] Receiving and parsing questions

[0656] server

[0657] The server receives question data sent from the user's device. The received question data is then sent to the generative AI. The generative AI analyzes the question using natural language processing and extracts keywords. For example, keywords such as "Excel," "graph," "create," and "method" are extracted.

[0658] Related document search and information extraction

[0659] server

[0660] The server runs a database query to find relevant technical documents, generates a list of hit documents, and a generative AI evaluates this list, extracts the most relevant information, and generates a response.

[0661] Generate and send a response

[0662] Generative Artificial Intelligence

[0663] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0664] server

[0665] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[0666] {

[0667] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[0668] }

[0669] Terminal

[0670] The user's terminal receives the response from the server and displays it on the screen.

[0671] Questions about disabilities and solutions

[0672] User

[0673] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0674] Terminal

[0675] The user's terminal transmits the question data to the server.

[0676] server

[0677] The server receives the question and passes it to the generative AI, which analyzes the question and collects relevant log data. For example, it analyzes system logs and error logs to identify the cause of the problem, which is "insufficient memory."

[0678] Generative Artificial Intelligence

[0679] AI analyzes the cause of the failure and countermeasures based on the log data, and generates a response message with the countermeasures. For example, the response message generated might read, "The cause of the application crash is likely a lack of memory. To resolve this, close any unnecessary applications or consider increasing the memory."

[0680] server

[0681] The generated response sentence is sent to the user's terminal.

[0682] Terminal

[0683] The user's terminal displays the response.

[0684] In this way, the system of the present invention is designed to enable users to efficiently obtain the information they need, and provides specific operating procedures and troubleshooting tips quickly and accurately.

[0685] The processing flow will be explained below.

[0686] Step 1:

[0687] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[0688] Step 2:

[0689] The device converts the question data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[0690] {

[0691] "question": "How do I create a graph in Excel?"

[0692] }

[0693] Step 3:

[0694] The server receives the question data sent from the user's terminal.

[0695] Step 4:

[0696] The server passes the received question data to the generative artificial intelligence.

[0697] Step 5:

[0698] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[0699] Step 6:

[0700] The server queries the database to find relevant technical documents, and generates a list of documents that are hit by the search.

[0701] Step 7:

[0702] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[0703] Step 8:

[0704] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0705] Step 9:

[0706] The server converts the generated response into JSON format and sends it to the user's device. For example, it will be sent in the following format:

[0707] {

[0708] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and select the chart type you want from the 'Charts' group."

[0709] }

[0710] Step 10:

[0711] The user's terminal receives the response from the server and displays it on the screen.

[0712] Step 11:

[0713] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0714] Step 12:

[0715] The device converts the question data into JSON format and sends it to the server.

[0716] Step 13:

[0717] The server receives question data about the problem sent from the user's terminal.

[0718] Step 14:

[0719] The server passes the received question data to the generative artificial intelligence.

[0720] Step 15:

[0721] Generative AI uses natural language processing to analyze the question and collect relevant log data, such as analyzing patterns in system logs and error logs.

[0722] Step 16:

[0723] Based on the log data collected by the generative AI, the cause of the failure is analyzed and a likely solution is generated. For example, the cause of the failure can be identified as "insufficient memory."

[0724] Step 17:

[0725] The generative AI generates a response with a solution, such as, "The application crash may be caused by insufficient memory. To solve this problem, close any unnecessary applications or consider increasing the memory."

[0726] Step 18:

[0727] The server sends the generated response to the user's terminal.

[0728] Step 19:

[0729] The user's terminal receives the response from the server and displays it on the screen.

[0730] The specific processing steps of the entire system have been explained above. At each step, the aim is to provide information to the user efficiently and accurately.

[0731] Example 1

[0732] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0733] Conventional information search systems have the problem that it is difficult for users to quickly obtain appropriate information on specific operation methods or troubleshooting. It is particularly difficult to provide appropriate information immediately for questions that require specialized knowledge. Furthermore, when a malfunction occurs, it is not possible to identify the cause and provide appropriate troubleshooting. This often results in a waste of time and effort on the part of the user.

[0734] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0735] In this invention, the server includes a means for receiving question data from a user's device, a means for transmitting the received question data to a generative artificial intelligence, and a means for the generative artificial intelligence to analyze the question content and understand the intent of the question. This enables users to quickly obtain high-quality information. Furthermore, the server includes a means for searching for related technical documents, a means for extracting the most relevant information from the search results and generating a response, a means for transmitting the generated response to the user's device, a means for converting the question into JSON format on the user's device and transmitting it to the server, and a means for displaying the received response on the user's device. This enables users to quickly and accurately resolve their questions and improve convenience. Furthermore, when the system receives a question about a failure, the server includes a means for collecting related log data, analyzing the cause of the failure, and generating a response that addresses the failure. This enables quick identification of the cause and presentation of a solution when a failure occurs, improving system reliability and user satisfaction.

[0736] A "user" is someone who uses the system to enter questions and obtain information.

[0737] A "terminal" is an electronic device, such as a computer or smartphone, that a user uses to input a question.

[0738] "Question data" refers to data including the question entered by the user via the terminal and sent to the server.

[0739] A "server" is a device that processes question data received from a user's terminal and passes it on to the generative artificial intelligence.

[0740] "Generative AI" is an AI model that analyzes received question data, understands the intent of the question, and generates appropriate information.

[0741] "Natural language processing" is a technology used by generative artificial intelligence to understand and process text data when analyzing question content.

[0742] "Technical documentation" means documentation that contains information about a particular technology.

[0743] A "response sentence" is an answer to a user's question generated by generative artificial intelligence.

[0744] "Log data" refers to data that records the system's operating status and error information.

[0745] A "failure" is an error or problem that prevents the system from functioning properly.

[0746] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[0747] Hardware and software configuration

[0748] User

[0749] Users use devices such as personal computers, tablets, and smartphones. These devices are connected to the Internet and display a web form for submitting questions through a web browser. Users then enter their questions and click the submit button.

[0750] Terminal

[0751] The device receives questions entered by users, converts the received questions into JSON format, and sends them to the server as an HTTPS request. The device operates using a browser or a dedicated application.

[0752] server

[0753] The server receives question data sent from the user's device. It then passes the received question data to a generative AI for analysis. This generative AI includes a model capable of natural language processing (e.g., OpenAI's GPT-3). The server converts the response received from the generative AI into JSON format and sends it to the user's device.

[0754] Data processing and calculation

[0755] Generative Artificial Intelligence

[0756] Generative AI analyzes the received question data. It uses natural language processing technology to understand the intent of the question and extract the necessary keywords. Based on the extracted keywords, it searches for related technical documents from a database. It extracts the most relevant information and generates an appropriate response based on that. For example, it uses a prompt such as, "Please tell me how to create a graph in Excel."

[0757] Specific examples

[0758] Entering and processing questions

[0759] For example, a user enters the question "How do I create a graph in Excel?" into a web form and submits it. The device converts this question into JSON format and sends it to the server, like this:

[0760] json

[0761] {

[0762] "question": "How do I create a graph in Excel?"

[0763] }

[0764] The server receives this data and passes it to the generative AI, which analyzes the question and generates a response like this:

[0765] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[0766] The generated response is returned to the server, which converts it to JSON format and sends it to the user's device, which receives it and displays it to the user.

[0767] Questions and handling of obstacles

[0768] For example, suppose a user posts a question such as "What causes my application to crash?" This question is also sent by the device to the server, where it is analyzed by the generative AI. Relevant log data is collected and it is determined that the cause is insufficient memory. A response is generated stating, "Insufficient memory is likely the cause of the application crash. To resolve this, please close unnecessary applications or consider increasing memory," and is provided to the user using the same procedure.

[0769] This system allows users to efficiently obtain the necessary information and quickly receive fault analysis and countermeasures.

[0770] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0771] Processing flow and specific operations at each step

[0772] Step 1:

[0773] User

[0774] A user fills out a web form with a question about a particular feature or operation and clicks the submit button. The input is in the form of text, such as "How do I create a chart in Excel?"

[0775] Step 2:

[0776] Terminal

[0777] The device receives the question entered by the user and converts it into JSON format. For example, it is converted as follows:

[0778] json

[0779] {

[0780] "question": "How do I create a graph in Excel?"

[0781] }

[0782] This JSON data is sent to the server as an HTTPS request.

[0783] Step 3:

[0784] server

[0785] The server receives question data sent from the device. It prepares the received JSON data to be passed to the generative AI. The input is the question data in JSON format, and the output is a prompt text for the generative AI. As a concrete example of the process, the data is converted as follows:

[0786] "Prompt: How do I create a graph in Excel?"

[0787] Step 4:

[0788] Generative Artificial Intelligence

[0789] A generative artificial intelligence analyzes the question based on the received prompt. The technology used is natural language processing. As a result of the analysis, keywords are extracted. For example, keywords such as "Excel," "graph," "create," and "how to" are extracted. Technical documents are searched based on these keywords. The input is the prompt and a database of technical documents, and the output is a set of related technical documents and information.

[0790] Step 5:

[0791] Generative Artificial Intelligence

[0792] It evaluates relevant technical documentation, extracts the most relevant information, and generates a response based on that information in a user-friendly format. For example, it might generate a response like this:

[0793] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[0794] The input is the relevant technical document and the output is the generated response sentence.

[0795] Step 6:

[0796] server

[0797] The generated response is received and converted to JSON format, for example:

[0798] json

[0799] {

[0800] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[0801] }

[0802] This JSON data is sent to the terminal as an HTTP response. The input is the generated response text, and the output is the JSON formatted response data.

[0803] Step 7:

[0804] Terminal

[0805] The device receives the response sent from the server. It parses the received JSON data and displays it on the screen in a format that is easy for the user to understand. For example, it might look like this:

[0806] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[0807] The input is the response data in JSON format, and the output is the answer that is displayed to the user.

[0808] At each step, the user, terminal, server, and generative artificial intelligence work together to efficiently provide appropriate information in response to the user's question.

[0809] (Application example 1)

[0810] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0811] In modern manufacturing, the introduction of factory robots is increasing, but when complex settings and operations or unexpected failures occur, quick and accurate troubleshooting is required. With current systems, it is difficult for operators to obtain immediate solutions when they encounter problems, which can have a negative impact on efficiency and productivity. To solve this problem, a system that provides accurate information in real time and enables rapid response is needed.

[0812] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0813] In this invention, the server includes means for receiving question data from a user's information processing device, means for transmitting the question data to a generative artificial intelligence, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for transmitting the generated response sentence to the user's information processing device, means for displaying the response sentence on the user's information processing device, means for the generative artificial intelligence to use natural language processing when analyzing the question content, means for collecting related log data and analyzing the cause of the failure when the system receives a question about a failure, means for generating a response sentence proposing countermeasures for the failure, and means for transmitting the generated response sentence to the user's information processing device and displaying it. This allows an operator to obtain information for solving problems in real time, enabling fast and accurate troubleshooting.

[0814] A "user's information processing device" is a device that allows a user to input questions and check responses, and includes a smartphone, smart glasses, a tablet, a personal computer, etc.

[0815] "Question data" refers to data on the content of a question entered by a user via an information processing device, and is sent to the server in a data format such as JSON.

[0816] "Generative AI" is AI that analyzes input questions and generates appropriate responses, and includes, for example, models that use natural language processing technology.

[0817] "Natural language processing" is a general term for technologies that allow computers to understand, analyze, and generate human language, and is a technology that enables the analysis of text data and understanding of intent.

[0818] A "response sentence" is a response sentence generated by a generative artificial intelligence in response to a user's question, and is text data used to provide the user with appropriate information and countermeasures.

[0819] "Technical documentation" means documentation that contains information about a specific technology or operating method, including manuals, technical reports, online help, etc.

[0820] "Log data" is data that records the operating status and error information of systems and applications, and is used to analyze the cause of failures.

[0821] A "failure" refers to a state in which a system or application does not function properly, and is a problem that requires a solution based on a question from a user.

[0822] "Troubleshooting" refers to a set of steps for identifying, analyzing, and correcting problems with a system or application.

[0823] The present invention relates to a system that allows factory robot operators to post questions in real time about problems or malfunctions they are facing, and uses generative artificial intelligence to instantly analyze the questions and provide appropriate solutions. This system enables operators to respond quickly on-site, particularly by using information processing devices such as smartphones and smart glasses.

[0824] Hardware and Software Overview

[0825] Hardware:

[0826] Information processing devices: smartphones, smart glasses, tablets, PCs, etc.

[0827] server

[0828] software:

[0829] Programming language: Python

[0830] Communication library: requests

[0831] Generative AI: For example, OpenAI's GPT-4

[0832] Natural language processing technology

[0833] System operation explanation

[0834] 1. Enter and submit your question:

[0835] An operator inputs questions about the operation of a factory robot or problems into an information processing device (e.g., a smartphone). For example, the following questions are input:

[0836] My robot suddenly stopped working, what should I do?

[0837] When the user clicks the submit button, the question data is converted to JSON format and sent to the server as an HTTPS request.

[0838] 2. Server query reception and analysis:

[0839] The server receives the question data sent from the information processing device. The received data is passed to generative AI, which uses natural language processing to analyze the question and understand its intent. This analysis extracts relevant keywords and context, and searches for appropriate technical documents.

[0840] 3. Search for and extract information from relevant technical documents:

[0841] The server runs a database query to find relevant technical documents. Generative AI evaluates the search results and extracts the most relevant information. For example, if a robot stopped working because of a power supply problem and the appropriate solution is to check the power cable, a response based on that information is generated.

[0842] 4. Generate and send the response:

[0843] Based on the extracted information, the generative AI generates a response in a format that is easy for the user to understand. The generated response is then converted back to JSON format and sent to the information processing device. For example, the following response may be generated:

[0844] My robot suddenly stopped working, what should I do?

[0845] This may be due to a power supply problem. Check the power cable.

[0846] 5. Displaying the response sentence on the information processing device:

[0847] The information processing device displays the received response to the user, allowing the operator to check the corrective action in real time and quickly resolve the problem.

[0848] As a specific example, if a user inputs the question "My robot suddenly stopped, what should I do?", the server analyzes the question and, if it determines that the problem is due to a power supply problem, generates a response saying "Check the power cable" and displays it to the user.

[0849] This allows operators to respond quickly on-site and minimizes downtime for factory robots, contributing to improved efficiency and productivity in the manufacturing industry.

[0850] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0851] Step 1:

[0852] A user inputs a question using an information processing device. The input question is converted into JSON format data. For example, if the question "My robot suddenly stopped, what should I do?" is input, the resulting JSON data will be as follows:

[0853] Input: [user question]

[0854] Output: {"question": "My robot suddenly stopped, what should I do?"}

[0855] Specific actions: The user uses a smartphone or smart glasses to enter a question and click the send button.

[0856] Step 2:

[0857] The device sends the question data to the server as an HTTPS request. The question data is sent in JSON format.

[0858] Input: {"question": "My robot suddenly stopped, what should I do?"}

[0859] Output: HTTPS request

[0860] Specific operation: By clicking the send button, the terminal sends the question data to the server.

[0861] Step 3:

[0862] The server receives the question data and passes it to the generative AI, which analyzes the question and understands its intent.

[0863] Input: HTTPS request

[0864] Output: Analysis results that understand the intent of the question (e.g., keywords such as "power supply," "stop," and "measures")

[0865] Specific operation: The server passes the question data to the analysis module, and a generative artificial intelligence (such as GPT-4) analyzes it and extracts keywords.

[0866] Step 4:

[0867] The server searches for relevant technical documents, and executes a database query based on the extracted keywords to find relevant documents.

[0868] Input: Keywords of analysis results

[0869] Output: List of related technical documents

[0870] Specific operation: The server searches the database for appropriate technical documents and generates a list of relevant documents.

[0871] Step 5:

[0872] Generative artificial intelligence extracts relevant technical information from the search results and generates a response.

[0873] Input: List of relevant technical documents

[0874] Output: Response (e.g. "This may be due to a problem with the power supply. Please check the power cable.")

[0875] Specific operation: The generative AI evaluates relevant technical documents and generates a response to the user based on the most appropriate information.

[0876] Step 6:

[0877] The server converts the generated response text into JSON format and sends it to the terminal.

[0878] Input: Response

[0879] Output: {"response": "This may be due to a power supply problem. Please check the power cable."}

[0880] Specific operation: The server converts the response text into JSON format and sends it to the terminal as an HTTPS request.

[0881] Step 7:

[0882] The terminal receives the response from the server and displays it to the user.

[0883] Input: {"response": "This may be due to a power supply problem. Please check the power cable."}

[0884] Output: What is displayed to the user

[0885] Specific operation: The terminal displays the received response on the screen, and the user can check it and take prompt action.

[0886] Using generative AI models and prompts, these steps work together to provide real-time, highly accurate information.

[0887] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0888] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[0889] overview

[0890] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[0891] Enter and submit questions and emotion data

[0892] User

[0893] A user enters a question about a specific function or operation method (e.g., "How do I create a graph in Excel?") into a web form and clicks the submit button. At the same time, the system analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data.

[0894] Terminal

[0895] The user's device converts the entered question and emotion data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[0896] {

[0897] "question": "How to create a graph in Excel?",

[0898] "emotion": "curious"

[0899] }

[0900] Receiving and analyzing questions and emotion data

[0901] server

[0902] The server receives the question data and emotion data sent from the user's terminal.

[0903] server

[0904] The server passes the received question data to the generative artificial intelligence and passes the emotion data to the emotion engine.

[0905] Generative Artificial Intelligence

[0906] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[0907] Emotion Engine

[0908] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[0909] Related document search and information extraction

[0910] server

[0911] The server runs a database query to find relevant technical documents, generating a list of hit documents that are then evaluated by a generative artificial intelligence.

[0912] Generative Artificial Intelligence

[0913] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[0914] Response generation and emotional reflection

[0915] Generative Artificial Intelligence

[0916] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0917] Emotion Engine

[0918] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[0919] Sending and displaying responses

[0920] server

[0921] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[0922] {

[0923] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[0924] }

[0925] Terminal

[0926] The user's terminal receives the response from the server and displays it on the screen.

[0927] Questions about disabilities and solutions

[0928] User

[0929] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0930] Terminal

[0931] The user's device sends question data and emotion data in JSON format to the server.

[0932] server

[0933] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[0934] Generative Artificial Intelligence

[0935] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[0936] Emotion Engine

[0937] The emotion engine analyzes the emotion data and identifies the user's emotion.

[0938] Generative Artificial Intelligence

[0939] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[0940] Emotion Engine

[0941] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[0942] server

[0943] The generated response sentence is sent to the user's terminal.

[0944] Terminal

[0945] The user's terminal receives the response and displays it on the screen.

[0946] The above is an embodiment of the present invention. This system not only allows users to efficiently obtain necessary information, but also provides emotional support.

[0947] The processing flow will be explained below.

[0948] Step 1:

[0949] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[0950] Step 2:

[0951] The user's device converts the entered question, along with emotional data obtained from the user's facial expressions and tone of voice, into JSON format and sends it to the server as an HTTPS request. For example, it will look like this:

[0952] {

[0953] "question": "How to create a graph in Excel?",

[0954] "emotion": "curious"

[0955] }

[0956] Step 3:

[0957] The server receives the question data and emotion data sent from the user's terminal.

[0958] Step 4:

[0959] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[0960] Step 5:

[0961] Generative AI uses natural language processing to analyze the question and extract key keywords, such as "Excel," "graph," "create," and "how."

[0962] Step 6:

[0963] The emotion engine analyzes the emotion data to identify the user's emotional state, for example, the emotion "curious."

[0964] Step 7:

[0965] The server queries the database to find relevant technical documents and generates a list of documents that are found as search results.

[0966] Step 8:

[0967] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[0968] Step 9:

[0969] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[0970] Step 10:

[0971] The emotion engine adjusts the response depending on the user's emotional state, for example adding a sentence that reflects the emotion "Interesting!"

[0972] Step 11:

[0973] The server converts the generated response into JSON format and sends it to the user's device, for example, in the following format:

[0974] {

[0975] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[0976] }

[0977] Step 12:

[0978] The user's terminal receives the response from the server and displays it on the screen.

[0979] Step 13:

[0980] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[0981] Step 14:

[0982] The user's device converts the question data and emotion data into JSON format and sends it to the server.

[0983] Step 15:

[0984] The server receives question data and emotion data about the problem transmitted from the user's terminal.

[0985] Step 16:

[0986] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[0987] Step 17:

[0988] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[0989] Step 18:

[0990] The emotion engine analyzes the emotion data and identifies the user's emotion.

[0991] Step 19:

[0992] The generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[0993] Step 20:

[0994] The emotion engine adjusts the response sentence depending on the user's emotional state, for example adding a sentence that reflects the emotion, "Don't worry, it's okay!"

[0995] Step 21:

[0996] The server sends the generated response to the user's terminal.

[0997] Step 22:

[0998] The user's terminal receives the response from the server and displays it on the screen.

[0999] Example 2

[1000] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1001] Current question-answering systems often only provide information without considering the user's feelings. This leaves the user experience unsatisfactory, making it difficult to alleviate user anxiety and stress, especially when dealing with problems. Furthermore, conventional systems often fail to provide accurate responses because they do not perform detailed analysis of questions or problems.

[1002] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1003] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for the emotion engine to analyze the emotion data and identify the user's emotion, means for searching relevant databases and technical documents, means for extracting the most relevant information from the search results and for the generative artificial intelligence to generate a response sentence, means for the emotion engine to reflect the user's emotion in the response sentence, and means for transmitting the generated response sentence to the user's terminal. This enables an appropriate and friendly response that takes the user's emotion into consideration.

[1004] A "user's terminal" is a computer device that allows a user to input and send a question, and includes a personal computer, a smartphone, or the like.

[1005] "Question data" is information relating to the content of an inquiry that a user inputs and sends via a terminal.

[1006] "Emotion data" is information about emotions extracted from the user's facial expressions, voice, etc.

[1007] "Generative AI" is an AI system that has the ability to analyze input data and generate answers.

[1008] An "emotion engine" is a system that analyzes emotional data collected from users and recognizes specific emotions.

[1009] "Natural language processing" is a technique used by generative artificial intelligence to understand the content of questions, and is a technology that analyzes natural language and extracts meaning.

[1010] A "database" is an information repository where technical documents and related information are stored.

[1011] A "technical document" is a document that contains detailed information about a particular technology and is used to answer user questions.

[1012] A "response sentence" is an answer generated by generative artificial intelligence and provided to the user.

[1013] "Log data" is data that records the operation history of a system or application.

[1014] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and suggest countermeasures. An embodiment of the present invention is described in detail below.

[1015] overview

[1016] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[1017] Hardware and software used

[1018] User

[1019] Users use devices such as PCs and smartphones that have a web browser installed, allowing them to enter questions via a web form.

[1020] Terminal

[1021] The terminal receives input from the user, converts question data and emotion data into JSON format, and sends it to the server.

[1022] server

[1023] The server utilizes cloud infrastructure, such as AWS EC2 instances, and uses generative artificial intelligence (e.g., OpenAI GPT-4) and emotion engines (e.g., Microsoft Azure Face API) to analyze and process the incoming data.

[1024] Generative Artificial Intelligence

[1025] Generative AI uses natural language processing to analyze the input question and generate a response based on related technical documentation.

[1026] Emotion Engine

[1027] The emotion engine acquires and analyzes emotion data from the user's facial expressions and tone of voice.

[1028] Enter and submit questions and emotion data

[1029] User

[1030] A user enters a question into a web form, such as "How do I create a graph in Excel?", and clicks the submit button. At this time, the system analyzes the user's facial expressions and tone of voice to obtain emotional data (e.g., "curious").

[1031] Receiving and analyzing questions and emotion data

[1032] server

[1033] The server receives the question data and emotion data sent from the user's device, passes the received question data to the generative AI, and passes the emotion data to the emotion engine.

[1034] Generative Artificial Intelligence

[1035] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[1036] Emotion Engine

[1037] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[1038] Related document search and information extraction

[1039] server

[1040] The server queries the database to find relevant technical documents, and the resulting list of documents is passed to the generative AI.

[1041] Generative Artificial Intelligence

[1042] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[1043] Response generation and emotional reflection

[1044] Generative Artificial Intelligence

[1045] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1046] Emotion Engine

[1047] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[1048] Sending and displaying responses

[1049] server

[1050] The generated response is converted to JSON format and sent to the user's device. For example, it is sent in the following format:

[1051] json

[1052] {

[1053] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[1054] }

[1055] Terminal

[1056] The terminal receives the response from the server and displays it on the screen.

[1057] Questions about disabilities and solutions

[1058] User

[1059] A user posts a question like, "What causes my application to crash?"

[1060] Terminal

[1061] The device sends question data and emotion data in JSON format to the server.

[1062] server

[1063] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[1064] Generative Artificial Intelligence

[1065] Generative artificial intelligence analyzes the question content and collects and analyzes related log data (system logs and error logs).

[1066] Emotion Engine

[1067] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1068] Generative Artificial Intelligence

[1069] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, if the cause of the problem is identified as "insufficient memory," it generates a response such as "Please close unnecessary applications or consider increasing memory."

[1070] Emotion Engine

[1071] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[1072] server

[1073] The generated response sentence is sent to the user's terminal.

[1074] Terminal

[1075] The terminal receives the response and displays it on the screen.

[1076] As described above, the system of the present invention not only allows users to efficiently obtain the information they need, but also provides emotional support. For example, in response to the question, "How do I create a graph in Excel?", the system provides the following response: "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the type of graph you want from the 'Chart' group. Interesting!" In this way, the system provides users with prompt and appropriate information tailored to their needs.

[1077] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1078] Processing flow

[1079] Step 1:

[1080] A user enters a question about a specific function or operation method into a web form and clicks the submit button. For example, the input might be, "How do I create a graph in Excel?" At this time, the user's facial expression and tone of voice are sent to the device as emotional data.

[1081] Step 2:

[1082] The device converts the input question and emotion data into JSON format and sends it to the server as an HTTPS request. The input data is in the following format:

[1083] json

[1084] {

[1085] "question": "How to create a graph in Excel?",

[1086] "emotion": "curious"

[1087] }

[1088] Step 3:

[1089] The server receives question data and emotion data sent from the user's device, processes the received data, and passes it to the generative AI and emotion engine in an appropriate format.

[1090] Step 4:

[1091] Generative AI analyzes question data. It uses natural language processing (NLP) to understand the question and extract key keywords. The input data is the question data, and the output data is the extracted keywords (e.g., "Excel," "graph," "create," "how to").

[1092] Step 5:

[1093] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "interesting").

[1094] Step 6:

[1095] The server executes a database query to search for relevant technical documents, with the input data being the extracted keywords and the output data being a list of hit technical documents.

[1096] Step 7:

[1097] Generative AI evaluates search results and extracts the most relevant information. The input data is a list of technical documents, and the output data is the extracted technical information.

[1098] Step 8:

[1099] Based on the information extracted by the generative artificial intelligence, a response is generated in a format that is easy for the user to understand. Specifically, the response generated is, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group." The input data is the extracted technical information, and the output data is the generated response.

[1100] Step 9:

[1101] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Interesting!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[1102] Step 10:

[1103] The server converts the generated response into JSON format and sends it to the user's device, specifically in the following format:

[1104] json

[1105] {

[1106] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[1107] }

[1108] Step 11:

[1109] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[1110] Inquiry about disabilities and provision of solutions

[1111] Step 12:

[1112] A user posts a question about a specific problem or error. For example, "What causes my application to crash?"

[1113] Step 13:

[1114] The device sends question data and emotion data in JSON format to the server. The input data is in the following format:

[1115] json

[1116] {

[1117] "question": "What causes my application to crash?",

[1118] "emotion": "worried"

[1119] }

[1120] Step 14:

[1121] The server receives the question data and emotion data and passes them to the generative AI and emotion engine. The server processes the received data and sends it in the appropriate format.

[1122] Step 15:

[1123] The generative AI analyzes the question content and collects and analyzes related log data (e.g., system logs, error logs). The input data is the question data, and the output data is the analyzed log data and the extracted cause of the failure.

[1124] Step 16:

[1125] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "worry").

[1126] Step 17:

[1127] The generative AI analyzes the cause of the failure and countermeasures, and generates a response. Specifically, it identifies the cause of the failure as "insufficient memory," and generates a response saying, "Please close unnecessary applications or consider increasing memory." The input data is the analyzed log data and the extracted cause of the failure, and the output data is the generated response.

[1128] Step 18:

[1129] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Don't worry, everything is fine!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[1130] Step 19:

[1131] The server converts the generated response text into JSON format and sends it to the user's device. The input data is the generated response text, and the output data is the JSON format data sent to the user's device.

[1132] Step 20:

[1133] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[1134] This is the specific processing flow of this system, which allows users to efficiently obtain information while also receiving emotional support.

[1135] (Application example 2)

[1136] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1137] Many users today are facing serious security-related problems. In these circumstances, conventional automated response systems must not only provide accurate information in response to users' questions and problems, but also provide emotional support. However, current systems lack emotional consideration, making it difficult for users who are anxious or nervous to feel at ease. Furthermore, there is a need for systems that can respond to users' emotions rather than simply providing technical information. A system that can appropriately address these issues is needed.

[1138] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1139] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for emotionally adjusting the response sentence based on the emotion data analyzed by the emotion engine, and means for transmitting the generated response sentence to the user's terminal. This makes it possible to provide not only accurate information in response to a user's questions or problems, but also emotional support.

[1140] "Question data" is text information about questions or problems that users input to the system.

[1141] "Emotion data" refers to emotion information extracted from the user's facial expression, tone of voice, and text content.

[1142] "Generative AI" is an AI system that uses natural language processing technology to analyze the content of a user's question, understand their intent, and generate an appropriate response.

[1143] An "emotion engine" is a system that analyzes a user's emotional data and generates a response that takes those emotions into consideration.

[1144] A "user's terminal" is an information device such as a smartphone or a personal computer used by a user.

[1145] The "server" is an information processing device that relays between the user's device and the generative AI and emotion engine, receiving and sending data and generating responses.

[1146] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[1147] "Technical documents" are documents that contain relevant technical information and are the subject of searches by the server.

[1148] "Log data" refers to data that records the operation history and error information of systems and applications.

[1149] A "response sentence" is an answer to a user's question generated by a generative artificial intelligence, and is a sentence adjusted by an emotion engine.

[1150] The present invention relates to a system that receives question data and emotion data from a user's device and uses generative artificial intelligence and an emotion engine to provide appropriate and emotionally sensitive responses. This system is particularly effective in responding to security-related questions and problems.

[1151] System Program Overview

[1152] 1. Receiving user questions and emotion data

[1153] The user enters a question in text and sends it from the terminal.

[1154] Emotional data is obtained from the user's facial expressions and tone of voice, and transmitted from the device.

[1155] 2. Question and Sentiment Data Analysis

[1156] A server receives the question data and the emotion data.

[1157] Generative AI analyzes the question using natural language processing and extracts important keywords.

[1158] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1159] 3. Technical Document Search and Response Generation

[1160] The server searches the database for relevant technical documents and retrieves the most relevant information.

[1161] Generative artificial intelligence generates appropriate responses from technical documents.

[1162] The emotion engine adjusts the response sentence according to the user's emotion.

[1163] 4. Sending a response

[1164] The server sends the final response to the terminal and displays it to the user.

[1165] Implementation hardware and software

[1166] Hardware

[1167] User's device: Information devices such as smartphones and PCs

[1168] Server: A high-performance server for data processing and analysis

[1169] software

[1170] Natural language processing technology: Google Cloud Natural Language API, IBM Watson, etc.

[1171] Sentiment analysis technology: Microsoft Azure Emotion API, Amazon Rekognition, etc.

[1172] Database: Relational database such as MySQL or PostgreSQL

[1173] Generative AI: GPT-4 and similar generative AI models

[1174] Example of system operation

[1175] Entering questions and sentiment data

[1176] The user types the question "Is this network safe?" and sends it from the device. At the same time, emotional data such as "anxiety" is obtained from the user's facial expression and tone of voice.

[1177] analysis

[1178] The server receives this data, and the generative AI extracts keywords such as "network" and "safety," while the emotion engine identifies the emotion "anxiety."

[1179] Response generation and adjustment

[1180] The generative AI generates a technical response such as "The current network is encrypted," while the emotion engine adds an emotional element: "Don't worry, we'll follow up."

[1181] Sending a response

[1182] The final response, "The current network is encrypted. Don't worry, we will follow up, so don't worry," is sent to the user's device and displayed.

[1183] Prompt Sentence Examples

[1184] "Is my network secure?"

[1185] "I think I might have the virus, what should I do?"

[1186] "How can I reduce the risk of my password being hacked?"

[1187] This can alleviate the user's doubts and anxieties and provide a sense of security.

[1188] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1189] Step 1:

[1190] The user inputs and sends question data and emotion data from the terminal.

[1191] Input: The user enters the question "Is this network safe?" and emotional data of "anxiety" is obtained from facial expressions and tone of voice.

[1192] Processing: The device converts this data into JSON format and sends it to the server as an HTTPS request.

[1193] Output: Question data and emotion data are sent to the server.

[1194] Step 2:

[1195] A server receives the question data and the emotion data.

[1196] Input: Question data and emotion data sent from the device

[1197] Processing: The server passes the received data to the generative artificial intelligence and emotion engine for analysis.

[1198] Output: Question data passed to the generative AI, emotion data passed to the emotion engine

[1199] Step 3:

[1200] Generative AI analyzes the content of the question and understands its intent.

[1201] Input: Query data passed from the server

[1202] Processing: Using natural language processing technology (e.g., Google Cloud Natural Language API), important keywords are extracted from the question.

[1203] Data processing and calculation: For example, extracting keywords such as "network" and "safety."

[1204] Output: Parsed question content and extracted keywords

[1205] Step 4:

[1206] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1207] Input: Emotion data passed from the server

[1208] Processing: Analyze the emotion data using emotion analysis technology (e.g., Microsoft Azure Emotion API).

[1209] Data processing and calculation: Identify the user's emotion as "anxiety."

[1210] Output: Analyzed user emotion information

[1211] Step 5:

[1212] The server searches the database for relevant technical documents and retrieves the most relevant information.

[1213] Input: Keywords passed from the generative AI

[1214] Processing: Executes database queries and retrieves technical documents.

[1215] Data processing and calculation: Narrow down relevant documents based on keywords.

[1216] Output: Search results for a list of technical documents

[1217] Step 6:

[1218] Generative artificial intelligence generates appropriate responses from technical documents.

[1219] Input: List of technical documents and questions

[1220] Processing: Generate the most appropriate response sentence based on the technical documentation.

[1221] Data processing and calculation: For example, generating a technical response such as "The current network is encrypted."

[1222] Output: Response

[1223] Step 7:

[1224] The emotion engine adjusts the response sentence according to the user's emotion.

[1225] Input: Response sentences from generative AI and analyzed emotional information

[1226] Processing: Emotionally adjusting the response sentence.

[1227] Data processing and calculation: For example, adding an emotional element such as "Don't worry, we will follow up, so don't worry."

[1228] Output: Emotionally tailored response sentence

[1229] Step 8:

[1230] The server sends the final response to the terminal and displays it to the user.

[1231] Input: Emotionally tailored response sentences

[1232] Processing: Convert the response into JSON format and send it to the terminal.

[1233] Output: The final response sent to the terminal.

[1234] These processing steps allow users to receive technically accurate and emotionally sensitive responses to their questions and concerns in real time.

[1235] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1236] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1237] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1238] [Third embodiment]

[1239] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1240] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1241] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1242] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1243] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1244] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1245] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1246] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1247] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1248] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1249] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1250] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1251] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[1252] overview

[1253] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data into JSON format and sends it to the server as an HTTPS request. The server receives the question data and passes it to a generative AI. The generative AI analyzes the question and searches for related technical documents. It then extracts the most relevant information from the search results, generates a response, and sends it to the user's device.

[1254] Enter and submit your question

[1255] User

[1256] A user fills out a web form with a question about a specific feature or procedure (e.g., "How do I create a chart in Excel?") and clicks the submit button.

[1257] Terminal

[1258] The user's device converts the entered question into JSON format and sends it to the server as an HTTPS request, for example, in the following format:

[1259] {

[1260] "question": "How do I create a graph in Excel?"

[1261] }

[1262] Receiving and parsing questions

[1263] server

[1264] The server receives question data sent from the user's device. The received question data is then sent to the generative AI. The generative AI analyzes the question using natural language processing and extracts keywords. For example, keywords such as "Excel," "graph," "create," and "method" are extracted.

[1265] Related document search and information extraction

[1266] server

[1267] The server runs a database query to find relevant technical documents, generates a list of hit documents, and a generative AI evaluates this list, extracts the most relevant information, and generates a response.

[1268] Generate and send a response

[1269] Generative Artificial Intelligence

[1270] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1271] server

[1272] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[1273] {

[1274] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[1275] }

[1276] Terminal

[1277] The user's terminal receives the response from the server and displays it on the screen.

[1278] Questions about disabilities and solutions

[1279] User

[1280] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[1281] Terminal

[1282] The user's terminal transmits the question data to the server.

[1283] server

[1284] The server receives the question and passes it to the generative AI, which analyzes the question and collects relevant log data. For example, it analyzes system logs and error logs to identify the cause of the problem, which is "insufficient memory."

[1285] Generative Artificial Intelligence

[1286] AI analyzes the cause of the failure and countermeasures based on the log data, and generates a response message with the countermeasures. For example, the response message generated might read, "The cause of the application crash is likely a lack of memory. To resolve this, close any unnecessary applications or consider increasing the memory."

[1287] server

[1288] The generated response sentence is sent to the user's terminal.

[1289] Terminal

[1290] The user's terminal displays the response.

[1291] In this way, the system of the present invention is designed to enable users to efficiently obtain the information they need, and provides specific operating procedures and troubleshooting tips quickly and accurately.

[1292] The processing flow will be explained below.

[1293] Step 1:

[1294] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[1295] Step 2:

[1296] The device converts the question data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[1297] {

[1298] "question": "How do I create a graph in Excel?"

[1299] }

[1300] Step 3:

[1301] The server receives the question data sent from the user's terminal.

[1302] Step 4:

[1303] The server passes the received question data to the generative artificial intelligence.

[1304] Step 5:

[1305] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[1306] Step 6:

[1307] The server queries the database to find relevant technical documents, and generates a list of documents that are hit by the search.

[1308] Step 7:

[1309] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[1310] Step 8:

[1311] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1312] Step 9:

[1313] The server converts the generated response into JSON format and sends it to the user's device. For example, it will be sent in the following format:

[1314] {

[1315] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and select the chart type you want from the 'Charts' group."

[1316] }

[1317] Step 10:

[1318] The user's terminal receives the response from the server and displays it on the screen.

[1319] Step 11:

[1320] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[1321] Step 12:

[1322] The device converts the question data into JSON format and sends it to the server.

[1323] Step 13:

[1324] The server receives question data about the problem sent from the user's terminal.

[1325] Step 14:

[1326] The server passes the received question data to the generative artificial intelligence.

[1327] Step 15:

[1328] Generative AI uses natural language processing to analyze the question and collect relevant log data, such as analyzing patterns in system logs and error logs.

[1329] Step 16:

[1330] Based on the log data collected by the generative AI, the cause of the failure is analyzed and a likely solution is generated. For example, the cause of the failure can be identified as "insufficient memory."

[1331] Step 17:

[1332] The generative AI generates a response with a solution, such as, "The application crash may be caused by insufficient memory. To solve this problem, close any unnecessary applications or consider increasing the memory."

[1333] Step 18:

[1334] The server sends the generated response to the user's terminal.

[1335] Step 19:

[1336] The user's terminal receives the response from the server and displays it on the screen.

[1337] The specific processing steps of the entire system have been explained above. At each step, the aim is to provide information to the user efficiently and accurately.

[1338] Example 1

[1339] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1340] Conventional information search systems have the problem that it is difficult for users to quickly obtain appropriate information on specific operation methods or troubleshooting. It is particularly difficult to provide appropriate information immediately for questions that require specialized knowledge. Furthermore, when a malfunction occurs, it is not possible to identify the cause and provide appropriate troubleshooting. This often results in a waste of time and effort on the part of the user.

[1341] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1342] In this invention, the server includes a means for receiving question data from a user's device, a means for transmitting the received question data to a generative artificial intelligence, and a means for the generative artificial intelligence to analyze the question content and understand the intent of the question. This enables users to quickly obtain high-quality information. Furthermore, the server includes a means for searching for related technical documents, a means for extracting the most relevant information from the search results and generating a response, a means for transmitting the generated response to the user's device, a means for converting the question into JSON format on the user's device and transmitting it to the server, and a means for displaying the received response on the user's device. This enables users to quickly and accurately resolve their questions and improve convenience. Furthermore, when the system receives a question about a failure, the server includes a means for collecting related log data, analyzing the cause of the failure, and generating a response that addresses the failure. This enables quick identification of the cause and presentation of a solution when a failure occurs, improving system reliability and user satisfaction.

[1343] A "user" is someone who uses the system to enter questions and obtain information.

[1344] A "terminal" is an electronic device, such as a computer or smartphone, that a user uses to input a question.

[1345] "Question data" refers to data including the question entered by the user via the terminal and sent to the server.

[1346] A "server" is a device that processes question data received from a user's terminal and passes it on to the generative artificial intelligence.

[1347] "Generative AI" is an AI model that analyzes received question data, understands the intent of the question, and generates appropriate information.

[1348] "Natural language processing" is a technology used by generative artificial intelligence to understand and process text data when analyzing question content.

[1349] "Technical documentation" means documentation that contains information about a particular technology.

[1350] A "response sentence" is an answer to a user's question generated by generative artificial intelligence.

[1351] "Log data" refers to data that records the system's operating status and error information.

[1352] A "failure" is an error or problem that prevents the system from functioning properly.

[1353] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[1354] Hardware and software configuration

[1355] User

[1356] Users use devices such as personal computers, tablets, and smartphones. These devices are connected to the Internet and display a web form for submitting questions through a web browser. Users then enter their questions and click the submit button.

[1357] Terminal

[1358] The device receives questions entered by users, converts the received questions into JSON format, and sends them to the server as an HTTPS request. The device operates using a browser or a dedicated application.

[1359] server

[1360] The server receives question data sent from the user's device. It then passes the received question data to a generative AI for analysis. This generative AI includes a model capable of natural language processing (e.g., OpenAI's GPT-3). The server converts the response received from the generative AI into JSON format and sends it to the user's device.

[1361] Data processing and calculation

[1362] Generative Artificial Intelligence

[1363] Generative AI analyzes the received question data. It uses natural language processing technology to understand the intent of the question and extract the necessary keywords. Based on the extracted keywords, it searches for related technical documents from a database. It extracts the most relevant information and generates an appropriate response based on that. For example, it uses a prompt such as, "Please tell me how to create a graph in Excel."

[1364] Specific examples

[1365] Entering and processing questions

[1366] For example, a user enters the question "How do I create a graph in Excel?" into a web form and submits it. The device converts this question into JSON format and sends it to the server, like this:

[1367] json

[1368] {

[1369] "question": "How do I create a graph in Excel?"

[1370] }

[1371] The server receives this data and passes it to the generative AI, which analyzes the question and generates a response like this:

[1372] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[1373] The generated response is returned to the server, which converts it to JSON format and sends it to the user's device, which receives it and displays it to the user.

[1374] Questions and handling of obstacles

[1375] For example, suppose a user posts a question such as "What causes my application to crash?" This question is also sent by the device to the server, where it is analyzed by the generative AI. Relevant log data is collected and it is determined that the cause is insufficient memory. A response is generated stating, "Insufficient memory is likely the cause of the application crash. To resolve this, please close unnecessary applications or consider increasing memory," and is provided to the user using the same procedure.

[1376] This system allows users to efficiently obtain the necessary information and quickly receive fault analysis and countermeasures.

[1377] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1378] Processing flow and specific operations at each step

[1379] Step 1:

[1380] User

[1381] A user fills out a web form with a question about a particular feature or operation and clicks the submit button. The input is in the form of text, such as "How do I create a chart in Excel?"

[1382] Step 2:

[1383] Terminal

[1384] The device receives the question entered by the user and converts it into JSON format. For example, it is converted as follows:

[1385] json

[1386] {

[1387] "question": "How do I create a graph in Excel?"

[1388] }

[1389] This JSON data is sent to the server as an HTTPS request.

[1390] Step 3:

[1391] server

[1392] The server receives question data sent from the device. It prepares the received JSON data to be passed to the generative AI. The input is the question data in JSON format, and the output is a prompt text for the generative AI. As a concrete example of the process, the data is converted as follows:

[1393] "Prompt: How do I create a graph in Excel?"

[1394] Step 4:

[1395] Generative Artificial Intelligence

[1396] A generative artificial intelligence analyzes the question based on the received prompt. The technology used is natural language processing. As a result of the analysis, keywords are extracted. For example, keywords such as "Excel," "graph," "create," and "how to" are extracted. Technical documents are searched based on these keywords. The input is the prompt and a database of technical documents, and the output is a set of related technical documents and information.

[1397] Step 5:

[1398] Generative Artificial Intelligence

[1399] It evaluates relevant technical documentation, extracts the most relevant information, and generates a response based on that information in a user-friendly format. For example, it might generate a response like this:

[1400] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[1401] The input is the relevant technical document and the output is the generated response sentence.

[1402] Step 6:

[1403] server

[1404] The generated response is received and converted to JSON format, for example:

[1405] json

[1406] {

[1407] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[1408] }

[1409] This JSON data is sent to the terminal as an HTTP response. The input is the generated response text, and the output is the JSON formatted response data.

[1410] Step 7:

[1411] Terminal

[1412] The device receives the response sent from the server. It parses the received JSON data and displays it on the screen in a format that is easy for the user to understand. For example, it might look like this:

[1413] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[1414] The input is the response data in JSON format, and the output is the answer that is displayed to the user.

[1415] At each step, the user, terminal, server, and generative artificial intelligence work together to efficiently provide appropriate information in response to the user's question.

[1416] (Application example 1)

[1417] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1418] In modern manufacturing, the introduction of factory robots is increasing, but when complex settings and operations or unexpected failures occur, quick and accurate troubleshooting is required. With current systems, it is difficult for operators to obtain immediate solutions when they encounter problems, which can have a negative impact on efficiency and productivity. To solve this problem, a system that provides accurate information in real time and enables rapid response is needed.

[1419] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1420] In this invention, the server includes means for receiving question data from a user's information processing device, means for transmitting the question data to a generative artificial intelligence, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for transmitting the generated response sentence to the user's information processing device, means for displaying the response sentence on the user's information processing device, means for the generative artificial intelligence to use natural language processing when analyzing the question content, means for collecting related log data and analyzing the cause of the failure when the system receives a question about a failure, means for generating a response sentence proposing countermeasures for the failure, and means for transmitting the generated response sentence to the user's information processing device and displaying it. This allows an operator to obtain information for solving problems in real time, enabling fast and accurate troubleshooting.

[1421] A "user's information processing device" is a device that allows a user to input questions and check responses, and includes a smartphone, smart glasses, a tablet, a personal computer, etc.

[1422] "Question data" refers to data on the content of a question entered by a user via an information processing device, and is sent to the server in a data format such as JSON.

[1423] "Generative AI" is AI that analyzes input questions and generates appropriate responses, and includes, for example, models that use natural language processing technology.

[1424] "Natural language processing" is a general term for technologies that allow computers to understand, analyze, and generate human language, and is a technology that enables the analysis of text data and understanding of intent.

[1425] A "response sentence" is a response sentence generated by a generative artificial intelligence in response to a user's question, and is text data used to provide the user with appropriate information and countermeasures.

[1426] "Technical documentation" means documentation that contains information about a specific technology or operating method, including manuals, technical reports, online help, etc.

[1427] "Log data" is data that records the operating status and error information of systems and applications, and is used to analyze the cause of failures.

[1428] A "failure" refers to a state in which a system or application does not function properly, and is a problem that requires a solution based on a question from a user.

[1429] "Troubleshooting" refers to a set of steps for identifying, analyzing, and correcting problems with a system or application.

[1430] The present invention relates to a system that allows factory robot operators to post questions in real time about problems or malfunctions they are facing, and uses generative artificial intelligence to instantly analyze the questions and provide appropriate solutions. This system enables operators to respond quickly on-site, particularly by using information processing devices such as smartphones and smart glasses.

[1431] Hardware and Software Overview

[1432] Hardware:

[1433] Information processing devices: smartphones, smart glasses, tablets, PCs, etc.

[1434] server

[1435] software:

[1436] Programming language: Python

[1437] Communication library: requests

[1438] Generative AI: For example, OpenAI's GPT-4

[1439] Natural language processing technology

[1440] System operation explanation

[1441] 1. Enter and submit your question:

[1442] An operator inputs questions about the operation of a factory robot or problems into an information processing device (e.g., a smartphone). For example, the following questions are input:

[1443] My robot suddenly stopped working, what should I do?

[1444] When the user clicks the submit button, the question data is converted to JSON format and sent to the server as an HTTPS request.

[1445] 2. Server query reception and analysis:

[1446] The server receives the question data sent from the information processing device. The received data is passed to generative AI, which uses natural language processing to analyze the question and understand its intent. This analysis extracts relevant keywords and context, and searches for appropriate technical documents.

[1447] 3. Search for and extract information from relevant technical documents:

[1448] The server runs a database query to find relevant technical documents. Generative AI evaluates the search results and extracts the most relevant information. For example, if a robot stopped working because of a power supply problem and the appropriate solution is to check the power cable, a response based on that information is generated.

[1449] 4. Generate and send the response:

[1450] Based on the extracted information, the generative AI generates a response in a format that is easy for the user to understand. The generated response is then converted back to JSON format and sent to the information processing device. For example, the following response may be generated:

[1451] My robot suddenly stopped working, what should I do?

[1452] This may be due to a power supply problem. Check the power cable.

[1453] 5. Displaying the response sentence on the information processing device:

[1454] The information processing device displays the received response to the user, allowing the operator to check the corrective action in real time and quickly resolve the problem.

[1455] As a specific example, if a user inputs the question "My robot suddenly stopped, what should I do?", the server analyzes the question and, if it determines that the problem is due to a power supply problem, generates a response saying "Check the power cable" and displays it to the user.

[1456] This allows operators to respond quickly on-site and minimizes downtime for factory robots, contributing to improved efficiency and productivity in the manufacturing industry.

[1457] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1458] Step 1:

[1459] A user inputs a question using an information processing device. The input question is converted into JSON format data. For example, if the question "My robot suddenly stopped, what should I do?" is input, the resulting JSON data will be as follows:

[1460] Input: [user question]

[1461] Output: {"question": "My robot suddenly stopped, what should I do?"}

[1462] Specific actions: The user uses a smartphone or smart glasses to enter a question and click the send button.

[1463] Step 2:

[1464] The device sends the question data to the server as an HTTPS request. The question data is sent in JSON format.

[1465] Input: {"question": "My robot suddenly stopped, what should I do?"}

[1466] Output: HTTPS request

[1467] Specific operation: By clicking the send button, the terminal sends the question data to the server.

[1468] Step 3:

[1469] The server receives the question data and passes it to the generative AI, which analyzes the question and understands its intent.

[1470] Input: HTTPS request

[1471] Output: Analysis results that understand the intent of the question (e.g., keywords such as "power supply," "stop," and "measures")

[1472] Specific operation: The server passes the question data to the analysis module, and a generative artificial intelligence (such as GPT-4) analyzes it and extracts keywords.

[1473] Step 4:

[1474] The server searches for relevant technical documents, and executes a database query based on the extracted keywords to find relevant documents.

[1475] Input: Keywords of analysis results

[1476] Output: List of related technical documents

[1477] Specific operation: The server searches the database for appropriate technical documents and generates a list of relevant documents.

[1478] Step 5:

[1479] Generative artificial intelligence extracts relevant technical information from the search results and generates a response.

[1480] Input: List of relevant technical documents

[1481] Output: Response (e.g. "This may be due to a problem with the power supply. Please check the power cable.")

[1482] Specific operation: The generative AI evaluates relevant technical documents and generates a response to the user based on the most appropriate information.

[1483] Step 6:

[1484] The server converts the generated response text into JSON format and sends it to the terminal.

[1485] Input: Response

[1486] Output: {"response": "This may be due to a power supply problem. Please check the power cable."}

[1487] Specific operation: The server converts the response text into JSON format and sends it to the terminal as an HTTPS request.

[1488] Step 7:

[1489] The terminal receives the response from the server and displays it to the user.

[1490] Input: {"response": "This may be due to a power supply problem. Please check the power cable."}

[1491] Output: What is displayed to the user

[1492] Specific operation: The terminal displays the received response on the screen, and the user can check it and take prompt action.

[1493] Using generative AI models and prompts, these steps work together to provide real-time, highly accurate information.

[1494] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1495] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[1496] overview

[1497] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[1498] Enter and submit questions and emotion data

[1499] User

[1500] A user enters a question about a specific function or operation method (e.g., "How do I create a graph in Excel?") into a web form and clicks the submit button. At the same time, the system analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data.

[1501] Terminal

[1502] The user's device converts the entered question and emotion data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[1503] {

[1504] "question": "How to create a graph in Excel?",

[1505] "emotion": "curious"

[1506] }

[1507] Receiving and analyzing questions and emotion data

[1508] server

[1509] The server receives the question data and emotion data sent from the user's terminal.

[1510] server

[1511] The server passes the received question data to the generative artificial intelligence and passes the emotion data to the emotion engine.

[1512] Generative Artificial Intelligence

[1513] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[1514] Emotion Engine

[1515] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[1516] Related document search and information extraction

[1517] server

[1518] The server runs a database query to find relevant technical documents, generating a list of hit documents that are then evaluated by a generative artificial intelligence.

[1519] Generative Artificial Intelligence

[1520] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[1521] Response generation and emotional reflection

[1522] Generative Artificial Intelligence

[1523] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1524] Emotion Engine

[1525] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[1526] Sending and displaying responses

[1527] server

[1528] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[1529] {

[1530] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[1531] }

[1532] Terminal

[1533] The user's terminal receives the response from the server and displays it on the screen.

[1534] Questions about disabilities and solutions

[1535] User

[1536] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[1537] Terminal

[1538] The user's device sends question data and emotion data in JSON format to the server.

[1539] server

[1540] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[1541] Generative Artificial Intelligence

[1542] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[1543] Emotion Engine

[1544] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1545] Generative Artificial Intelligence

[1546] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[1547] Emotion Engine

[1548] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[1549] server

[1550] The generated response sentence is sent to the user's terminal.

[1551] Terminal

[1552] The user's terminal receives the response and displays it on the screen.

[1553] The above is an embodiment of the present invention. This system not only allows users to efficiently obtain necessary information, but also provides emotional support.

[1554] The processing flow will be explained below.

[1555] Step 1:

[1556] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[1557] Step 2:

[1558] The user's device converts the entered question, along with emotional data obtained from the user's facial expressions and tone of voice, into JSON format and sends it to the server as an HTTPS request. For example, it will look like this:

[1559] {

[1560] "question": "How to create a graph in Excel?",

[1561] "emotion": "curious"

[1562] }

[1563] Step 3:

[1564] The server receives the question data and emotion data sent from the user's terminal.

[1565] Step 4:

[1566] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[1567] Step 5:

[1568] Generative AI uses natural language processing to analyze the question and extract key keywords, such as "Excel," "graph," "create," and "how."

[1569] Step 6:

[1570] The emotion engine analyzes the emotion data to identify the user's emotional state, for example, the emotion "curious."

[1571] Step 7:

[1572] The server queries the database to find relevant technical documents and generates a list of documents that are found as search results.

[1573] Step 8:

[1574] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[1575] Step 9:

[1576] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1577] Step 10:

[1578] The emotion engine adjusts the response depending on the user's emotional state, for example adding a sentence that reflects the emotion "Interesting!"

[1579] Step 11:

[1580] The server converts the generated response into JSON format and sends it to the user's device, for example, in the following format:

[1581] {

[1582] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[1583] }

[1584] Step 12:

[1585] The user's terminal receives the response from the server and displays it on the screen.

[1586] Step 13:

[1587] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[1588] Step 14:

[1589] The user's device converts the question data and emotion data into JSON format and sends it to the server.

[1590] Step 15:

[1591] The server receives question data and emotion data about the problem transmitted from the user's terminal.

[1592] Step 16:

[1593] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[1594] Step 17:

[1595] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[1596] Step 18:

[1597] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1598] Step 19:

[1599] The generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[1600] Step 20:

[1601] The emotion engine adjusts the response sentence depending on the user's emotional state, for example adding a sentence that reflects the emotion, "Don't worry, it's okay!"

[1602] Step 21:

[1603] The server sends the generated response to the user's terminal.

[1604] Step 22:

[1605] The user's terminal receives the response from the server and displays it on the screen.

[1606] Example 2

[1607] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1608] Current question-answering systems often only provide information without considering the user's feelings. This leaves the user experience unsatisfactory, making it difficult to alleviate user anxiety and stress, especially when dealing with problems. Furthermore, conventional systems often fail to provide accurate responses because they do not perform detailed analysis of questions or problems.

[1609] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1610] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for the emotion engine to analyze the emotion data and identify the user's emotion, means for searching relevant databases and technical documents, means for extracting the most relevant information from the search results and for the generative artificial intelligence to generate a response sentence, means for the emotion engine to reflect the user's emotion in the response sentence, and means for transmitting the generated response sentence to the user's terminal. This enables an appropriate and friendly response that takes the user's emotion into consideration.

[1611] A "user's terminal" is a computer device that allows a user to input and send a question, and includes a personal computer, a smartphone, or the like.

[1612] "Question data" is information relating to the content of an inquiry that a user inputs and sends via a terminal.

[1613] "Emotion data" is information about emotions extracted from the user's facial expressions, voice, etc.

[1614] "Generative AI" is an AI system that has the ability to analyze input data and generate answers.

[1615] An "emotion engine" is a system that analyzes emotional data collected from users and recognizes specific emotions.

[1616] "Natural language processing" is a technique used by generative artificial intelligence to understand the content of questions, and is a technology that analyzes natural language and extracts meaning.

[1617] A "database" is an information repository where technical documents and related information are stored.

[1618] A "technical document" is a document that contains detailed information about a particular technology and is used to answer user questions.

[1619] A "response sentence" is an answer generated by generative artificial intelligence and provided to the user.

[1620] "Log data" is data that records the operation history of a system or application.

[1621] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and suggest countermeasures. An embodiment of the present invention is described in detail below.

[1622] overview

[1623] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[1624] Hardware and software used

[1625] User

[1626] Users use devices such as PCs and smartphones that have a web browser installed, allowing them to enter questions via a web form.

[1627] Terminal

[1628] The terminal receives input from the user, converts question data and emotion data into JSON format, and sends it to the server.

[1629] server

[1630] The server utilizes cloud infrastructure, such as AWS EC2 instances, and uses generative artificial intelligence (e.g., OpenAI GPT-4) and emotion engines (e.g., Microsoft Azure Face API) to analyze and process the incoming data.

[1631] Generative Artificial Intelligence

[1632] Generative AI uses natural language processing to analyze the input question and generate a response based on related technical documentation.

[1633] Emotion Engine

[1634] The emotion engine acquires and analyzes emotion data from the user's facial expressions and tone of voice.

[1635] Enter and submit questions and emotion data

[1636] User

[1637] A user enters a question into a web form, such as "How do I create a graph in Excel?", and clicks the submit button. At this time, the system analyzes the user's facial expressions and tone of voice to obtain emotional data (e.g., "curious").

[1638] Receiving and analyzing questions and emotion data

[1639] server

[1640] The server receives the question data and emotion data sent from the user's device, passes the received question data to the generative AI, and passes the emotion data to the emotion engine.

[1641] Generative Artificial Intelligence

[1642] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[1643] Emotion Engine

[1644] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[1645] Related document search and information extraction

[1646] server

[1647] The server queries the database to find relevant technical documents, and the resulting list of documents is passed to the generative AI.

[1648] Generative Artificial Intelligence

[1649] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[1650] Response generation and emotional reflection

[1651] Generative Artificial Intelligence

[1652] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1653] Emotion Engine

[1654] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[1655] Sending and displaying responses

[1656] server

[1657] The generated response is converted to JSON format and sent to the user's device. For example, it is sent in the following format:

[1658] json

[1659] {

[1660] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[1661] }

[1662] Terminal

[1663] The terminal receives the response from the server and displays it on the screen.

[1664] Questions about disabilities and solutions

[1665] User

[1666] A user posts a question like, "What causes my application to crash?"

[1667] Terminal

[1668] The device sends question data and emotion data in JSON format to the server.

[1669] server

[1670] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[1671] Generative Artificial Intelligence

[1672] Generative artificial intelligence analyzes the question content and collects and analyzes related log data (system logs and error logs).

[1673] Emotion Engine

[1674] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1675] Generative Artificial Intelligence

[1676] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, if the cause of the problem is identified as "insufficient memory," it generates a response such as "Please close unnecessary applications or consider increasing memory."

[1677] Emotion Engine

[1678] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[1679] server

[1680] The generated response sentence is sent to the user's terminal.

[1681] Terminal

[1682] The terminal receives the response and displays it on the screen.

[1683] As described above, the system of the present invention not only allows users to efficiently obtain the information they need, but also provides emotional support. For example, in response to the question, "How do I create a graph in Excel?", the system provides the following response: "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the type of graph you want from the 'Chart' group. Interesting!" In this way, the system provides users with prompt and appropriate information tailored to their needs.

[1684] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1685] Processing flow

[1686] Step 1:

[1687] A user enters a question about a specific function or operation method into a web form and clicks the submit button. For example, the input might be, "How do I create a graph in Excel?" At this time, the user's facial expression and tone of voice are sent to the device as emotional data.

[1688] Step 2:

[1689] The device converts the input question and emotion data into JSON format and sends it to the server as an HTTPS request. The input data is in the following format:

[1690] json

[1691] {

[1692] "question": "How to create a graph in Excel?",

[1693] "emotion": "curious"

[1694] }

[1695] Step 3:

[1696] The server receives question data and emotion data sent from the user's device, processes the received data, and passes it to the generative AI and emotion engine in an appropriate format.

[1697] Step 4:

[1698] Generative AI analyzes question data. It uses natural language processing (NLP) to understand the question and extract key keywords. The input data is the question data, and the output data is the extracted keywords (e.g., "Excel," "graph," "create," "how to").

[1699] Step 5:

[1700] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "interesting").

[1701] Step 6:

[1702] The server executes a database query to search for relevant technical documents, with the input data being the extracted keywords and the output data being a list of hit technical documents.

[1703] Step 7:

[1704] Generative AI evaluates search results and extracts the most relevant information. The input data is a list of technical documents, and the output data is the extracted technical information.

[1705] Step 8:

[1706] Based on the information extracted by the generative artificial intelligence, a response is generated in a format that is easy for the user to understand. Specifically, the response generated is, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group." The input data is the extracted technical information, and the output data is the generated response.

[1707] Step 9:

[1708] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Interesting!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[1709] Step 10:

[1710] The server converts the generated response into JSON format and sends it to the user's device, specifically in the following format:

[1711] json

[1712] {

[1713] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[1714] }

[1715] Step 11:

[1716] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[1717] Inquiry about disabilities and provision of solutions

[1718] Step 12:

[1719] A user posts a question about a specific problem or error. For example, "What causes my application to crash?"

[1720] Step 13:

[1721] The device sends question data and emotion data in JSON format to the server. The input data is in the following format:

[1722] json

[1723] {

[1724] "question": "What causes my application to crash?",

[1725] "emotion": "worried"

[1726] }

[1727] Step 14:

[1728] The server receives the question data and emotion data and passes them to the generative AI and emotion engine. The server processes the received data and sends it in the appropriate format.

[1729] Step 15:

[1730] The generative AI analyzes the question content and collects and analyzes related log data (e.g., system logs, error logs). The input data is the question data, and the output data is the analyzed log data and the extracted cause of the failure.

[1731] Step 16:

[1732] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "worry").

[1733] Step 17:

[1734] The generative AI analyzes the cause of the failure and countermeasures, and generates a response. Specifically, it identifies the cause of the failure as "insufficient memory," and generates a response saying, "Please close unnecessary applications or consider increasing memory." The input data is the analyzed log data and the extracted cause of the failure, and the output data is the generated response.

[1735] Step 18:

[1736] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Don't worry, everything is fine!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[1737] Step 19:

[1738] The server converts the generated response text into JSON format and sends it to the user's device. The input data is the generated response text, and the output data is the JSON format data sent to the user's device.

[1739] Step 20:

[1740] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[1741] This is the specific processing flow of this system, which allows users to efficiently obtain information while also receiving emotional support.

[1742] (Application example 2)

[1743] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1744] Many users today are facing serious security-related problems. In these circumstances, conventional automated response systems must not only provide accurate information in response to users' questions and problems, but also provide emotional support. However, current systems lack emotional consideration, making it difficult for users who are anxious or nervous to feel at ease. Furthermore, there is a need for systems that can respond to users' emotions rather than simply providing technical information. A system that can appropriately address these issues is needed.

[1745] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1746] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for emotionally adjusting the response sentence based on the emotion data analyzed by the emotion engine, and means for transmitting the generated response sentence to the user's terminal. This makes it possible to provide not only accurate information in response to a user's questions or problems, but also emotional support.

[1747] "Question data" is text information about questions or problems that users input to the system.

[1748] "Emotion data" refers to emotion information extracted from the user's facial expression, tone of voice, and text content.

[1749] "Generative AI" is an AI system that uses natural language processing technology to analyze the content of a user's question, understand their intent, and generate an appropriate response.

[1750] An "emotion engine" is a system that analyzes a user's emotional data and generates a response that takes those emotions into consideration.

[1751] A "user's terminal" is an information device such as a smartphone or a personal computer used by a user.

[1752] The "server" is an information processing device that relays between the user's device and the generative AI and emotion engine, receiving and sending data and generating responses.

[1753] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[1754] "Technical documents" are documents that contain relevant technical information and are the subject of searches by the server.

[1755] "Log data" refers to data that records the operation history and error information of systems and applications.

[1756] A "response sentence" is an answer to a user's question generated by a generative artificial intelligence, and is a sentence adjusted by an emotion engine.

[1757] The present invention relates to a system that receives question data and emotion data from a user's device and uses generative artificial intelligence and an emotion engine to provide appropriate and emotionally sensitive responses. This system is particularly effective in responding to security-related questions and problems.

[1758] System Program Overview

[1759] 1. Receiving user questions and emotion data

[1760] The user enters a question in text and sends it from the terminal.

[1761] Emotional data is obtained from the user's facial expressions and tone of voice, and transmitted from the device.

[1762] 2. Question and Sentiment Data Analysis

[1763] A server receives the question data and the emotion data.

[1764] Generative AI analyzes the question using natural language processing and extracts important keywords.

[1765] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1766] 3. Technical Document Search and Response Generation

[1767] The server searches the database for relevant technical documents and retrieves the most relevant information.

[1768] Generative artificial intelligence generates appropriate responses from technical documents.

[1769] The emotion engine adjusts the response sentence according to the user's emotion.

[1770] 4. Sending a response

[1771] The server sends the final response to the terminal and displays it to the user.

[1772] Implementation hardware and software

[1773] Hardware

[1774] User's device: Information devices such as smartphones and PCs

[1775] Server: A high-performance server for data processing and analysis

[1776] software

[1777] Natural language processing technology: Google Cloud Natural Language API, IBM Watson, etc.

[1778] Sentiment analysis technology: Microsoft Azure Emotion API, Amazon Rekognition, etc.

[1779] Database: Relational database such as MySQL or PostgreSQL

[1780] Generative AI: GPT-4 and similar generative AI models

[1781] Example of system operation

[1782] Entering questions and sentiment data

[1783] The user types the question "Is this network safe?" and sends it from the device. At the same time, emotional data such as "anxiety" is obtained from the user's facial expression and tone of voice.

[1784] analysis

[1785] The server receives this data, and the generative AI extracts keywords such as "network" and "safety," while the emotion engine identifies the emotion "anxiety."

[1786] Response generation and adjustment

[1787] The generative AI generates a technical response such as "The current network is encrypted," while the emotion engine adds an emotional element: "Don't worry, we'll follow up."

[1788] Sending a response

[1789] The final response, "The current network is encrypted. Don't worry, we will follow up, so don't worry," is sent to the user's device and displayed.

[1790] Prompt Sentence Examples

[1791] "Is my network secure?"

[1792] "I think I might have the virus, what should I do?"

[1793] "How can I reduce the risk of my password being hacked?"

[1794] This can alleviate the user's doubts and anxieties and provide a sense of security.

[1795] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1796] Step 1:

[1797] The user inputs and sends question data and emotion data from the terminal.

[1798] Input: The user enters the question "Is this network safe?" and emotional data of "anxiety" is obtained from facial expressions and tone of voice.

[1799] Processing: The device converts this data into JSON format and sends it to the server as an HTTPS request.

[1800] Output: Question data and emotion data are sent to the server.

[1801] Step 2:

[1802] A server receives the question data and the emotion data.

[1803] Input: Question data and emotion data sent from the device

[1804] Processing: The server passes the received data to the generative artificial intelligence and emotion engine for analysis.

[1805] Output: Question data passed to the generative AI, emotion data passed to the emotion engine

[1806] Step 3:

[1807] Generative AI analyzes the content of the question and understands its intent.

[1808] Input: Query data passed from the server

[1809] Processing: Using natural language processing technology (e.g., Google Cloud Natural Language API), important keywords are extracted from the question.

[1810] Data processing and calculation: For example, extracting keywords such as "network" and "safety."

[1811] Output: Parsed question content and extracted keywords

[1812] Step 4:

[1813] The emotion engine analyzes the emotion data and identifies the user's emotion.

[1814] Input: Emotion data passed from the server

[1815] Processing: Analyze the emotion data using emotion analysis technology (e.g., Microsoft Azure Emotion API).

[1816] Data processing and calculation: Identify the user's emotion as "anxiety."

[1817] Output: Analyzed user emotion information

[1818] Step 5:

[1819] The server searches the database for relevant technical documents and retrieves the most relevant information.

[1820] Input: Keywords passed from the generative AI

[1821] Processing: Executes database queries and retrieves technical documents.

[1822] Data processing and calculation: Narrow down relevant documents based on keywords.

[1823] Output: Search results for a list of technical documents

[1824] Step 6:

[1825] Generative artificial intelligence generates appropriate responses from technical documents.

[1826] Input: List of technical documents and questions

[1827] Processing: Generate the most appropriate response sentence based on the technical documentation.

[1828] Data processing and calculation: For example, generating a technical response such as "The current network is encrypted."

[1829] Output: Response

[1830] Step 7:

[1831] The emotion engine adjusts the response sentence according to the user's emotion.

[1832] Input: Response sentences from generative AI and analyzed emotional information

[1833] Processing: Emotionally adjusting the response sentence.

[1834] Data processing and calculation: For example, adding an emotional element such as "Don't worry, we will follow up, so don't worry."

[1835] Output: Emotionally tailored response sentence

[1836] Step 8:

[1837] The server sends the final response to the terminal and displays it to the user.

[1838] Input: Emotionally tailored response sentences

[1839] Processing: Convert the response into JSON format and send it to the terminal.

[1840] Output: The final response sent to the terminal.

[1841] These processing steps allow users to receive technically accurate and emotionally sensitive responses to their questions and concerns in real time.

[1842] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1843] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1844] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1845] [Fourth embodiment]

[1846] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1847] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1848] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1849] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1850] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1851] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1852] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1853] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1854] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1855] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1856] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1857] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1858] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1859] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[1860] overview

[1861] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data into JSON format and sends it to the server as an HTTPS request. The server receives the question data and passes it to a generative AI. The generative AI analyzes the question and searches for related technical documents. It then extracts the most relevant information from the search results, generates a response, and sends it to the user's device.

[1862] Enter and submit your question

[1863] User

[1864] A user fills out a web form with a question about a specific feature or procedure (e.g., "How do I create a chart in Excel?") and clicks the submit button.

[1865] Terminal

[1866] The user's device converts the entered question into JSON format and sends it to the server as an HTTPS request, for example, in the following format:

[1867] {

[1868] "question": "How do I create a graph in Excel?"

[1869] }

[1870] Receiving and parsing questions

[1871] server

[1872] The server receives question data sent from the user's device. The received question data is then sent to the generative AI. The generative AI analyzes the question using natural language processing and extracts keywords. For example, keywords such as "Excel," "graph," "create," and "method" are extracted.

[1873] Related document search and information extraction

[1874] server

[1875] The server runs a database query to find relevant technical documents, generates a list of hit documents, and a generative AI evaluates this list, extracts the most relevant information, and generates a response.

[1876] Generate and send a response

[1877] Generative Artificial Intelligence

[1878] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1879] server

[1880] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[1881] {

[1882] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[1883] }

[1884] Terminal

[1885] The user's terminal receives the response from the server and displays it on the screen.

[1886] Questions about disabilities and solutions

[1887] User

[1888] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[1889] Terminal

[1890] The user's terminal transmits the question data to the server.

[1891] server

[1892] The server receives the question and passes it to the generative AI, which analyzes the question and collects relevant log data. For example, it analyzes system logs and error logs to identify the cause of the problem, which is "insufficient memory."

[1893] Generative Artificial Intelligence

[1894] AI analyzes the cause of the failure and countermeasures based on the log data, and generates a response message with the countermeasures. For example, the response message generated might read, "The cause of the application crash is likely a lack of memory. To resolve this, close any unnecessary applications or consider increasing the memory."

[1895] server

[1896] The generated response sentence is sent to the user's terminal.

[1897] Terminal

[1898] The user's terminal displays the response.

[1899] In this way, the system of the present invention is designed to enable users to efficiently obtain the information they need, and provides specific operating procedures and troubleshooting tips quickly and accurately.

[1900] The processing flow will be explained below.

[1901] Step 1:

[1902] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[1903] Step 2:

[1904] The device converts the question data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[1905] {

[1906] "question": "How do I create a graph in Excel?"

[1907] }

[1908] Step 3:

[1909] The server receives the question data sent from the user's terminal.

[1910] Step 4:

[1911] The server passes the received question data to the generative artificial intelligence.

[1912] Step 5:

[1913] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[1914] Step 6:

[1915] The server queries the database to find relevant technical documents, and generates a list of documents that are hit by the search.

[1916] Step 7:

[1917] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[1918] Step 8:

[1919] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[1920] Step 9:

[1921] The server converts the generated response into JSON format and sends it to the user's device. For example, it will be sent in the following format:

[1922] {

[1923] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and select the chart type you want from the 'Charts' group."

[1924] }

[1925] Step 10:

[1926] The user's terminal receives the response from the server and displays it on the screen.

[1927] Step 11:

[1928] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[1929] Step 12:

[1930] The device converts the question data into JSON format and sends it to the server.

[1931] Step 13:

[1932] The server receives question data about the problem sent from the user's terminal.

[1933] Step 14:

[1934] The server passes the received question data to the generative artificial intelligence.

[1935] Step 15:

[1936] Generative AI uses natural language processing to analyze the question and collect relevant log data, such as analyzing patterns in system logs and error logs.

[1937] Step 16:

[1938] Based on the log data collected by the generative AI, the cause of the failure is analyzed and a likely solution is generated. For example, the cause of the failure can be identified as "insufficient memory."

[1939] Step 17:

[1940] The generative AI generates a response with a solution, such as, "The application crash may be caused by insufficient memory. To solve this problem, close any unnecessary applications or consider increasing the memory."

[1941] Step 18:

[1942] The server sends the generated response to the user's terminal.

[1943] Step 19:

[1944] The user's terminal receives the response from the server and displays it on the screen.

[1945] The specific processing steps of the entire system have been explained above. At each step, the aim is to provide information to the user efficiently and accurately.

[1946] Example 1

[1947] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1948] Conventional information search systems have the problem that it is difficult for users to quickly obtain appropriate information on specific operation methods or troubleshooting. It is particularly difficult to provide appropriate information immediately for questions that require specialized knowledge. Furthermore, when a malfunction occurs, it is not possible to identify the cause and provide appropriate troubleshooting. This often results in a waste of time and effort on the part of the user.

[1949] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1950] In this invention, the server includes a means for receiving question data from a user's device, a means for transmitting the received question data to a generative artificial intelligence, and a means for the generative artificial intelligence to analyze the question content and understand the intent of the question. This enables users to quickly obtain high-quality information. Furthermore, the server includes a means for searching for related technical documents, a means for extracting the most relevant information from the search results and generating a response, a means for transmitting the generated response to the user's device, a means for converting the question into JSON format on the user's device and transmitting it to the server, and a means for displaying the received response on the user's device. This enables users to quickly and accurately resolve their questions and improve convenience. Furthermore, when the system receives a question about a failure, the server includes a means for collecting related log data, analyzing the cause of the failure, and generating a response that addresses the failure. This enables quick identification of the cause and presentation of a solution when a failure occurs, improving system reliability and user satisfaction.

[1951] A "user" is someone who uses the system to enter questions and obtain information.

[1952] A "terminal" is an electronic device, such as a computer or smartphone, that a user uses to input a question.

[1953] "Question data" refers to data including the question entered by the user via the terminal and sent to the server.

[1954] A "server" is a device that processes question data received from a user's terminal and passes it on to the generative artificial intelligence.

[1955] "Generative AI" is an AI model that analyzes received question data, understands the intent of the question, and generates appropriate information.

[1956] "Natural language processing" is a technology used by generative artificial intelligence to understand and process text data when analyzing question content.

[1957] "Technical documentation" means documentation that contains information about a particular technology.

[1958] A "response sentence" is an answer to a user's question generated by generative artificial intelligence.

[1959] "Log data" refers to data that records the system's operating status and error information.

[1960] A "failure" is an error or problem that prevents the system from functioning properly.

[1961] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence to provide appropriate information. Furthermore, the system can analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[1962] Hardware and software configuration

[1963] User

[1964] Users use devices such as personal computers, tablets, and smartphones. These devices are connected to the Internet and display a web form for submitting questions through a web browser. Users then enter their questions and click the submit button.

[1965] Terminal

[1966] The device receives questions entered by users, converts the received questions into JSON format, and sends them to the server as an HTTPS request. The device operates using a browser or a dedicated application.

[1967] server

[1968] The server receives question data sent from the user's device. It then passes the received question data to a generative AI for analysis. This generative AI includes a model capable of natural language processing (e.g., OpenAI's GPT-3). The server converts the response received from the generative AI into JSON format and sends it to the user's device.

[1969] Data processing and calculation

[1970] Generative Artificial Intelligence

[1971] Generative AI analyzes the received question data. It uses natural language processing technology to understand the intent of the question and extract the necessary keywords. Based on the extracted keywords, it searches for related technical documents from a database. It extracts the most relevant information and generates an appropriate response based on that. For example, it uses a prompt such as, "Please tell me how to create a graph in Excel."

[1972] Specific examples

[1973] Entering and processing questions

[1974] For example, a user enters the question "How do I create a graph in Excel?" into a web form and submits it. The device converts this question into JSON format and sends it to the server, like this:

[1975] json

[1976] {

[1977] "question": "How do I create a graph in Excel?"

[1978] }

[1979] The server receives this data and passes it to the generative AI, which analyzes the question and generates a response like this:

[1980] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[1981] The generated response is returned to the server, which converts it to JSON format and sends it to the user's device, which receives it and displays it to the user.

[1982] Questions and handling of obstacles

[1983] For example, suppose a user posts a question such as "What causes my application to crash?" This question is also sent by the device to the server, where it is analyzed by the generative AI. Relevant log data is collected and it is determined that the cause is insufficient memory. A response is generated stating, "Insufficient memory is likely the cause of the application crash. To resolve this, please close unnecessary applications or consider increasing memory," and is provided to the user using the same procedure.

[1984] This system allows users to efficiently obtain the necessary information and quickly receive fault analysis and countermeasures.

[1985] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1986] Processing flow and specific operations at each step

[1987] Step 1:

[1988] User

[1989] A user fills out a web form with a question about a particular feature or operation and clicks the submit button. The input is in the form of text, such as "How do I create a chart in Excel?"

[1990] Step 2:

[1991] Terminal

[1992] The device receives the question entered by the user and converts it into JSON format. For example, it is converted as follows:

[1993] json

[1994] {

[1995] "question": "How do I create a graph in Excel?"

[1996] }

[1997] This JSON data is sent to the server as an HTTPS request.

[1998] Step 3:

[1999] server

[2000] The server receives question data sent from the device. It prepares the received JSON data to be passed to the generative AI. The input is the question data in JSON format, and the output is a prompt text for the generative AI. As a concrete example of the process, the data is converted as follows:

[2001] "Prompt: How do I create a graph in Excel?"

[2002] Step 4:

[2003] Generative Artificial Intelligence

[2004] A generative artificial intelligence analyzes the question based on the received prompt. The technology used is natural language processing. As a result of the analysis, keywords are extracted. For example, keywords such as "Excel," "graph," "create," and "how to" are extracted. Technical documents are searched based on these keywords. The input is the prompt and a database of technical documents, and the output is a set of related technical documents and information.

[2005] Step 5:

[2006] Generative Artificial Intelligence

[2007] It evaluates relevant technical documentation, extracts the most relevant information, and generates a response based on that information in a user-friendly format. For example, it might generate a response like this:

[2008] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[2009] The input is the relevant technical document and the output is the generated response sentence.

[2010] Step 6:

[2011] server

[2012] The generated response is received and converted to JSON format, for example:

[2013] json

[2014] {

[2015] "response": "To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and choose the chart type you want from the "Charts" group."

[2016] }

[2017] This JSON data is sent to the terminal as an HTTP response. The input is the generated response text, and the output is the JSON formatted response data.

[2018] Step 7:

[2019] Terminal

[2020] The device receives the response sent from the server. It parses the received JSON data and displays it on the screen in a format that is easy for the user to understand. For example, it might look like this:

[2021] To create a chart in Excel, select a data range, click the "Insert" tab in the top menu, and select the chart type you want from the "Charts" group.

[2022] The input is the response data in JSON format, and the output is the answer that is displayed to the user.

[2023] At each step, the user, terminal, server, and generative artificial intelligence work together to efficiently provide appropriate information in response to the user's question.

[2024] (Application example 1)

[2025] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2026] In modern manufacturing, the introduction of factory robots is increasing, but when complex settings and operations or unexpected failures occur, quick and accurate troubleshooting is required. With current systems, it is difficult for operators to obtain immediate solutions when they encounter problems, which can have a negative impact on efficiency and productivity. To solve this problem, a system that provides accurate information in real time and enables rapid response is needed.

[2027] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2028] In this invention, the server includes means for receiving question data from a user's information processing device, means for transmitting the question data to a generative artificial intelligence, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for transmitting the generated response sentence to the user's information processing device, means for displaying the response sentence on the user's information processing device, means for the generative artificial intelligence to use natural language processing when analyzing the question content, means for collecting related log data and analyzing the cause of the failure when the system receives a question about a failure, means for generating a response sentence proposing countermeasures for the failure, and means for transmitting the generated response sentence to the user's information processing device and displaying it. This allows an operator to obtain information for solving problems in real time, enabling fast and accurate troubleshooting.

[2029] A "user's information processing device" is a device that allows a user to input questions and check responses, and includes a smartphone, smart glasses, a tablet, a personal computer, etc.

[2030] "Question data" refers to data on the content of a question entered by a user via an information processing device, and is sent to the server in a data format such as JSON.

[2031] "Generative AI" is AI that analyzes input questions and generates appropriate responses, and includes, for example, models that use natural language processing technology.

[2032] "Natural language processing" is a general term for technologies that allow computers to understand, analyze, and generate human language, and is a technology that enables the analysis of text data and understanding of intent.

[2033] A "response sentence" is a response sentence generated by a generative artificial intelligence in response to a user's question, and is text data used to provide the user with appropriate information and countermeasures.

[2034] "Technical documentation" means documentation that contains information about a specific technology or operating method, including manuals, technical reports, online help, etc.

[2035] "Log data" is data that records the operating status and error information of systems and applications, and is used to analyze the cause of failures.

[2036] A "failure" refers to a state in which a system or application does not function properly, and is a problem that requires a solution based on a question from a user.

[2037] "Troubleshooting" refers to a set of steps for identifying, analyzing, and correcting problems with a system or application.

[2038] The present invention relates to a system that allows factory robot operators to post questions in real time about problems or malfunctions they are facing, and uses generative artificial intelligence to instantly analyze the questions and provide appropriate solutions. This system enables operators to respond quickly on-site, particularly by using information processing devices such as smartphones and smart glasses.

[2039] Hardware and Software Overview

[2040] Hardware:

[2041] Information processing devices: smartphones, smart glasses, tablets, PCs, etc.

[2042] server

[2043] software:

[2044] Programming language: Python

[2045] Communication library: requests

[2046] Generative AI: For example, OpenAI's GPT-4

[2047] Natural language processing technology

[2048] System operation explanation

[2049] 1. Enter and submit your question:

[2050] An operator inputs questions about the operation of a factory robot or problems into an information processing device (e.g., a smartphone). For example, the following questions are input:

[2051] My robot suddenly stopped working, what should I do?

[2052] When the user clicks the submit button, the question data is converted to JSON format and sent to the server as an HTTPS request.

[2053] 2. Server query reception and analysis:

[2054] The server receives the question data sent from the information processing device. The received data is passed to generative AI, which uses natural language processing to analyze the question and understand its intent. This analysis extracts relevant keywords and context, and searches for appropriate technical documents.

[2055] 3. Search for and extract information from relevant technical documents:

[2056] The server runs a database query to find relevant technical documents. Generative AI evaluates the search results and extracts the most relevant information. For example, if a robot stopped working because of a power supply problem and the appropriate solution is to check the power cable, a response based on that information is generated.

[2057] 4. Generate and send the response:

[2058] Based on the extracted information, the generative AI generates a response in a format that is easy for the user to understand. The generated response is then converted back to JSON format and sent to the information processing device. For example, the following response may be generated:

[2059] My robot suddenly stopped working, what should I do?

[2060] This may be due to a power supply problem. Check the power cable.

[2061] 5. Displaying the response sentence on the information processing device:

[2062] The information processing device displays the received response to the user, allowing the operator to check the corrective action in real time and quickly resolve the problem.

[2063] As a specific example, if a user inputs the question "My robot suddenly stopped, what should I do?", the server analyzes the question and, if it determines that the problem is due to a power supply problem, generates a response saying "Check the power cable" and displays it to the user.

[2064] This allows operators to respond quickly on-site and minimizes downtime for factory robots, contributing to improved efficiency and productivity in the manufacturing industry.

[2065] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2066] Step 1:

[2067] A user inputs a question using an information processing device. The input question is converted into JSON format data. For example, if the question "My robot suddenly stopped, what should I do?" is input, the resulting JSON data will be as follows:

[2068] Input: [user question]

[2069] Output: {"question": "My robot suddenly stopped, what should I do?"}

[2070] Specific actions: The user uses a smartphone or smart glasses to enter a question and click the send button.

[2071] Step 2:

[2072] The device sends the question data to the server as an HTTPS request. The question data is sent in JSON format.

[2073] Input: {"question": "My robot suddenly stopped, what should I do?"}

[2074] Output: HTTPS request

[2075] Specific operation: By clicking the send button, the terminal sends the question data to the server.

[2076] Step 3:

[2077] The server receives the question data and passes it to the generative AI, which analyzes the question and understands its intent.

[2078] Input: HTTPS request

[2079] Output: Analysis results that understand the intent of the question (e.g., keywords such as "power supply," "stop," and "measures")

[2080] Specific operation: The server passes the question data to the analysis module, and a generative artificial intelligence (such as GPT-4) analyzes it and extracts keywords.

[2081] Step 4:

[2082] The server searches for relevant technical documents, and executes a database query based on the extracted keywords to find relevant documents.

[2083] Input: Keywords of analysis results

[2084] Output: List of related technical documents

[2085] Specific operation: The server searches the database for appropriate technical documents and generates a list of relevant documents.

[2086] Step 5:

[2087] Generative artificial intelligence extracts relevant technical information from the search results and generates a response.

[2088] Input: List of relevant technical documents

[2089] Output: Response (e.g. "This may be due to a problem with the power supply. Please check the power cable.")

[2090] Specific operation: The generative AI evaluates relevant technical documents and generates a response to the user based on the most appropriate information.

[2091] Step 6:

[2092] The server converts the generated response text into JSON format and sends it to the terminal.

[2093] Input: Response

[2094] Output: {"response": "This may be due to a power supply problem. Please check the power cable."}

[2095] Specific operation: The server converts the response text into JSON format and sends it to the terminal as an HTTPS request.

[2096] Step 7:

[2097] The terminal receives the response from the server and displays it to the user.

[2098] Input: {"response": "This may be due to a power supply problem. Please check the power cable."}

[2099] Output: What is displayed to the user

[2100] Specific operation: The terminal displays the received response on the screen, and the user can check it and take prompt action.

[2101] Using generative AI models and prompts, these steps work together to provide real-time, highly accurate information.

[2102] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2103] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and consider countermeasures. An embodiment of the present invention is described in detail below.

[2104] overview

[2105] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[2106] Enter and submit questions and emotion data

[2107] User

[2108] A user enters a question about a specific function or operation method (e.g., "How do I create a graph in Excel?") into a web form and clicks the submit button. At the same time, the system analyzes the user's facial expressions, tone of voice, etc. to obtain emotional data.

[2109] Terminal

[2110] The user's device converts the entered question and emotion data into JSON format and sends it to the server as an HTTPS request, for example in the following format:

[2111] {

[2112] "question": "How to create a graph in Excel?",

[2113] "emotion": "curious"

[2114] }

[2115] Receiving and analyzing questions and emotion data

[2116] server

[2117] The server receives the question data and emotion data sent from the user's terminal.

[2118] server

[2119] The server passes the received question data to the generative artificial intelligence and passes the emotion data to the emotion engine.

[2120] Generative Artificial Intelligence

[2121] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[2122] Emotion Engine

[2123] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[2124] Related document search and information extraction

[2125] server

[2126] The server runs a database query to find relevant technical documents, generating a list of hit documents that are then evaluated by a generative artificial intelligence.

[2127] Generative Artificial Intelligence

[2128] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[2129] Response generation and emotional reflection

[2130] Generative Artificial Intelligence

[2131] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[2132] Emotion Engine

[2133] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[2134] Sending and displaying responses

[2135] server

[2136] The generated response is converted to JSON format and sent to the user's device, for example, in the following format:

[2137] {

[2138] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[2139] }

[2140] Terminal

[2141] The user's terminal receives the response from the server and displays it on the screen.

[2142] Questions about disabilities and solutions

[2143] User

[2144] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[2145] Terminal

[2146] The user's device sends question data and emotion data in JSON format to the server.

[2147] server

[2148] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[2149] Generative Artificial Intelligence

[2150] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[2151] Emotion Engine

[2152] The emotion engine analyzes the emotion data and identifies the user's emotion.

[2153] Generative Artificial Intelligence

[2154] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[2155] Emotion Engine

[2156] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[2157] server

[2158] The generated response sentence is sent to the user's terminal.

[2159] Terminal

[2160] The user's terminal receives the response and displays it on the screen.

[2161] The above is an embodiment of the present invention. This system not only allows users to efficiently obtain necessary information, but also provides emotional support.

[2162] The processing flow will be explained below.

[2163] Step 1:

[2164] A user fills out a web form with a question about a particular feature or procedure and clicks submit.

[2165] Step 2:

[2166] The user's device converts the entered question, along with emotional data obtained from the user's facial expressions and tone of voice, into JSON format and sends it to the server as an HTTPS request. For example, it will look like this:

[2167] {

[2168] "question": "How to create a graph in Excel?",

[2169] "emotion": "curious"

[2170] }

[2171] Step 3:

[2172] The server receives the question data and emotion data sent from the user's terminal.

[2173] Step 4:

[2174] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[2175] Step 5:

[2176] Generative AI uses natural language processing to analyze the question and extract key keywords, such as "Excel," "graph," "create," and "how."

[2177] Step 6:

[2178] The emotion engine analyzes the emotion data to identify the user's emotional state, for example, the emotion "curious."

[2179] Step 7:

[2180] The server queries the database to find relevant technical documents and generates a list of documents that are found as search results.

[2181] Step 8:

[2182] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[2183] Step 9:

[2184] Based on the information extracted by the generative AI, a response is generated in a format that is easy for the user to understand. For example, a response might be generated that reads, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[2185] Step 10:

[2186] The emotion engine adjusts the response depending on the user's emotional state, for example adding a sentence that reflects the emotion "Interesting!"

[2187] Step 11:

[2188] The server converts the generated response into JSON format and sends it to the user's device, for example, in the following format:

[2189] {

[2190] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[2191] }

[2192] Step 12:

[2193] The user's terminal receives the response from the server and displays it on the screen.

[2194] Step 13:

[2195] A user posts a question about a specific problem or error (e.g., "What causes my application to crash?").

[2196] Step 14:

[2197] The user's device converts the question data and emotion data into JSON format and sends it to the server.

[2198] Step 15:

[2199] The server receives question data and emotion data about the problem transmitted from the user's terminal.

[2200] Step 16:

[2201] The server passes the received question data to the generative artificial intelligence and the emotion data to the emotion engine.

[2202] Step 17:

[2203] Generative AI analyzes the questions and collects relevant log data, such as patterns in system logs and error logs.

[2204] Step 18:

[2205] The emotion engine analyzes the emotion data and identifies the user's emotion.

[2206] Step 19:

[2207] The generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, it identifies the cause of the problem as "insufficient memory," and generates a response such as "Please close unnecessary applications or consider increasing memory."

[2208] Step 20:

[2209] The emotion engine adjusts the response sentence depending on the user's emotional state, for example adding a sentence that reflects the emotion, "Don't worry, it's okay!"

[2210] Step 21:

[2211] The server sends the generated response to the user's terminal.

[2212] Step 22:

[2213] The user's terminal receives the response from the server and displays it on the screen.

[2214] Example 2

[2215] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2216] Current question-answering systems often only provide information without considering the user's feelings. This leaves the user experience unsatisfactory, making it difficult to alleviate user anxiety and stress, especially when dealing with problems. Furthermore, conventional systems often fail to provide accurate responses because they do not perform detailed analysis of questions or problems.

[2217] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2218] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for the emotion engine to analyze the emotion data and identify the user's emotion, means for searching relevant databases and technical documents, means for extracting the most relevant information from the search results and for the generative artificial intelligence to generate a response sentence, means for the emotion engine to reflect the user's emotion in the response sentence, and means for transmitting the generated response sentence to the user's terminal. This enables an appropriate and friendly response that takes the user's emotion into consideration.

[2219] A "user's terminal" is a computer device that allows a user to input and send a question, and includes a personal computer, a smartphone, or the like.

[2220] "Question data" is information relating to the content of an inquiry that a user inputs and sends via a terminal.

[2221] "Emotion data" is information about emotions extracted from the user's facial expressions, voice, etc.

[2222] "Generative AI" is an AI system that has the ability to analyze input data and generate answers.

[2223] An "emotion engine" is a system that analyzes emotional data collected from users and recognizes specific emotions.

[2224] "Natural language processing" is a technique used by generative artificial intelligence to understand the content of questions, and is a technology that analyzes natural language and extracts meaning.

[2225] A "database" is an information repository where technical documents and related information are stored.

[2226] A "technical document" is a document that contains detailed information about a particular technology and is used to answer user questions.

[2227] A "response sentence" is an answer generated by generative artificial intelligence and provided to the user.

[2228] "Log data" is data that records the operation history of a system or application.

[2229] The present invention relates to a system in which a user posts a question about a specific function or operation method, and a server uses generative artificial intelligence and an emotion engine to provide appropriate information and emotional responses. Furthermore, the system can also analyze faults and suggest countermeasures. An embodiment of the present invention is described in detail below.

[2230] overview

[2231] First, a user enters a question into a web form and clicks the submit button. At this time, the user's device converts the question data along with emotion data into JSON format and sends it as an HTTPS request to the server. The server receives the question data and emotion data and passes it to the generative AI and emotion engine. The generative AI analyzes the question and searches for related technical documents. The emotion engine analyzes the user's emotion and reflects it in a response. The final response is sent to the user's device.

[2232] Hardware and software used

[2233] User

[2234] Users use devices such as PCs and smartphones that have a web browser installed, allowing them to enter questions via a web form.

[2235] Terminal

[2236] The terminal receives input from the user, converts question data and emotion data into JSON format, and sends it to the server.

[2237] server

[2238] The server utilizes cloud infrastructure, such as AWS EC2 instances, and uses generative artificial intelligence (e.g., OpenAI GPT-4) and emotion engines (e.g., Microsoft Azure Face API) to analyze and process the incoming data.

[2239] Generative Artificial Intelligence

[2240] Generative AI uses natural language processing to analyze the input question and generate a response based on related technical documentation.

[2241] Emotion Engine

[2242] The emotion engine acquires and analyzes emotion data from the user's facial expressions and tone of voice.

[2243] Enter and submit questions and emotion data

[2244] User

[2245] A user enters a question into a web form, such as "How do I create a graph in Excel?", and clicks the submit button. At this time, the system analyzes the user's facial expressions and tone of voice to obtain emotional data (e.g., "curious").

[2246] Receiving and analyzing questions and emotion data

[2247] server

[2248] The server receives the question data and emotion data sent from the user's device, passes the received question data to the generative AI, and passes the emotion data to the emotion engine.

[2249] Generative Artificial Intelligence

[2250] Generative AI uses natural language processing to analyze the question and extract important keywords, such as "Excel," "graph," "create," and "method."

[2251] Emotion Engine

[2252] The emotion engine analyzes the emotion data and identifies the user's emotion, for example, "curious."

[2253] Related document search and information extraction

[2254] server

[2255] The server queries the database to find relevant technical documents, and the resulting list of documents is passed to the generative AI.

[2256] Generative Artificial Intelligence

[2257] Generative artificial intelligence evaluates search results and extracts the most relevant information.

[2258] Response generation and emotional reflection

[2259] Generative Artificial Intelligence

[2260] Based on the information extracted by the generative AI, it generates a response in a format that is easy for the user to understand. For example, it might generate a response such as, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group."

[2261] Emotion Engine

[2262] The emotion engine adjusts the response depending on the user's emotions, for example adding "Interesting!"

[2263] Sending and displaying responses

[2264] server

[2265] The generated response is converted to JSON format and sent to the user's device. For example, it is sent in the following format:

[2266] json

[2267] {

[2268] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[2269] }

[2270] Terminal

[2271] The terminal receives the response from the server and displays it on the screen.

[2272] Questions about disabilities and solutions

[2273] User

[2274] A user posts a question like, "What causes my application to crash?"

[2275] Terminal

[2276] The device sends question data and emotion data in JSON format to the server.

[2277] server

[2278] The server receives the question data and emotion data and passes them to the generative artificial intelligence and emotion engine.

[2279] Generative Artificial Intelligence

[2280] Generative artificial intelligence analyzes the question content and collects and analyzes related log data (system logs and error logs).

[2281] Emotion Engine

[2282] The emotion engine analyzes the emotion data and identifies the user's emotion.

[2283] Generative Artificial Intelligence

[2284] Generative AI analyzes the cause of the problem and countermeasures, and generates a response. For example, if the cause of the problem is identified as "insufficient memory," it generates a response such as "Please close unnecessary applications or consider increasing memory."

[2285] Emotion Engine

[2286] The emotion engine adjusts the response depending on the user's emotions, for example adding the sentence "Don't worry, it's okay!"

[2287] server

[2288] The generated response sentence is sent to the user's terminal.

[2289] Terminal

[2290] The terminal receives the response and displays it on the screen.

[2291] As described above, the system of the present invention not only allows users to efficiently obtain the information they need, but also provides emotional support. For example, in response to the question, "How do I create a graph in Excel?", the system provides the following response: "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the type of graph you want from the 'Chart' group. Interesting!" In this way, the system provides users with prompt and appropriate information tailored to their needs.

[2292] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2293] Processing flow

[2294] Step 1:

[2295] A user enters a question about a specific function or operation method into a web form and clicks the submit button. For example, the input might be, "How do I create a graph in Excel?" At this time, the user's facial expression and tone of voice are sent to the device as emotional data.

[2296] Step 2:

[2297] The device converts the input question and emotion data into JSON format and sends it to the server as an HTTPS request. The input data is in the following format:

[2298] json

[2299] {

[2300] "question": "How to create a graph in Excel?",

[2301] "emotion": "curious"

[2302] }

[2303] Step 3:

[2304] The server receives question data and emotion data sent from the user's device, processes the received data, and passes it to the generative AI and emotion engine in an appropriate format.

[2305] Step 4:

[2306] Generative AI analyzes question data. It uses natural language processing (NLP) to understand the question and extract key keywords. The input data is the question data, and the output data is the extracted keywords (e.g., "Excel," "graph," "create," "how to").

[2307] Step 5:

[2308] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "interesting").

[2309] Step 6:

[2310] The server executes a database query to search for relevant technical documents, with the input data being the extracted keywords and the output data being a list of hit technical documents.

[2311] Step 7:

[2312] Generative AI evaluates search results and extracts the most relevant information. The input data is a list of technical documents, and the output data is the extracted technical information.

[2313] Step 8:

[2314] Based on the information extracted by the generative artificial intelligence, a response is generated in a format that is easy for the user to understand. Specifically, the response generated is, "To create a graph in Excel, select a data range, click the 'Insert' tab in the top menu, and select the desired graph type from the 'Chart' group." The input data is the extracted technical information, and the output data is the generated response.

[2315] Step 9:

[2316] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Interesting!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[2317] Step 10:

[2318] The server converts the generated response into JSON format and sends it to the user's device, specifically in the following format:

[2319] json

[2320] {

[2321] "response": "To create a chart in Excel, select a data range, click the 'Insert' tab in the top menu, and choose the chart type you want from the 'Charts' group. Interesting!"

[2322] }

[2323] Step 11:

[2324] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[2325] Inquiry about disabilities and provision of solutions

[2326] Step 12:

[2327] A user posts a question about a specific problem or error. For example, "What causes my application to crash?"

[2328] Step 13:

[2329] The device sends question data and emotion data in JSON format to the server. The input data is in the following format:

[2330] json

[2331] {

[2332] "question": "What causes my application to crash?",

[2333] "emotion": "worried"

[2334] }

[2335] Step 14:

[2336] The server receives the question data and emotion data and passes them to the generative AI and emotion engine. The server processes the received data and sends it in the appropriate format.

[2337] Step 15:

[2338] The generative AI analyzes the question content and collects and analyzes related log data (e.g., system logs, error logs). The input data is the question data, and the output data is the analyzed log data and the extracted cause of the failure.

[2339] Step 16:

[2340] The emotion engine analyzes the emotion data and identifies the user's emotion. The input data is the emotion data, and the output data is the identified emotion (e.g., "worry").

[2341] Step 17:

[2342] The generative AI analyzes the cause of the failure and countermeasures, and generates a response. Specifically, it identifies the cause of the failure as "insufficient memory," and generates a response saying, "Please close unnecessary applications or consider increasing memory." The input data is the analyzed log data and the extracted cause of the failure, and the output data is the generated response.

[2343] Step 18:

[2344] The emotion engine adjusts the response sentence according to the user's emotion. Specifically, it adds the sentence "Don't worry, everything is fine!". The input data is the response sentence and the identified emotion, and the output data is the response sentence that reflects the emotion.

[2345] Step 19:

[2346] The server converts the generated response text into JSON format and sends it to the user's device. The input data is the generated response text, and the output data is the JSON format data sent to the user's device.

[2347] Step 20:

[2348] The terminal receives the response from the server and displays it on the screen, and the user can check the information obtained through the terminal screen.

[2349] This is the specific processing flow of this system, which allows users to efficiently obtain information while also receiving emotional support.

[2350] (Application example 2)

[2351] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2352] Many users today are facing serious security-related problems. In these circumstances, conventional automated response systems must not only provide accurate information in response to users' questions and problems, but also provide emotional support. However, current systems lack emotional consideration, making it difficult for users who are anxious or nervous to feel at ease. Furthermore, there is a need for systems that can respond to users' emotions rather than simply providing technical information. A system that can appropriately address these issues is needed.

[2353] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2354] In this invention, the server includes means for receiving question data and emotion data from a user's terminal, means for transmitting the received question data and emotion data to a generative artificial intelligence and an emotion engine, means for the generative artificial intelligence to analyze the question content and understand the intent of the question, means for searching for related technical documents, means for extracting the most relevant information from the search results and generating a response sentence, means for emotionally adjusting the response sentence based on the emotion data analyzed by the emotion engine, and means for transmitting the generated response sentence to the user's terminal. This makes it possible to provide not only accurate information in response to a user's questions or problems, but also emotional support.

[2355] "Question data" is text information about questions or problems that users input to the system.

[2356] "Emotion data" refers to emotion information extracted from the user's facial expression, tone of voice, and text content.

[2357] "Generative AI" is an AI system that uses natural language processing technology to analyze the content of a user's question, understand their intent, and generate an appropriate response.

[2358] An "emotion engine" is a system that analyzes a user's emotional data and generates a response that takes those emotions into consideration.

[2359] A "user's terminal" is an information device such as a smartphone or a personal computer used by a user.

[2360] The "server" is an information processing device that relays between the user's device and the generative AI and emotion engine, receiving and sending data and generating responses.

[2361] "Natural language processing" is a technology that allows computers to understand, analyze, and generate human language.

[2362] "Technical documents" are documents that contain relevant technical information and are the subject of searches by the server.

[2363] "Log data" refers to data that records the operation history and error information of systems and applications.

[2364] A "response sentence" is an answer to a user's question generated by a generative artificial intelligence, and is a sentence adjusted by an emotion engine.

[2365] The present invention relates to a system that receives question data and emotion data from a user's device and uses generative artificial intelligence and an emotion engine to provide appropriate and emotionally sensitive responses. This system is particularly effective in responding to security-related questions and problems.

[2366] System Program Overview

[2367] 1. Receiving user questions and emotion data

[2368] The user enters a question in text and sends it from the terminal.

[2369] Emotional data is obtained from the user's facial expressions and tone of voice, and transmitted from the device.

[2370] 2. Question and Sentiment Data Analysis

[2371] A server receives the question data and the emotion data.

[2372] Generative AI analyzes the question using natural language processing and extracts important keywords.

[2373] The emotion engine analyzes the emotion data and identifies the user's emotion.

[2374] 3. Technical Document Search and Response Generation

[2375] The server searches the database for relevant technical documents and retrieves the most relevant information.

[2376] Generative artificial intelligence generates appropriate responses from technical documents.

[2377] The emotion engine adjusts the response sentence according to the user's emotion.

[2378] 4. Sending a response

[2379] The server sends the final response to the terminal and displays it to the user.

[2380] Implementation hardware and software

[2381] Hardware

[2382] User's device: Information devices such as smartphones and PCs

[2383] Server: A high-performance server for data processing and analysis

[2384] software

[2385] Natural language processing technology: Google Cloud Natural Language API, IBM Watson, etc.

[2386] Sentiment analysis technology: Microsoft Azure Emotion API, Amazon Rekognition, etc.

[2387] Database: Relational database such as MySQL or PostgreSQL

[2388] Generative AI: GPT-4 and similar generative AI models

[2389] Example of system operation

[2390] Entering questions and sentiment data

[2391] The user types the question "Is this network safe?" and sends it from the device. At the same time, emotional data such as "anxiety" is obtained from the user's facial expression and tone of voice.

[2392] analysis

[2393] The server receives this data, and the generative AI extracts keywords such as "network" and "safety," while the emotion engine identifies the emotion "anxiety."

[2394] Response generation and adjustment

[2395] The generative AI generates a technical response such as "The current network is encrypted," while the emotion engine adds an emotional element: "Don't worry, we'll follow up."

[2396] Sending a response

[2397] The final response, "The current network is encrypted. Don't worry, we will follow up, so don't worry," is sent to the user's device and displayed.

[2398] Prompt Sentence Examples

[2399] "Is my network secure?"

[2400] "I think I might have the virus, what should I do?"

[2401] "How can I reduce the risk of my password being hacked?"

[2402] This can alleviate the user's doubts and anxieties and provide a sense of security.

[2403] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2404] Step 1:

[2405] The user inputs and sends question data and emotion data from the terminal.

[2406] Input: The user enters the question "Is this network safe?" and emotional data of "anxiety" is obtained from facial expressions and tone of voice.

[2407] Processing: The device converts this data into JSON format and sends it to the server as an HTTPS request.

[2408] Output: Question data and emotion data are sent to the server.

[2409] Step 2:

[2410] A server receives the question data and the emotion data.

[2411] Input: Question data and emotion data sent from the device

[2412] Processing: The server passes the received data to the generative artificial intelligence and emotion engine for analysis.

[2413] Output: Question data passed to the generative AI, emotion data passed to the emotion engine

[2414] Step 3:

[2415] Generative AI analyzes the content of the question and understands its intent.

[2416] Input: Query data passed from the server

[2417] Processing: Using natural language processing technology (e.g., Google Cloud Natural Language API), important keywords are extracted from the question.

[2418] Data processing and calculation: For example, extracting keywords such as "network" and "safety."

[2419] Output: Parsed question content and extracted keywords

[2420] Step 4:

[2421] The emotion engine analyzes the emotion data and identifies the user's emotion.

[2422] Input: Emotion data passed from the server

[2423] Processing: Analyze the emotion data using emotion analysis technology (e.g., Microsoft Azure Emotion API).

[2424] Data processing and calculation: Identify the user's emotion as "anxiety."

[2425] Output: Analyzed user emotion information

[2426] Step 5:

[2427] The server searches the database for relevant technical documents and retrieves the most relevant information.

[2428] Input: Keywords passed from the generative AI

[2429] Processing: Executes database queries and retrieves technical documents.

[2430] Data processing and calculation: Narrow down relevant documents based on keywords.

[2431] Output: Search results for a list of technical documents

[2432] Step 6:

[2433] Generative artificial intelligence generates appropriate responses from technical documents.

[2434] Input: List of technical documents and questions

[2435] Processing: Generate the most appropriate response sentence based on the technical documentation.

[2436] Data processing and calculation: For example, generating a technical response such as "The current network is encrypted."

[2437] Output: Response

[2438] Step 7:

[2439] The emotion engine adjusts the response sentence according to the user's emotion.

[2440] Input: Response sentences from generative AI and analyzed emotional information

[2441] Processing: Emotionally adjusting the response sentence.

[2442] Data processing and calculation: For example, adding an emotional element such as "Don't worry, we will follow up, so don't worry."

[2443] Output: Emotionally tailored response sentence

[2444] Step 8:

[2445] The server sends the final response to the terminal and displays it to the user.

[2446] Input: Emotionally tailored response sentences

[2447] Processing: Convert the response into JSON format and send it to the terminal.

[2448] Output: The final response sent to the terminal.

[2449] These processing steps allow users to receive technically accurate and emotionally sensitive responses to their questions and concerns in real time.

[2450] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2451] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2452] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2453] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2454] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive...

Claims

1. means for receiving question data from a user terminal; A means for transmitting the received question data to a generative artificial intelligence; A generative AI analyzes the content of the question and understands its intent. a means of retrieving relevant technical documentation; means for extracting the most relevant information from the search results and generating a response sentence; means for transmitting the generated response sentence to a user terminal; A system including:

2. 2. The system of claim 1, further comprising means for using natural language processing when the generative artificial intelligence analyzes the content of the question.

3. When the system receives a question about a failure, a means for collecting relevant log data and analyzing the cause of the failure; 2. The system according to claim 1, further comprising means for generating a response statement for taking measures against the failure.

Images