System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting an operation log of a user in real time; means for transmitting the collected operation log to a server; means for analyzing the received operation log and identifying a problem that the user is facing; means for retrieving appropriate support information for the identified problem from a database; means for generating a solution statement using a generative AI model based on the retrieved support information; and means for displaying the generated solution statement on a user device.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, IT service specifications have become increasingly complex, and when users encounter problems, error messages are often standardized and unclear. Another issue is that self-help content may not be appropriate for their own situation, and contacting customer support can be time-consuming. This can prevent users from quickly resolving their problems, leading to concerns that their service experience may be adversely affected. [Means for solving the problem]

[0005] The present invention provides a system including means for collecting user operation logs in real time, means for transmitting the collected operation logs to a server, means for analyzing the received operation logs to identify the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, and means for displaying the generated solution message on a user terminal. This allows the user to immediately receive a specific solution instead of an error message, thereby accelerating problem resolution and improving the user experience.

[0006] An "operation log" is data that records information about operations performed by a user on a system or application.

[0007] "Real-time" refers to ongoing events and data processing that occurs immediately and without delay.

[0008] A "server" is a central computer system that provides data in response to requests from client computers.

[0009] "Analysis" is the act of examining collected data in detail to understand its meaning and structure and to identify specific patterns or problems.

[0010] "Support Information" means information that contains instructions or procedures for resolving problems that users encounter.

[0011] A "database" is a system for efficiently storing, retrieving, and managing large amounts of data.

[0012] A "generative AI model" is a computational model that uses artificial intelligence technology to automatically generate text or data based on specific input information.

[0013] A "solution text" is a text message that tells the user how to solve the problem.

[0014] A "user terminal" is an electronic device such as a computer, smartphone, or tablet that is directly operated by a user. [Brief explanation of the drawings]

[0015] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0016] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0017] First, the terms used in the following description will be explained.

[0018] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0019] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0020] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0021] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0022] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0023] [First embodiment]

[0024] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0025] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0026] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0027] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0028] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0029] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0030] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0031] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0032] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0034] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0036] This invention relates to a system for solving problems that occur when users use IT services in real time. This system collects user operation logs, generates appropriate solutions based on those logs, and presents them to the users.

[0037] Collecting user operation logs (device)

[0038] The device has the ability to monitor user operations (clicks, inputs, error messages, etc.) in real time. Each operation event is logged with a timestamp. For example, if a user clicks the "Upload File" button but the upload fails, the operation and error message are recorded in the log.

[0039] Sending operation logs (terminal)

[0040] The collected operation logs are sent to the server at regular intervals, so that the server always has the latest operation information and can respond quickly if a problem occurs.

[0041] Receiving and analyzing operation logs (server)

[0042] The server has the ability to analyze the received operation logs. This analysis detects error messages and consecutive operation patterns, among other things, to identify problems the user may be facing. For example, if a file upload fails, the error message can be analyzed to identify the cause of the problem (e.g., file size exceeded).

[0043] Extracting appropriate support information (server)

[0044] The server has the ability to retrieve appropriate support information from the database for the identified problem, including FAQs, guidelines, troubleshooting procedures, etc. For example, in response to a file size exceeding error message, it retrieves "information about the maximum allowed file size."

[0045] Solution text generation (server)

[0046] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is tailored to help the user understand the problem. Specifically, a solution such as "The file may be too large. The maximum file size is 10MB" is generated.

[0047] Presentation of the resolution text (terminal)

[0048] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[0049] Specific examples

[0050] Example 1:

[0051] Here's what happens if a user tries to upload a file but fails:

[0052] 1. The user clicks the "upload button."

[0053] 2. The device will log this operation and also log a subsequent "upload failed" error.

[0054] 3. Logs are sent to the server at regular intervals.

[0055] 4. The server receives the log, analyzes it, and determines that the cause of the "upload failure" is excessive file size.

[0056] 5. The server retrieves the support information about the "maximum file size" from the database.

[0057] 6. Based on the information obtained, the generative AI model generates a solution message: "The file may be too large. The maximum file size is 10MB."

[0058] 7. The device displays the generated solution to the user and provides specific instructions for resolving the problem.

[0059] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process, resulting in faster problem resolution and a better user experience.

[0060] The processing flow will be explained below.

[0061] Step 1:

[0062] A user performs a specific action on the system, for example clicking a "File Upload" button.

[0063] Step 2:

[0064] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, etc.).

[0065] Step 3:

[0066] The terminal sends the collected log data to the server at regular intervals, which can be adjusted according to the system settings.

[0067] Step 4:

[0068] Analyze the log data received by the server. The log data contains various events, and identify problems by paying particular attention to error messages and consecutive operation patterns.

[0069] Step 5:

[0070] The server retrieves the appropriate support information for the identified problem from its database, which may include FAQs, guidelines, troubleshooting procedures, etc.

[0071] Step 6:

[0072] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model then adjusts the message appropriately to make it easier for the user to understand the problem.

[0073] Step 7:

[0074] The server generates a solution message and sends it to the device, which includes specific solutions and operational procedures.

[0075] Step 8:

[0076] The device will display the solution text in the user interface, allowing the user to take appropriate action to resolve the issue.

[0077] Specific examples

[0078] Example: File upload failure

[0079] Step 1:

[0080] The user clicks the "File Upload" button.

[0081] Step 2:

[0082] The device will log the click and then log the upload failure error message.

[0083] Step 3:

[0084] The terminal sends the collected log data to the server every 60 seconds.

[0085] Step 4:

[0086] Analyze the logs received by the server and identify the "upload failed" error.

[0087] Step 5:

[0088] The server retrieves support information about "file size exceeded" from the database.

[0089] Step 6:

[0090] The server uses the generative AI model to generate a solution message: "The file may be too large. The maximum file size is 10MB."

[0091] Step 7:

[0092] The server sends the generated solution text to the terminal.

[0093] Step 8:

[0094] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] Traditional IT services lacked the means to instantly resolve problems users faced. As a result, users took time to identify the cause of the problem and had to use multiple support channels to find the appropriate solution. In particular, when an error message occurred, the process of understanding the message and finding a solution was cumbersome, resulting in a poor user experience. To solve these problems, a system that automatically identifies problems based on user operation logs and provides appropriate support information is needed.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for providing a prompt message to the generative AI model based on the cause of the problem, and means for displaying the generated solution message on a user interface. This enables the user to quickly identify the cause of the problem and immediately obtain appropriate support information and a solution.

[0100] A "user operation log" is an operation history that includes clicks, inputs, error messages, etc. that is generated when a user uses a system or application.

[0101] "Means of collecting data in real time" refers to the function of instantly capturing the operations performed by the user and recording that information as a log.

[0102] "Means for sending to a server" refers to the function of transferring collected operation logs to a server via a network such as the Internet periodically or based on certain conditions.

[0103] "Means for analyzing received operation logs" refers to the function of processing the data in the operation logs received by the server to detect error messages and operation patterns contained in the logs and identify the cause of the problem.

[0104] "Means for identifying problems faced by users" refers to the function of identifying errors or problems that users are currently facing from the analyzed operation logs.

[0105] "Means for retrieving appropriate support information from the database" refers to the function of searching and retrieving support information such as related solutions, FAQs, guidelines, etc. in the database according to the identified problem.

[0106] "Means for generating a solution message using a generative AI model" refers to the function of utilizing a generative AI model to create a solution message that is easy for users to understand based on the acquired support information and the cause of the problem.

[0107] "Means for providing prompt sentences" refers to the function of supplying sentences that provide the generative AI model with background information and instructions for generating an appropriate solution sentence.

[0108] "Means for displaying on the user interface" refers to the function of displaying the generated solution text on the screen of the user's terminal and providing it visually.

[0109] MODE FOR CARRYING OUT THE INVENTION

[0110] This invention relates to a system for solving problems that occur when a user uses an IT service in real time. This section describes the hardware and software required to configure this system, as well as its operating method.

[0111] Collecting user operation logs (device)

[0112] The device monitors user actions in real time. Events such as buttons clicked by the user, text entered, and error messages are logged with a timestamp. This typically involves sensors and monitoring software (e.g., keyloggers, click-monitoring tools).

[0113] For example, if a user clicks the "Upload button", if this operation is successful, a success event will be logged. Conversely, if the operation fails and an error message such as "File size too large" is displayed, this error information will also be logged.

[0114] Sending operation logs (terminal)

[0115] The collected operation logs are sent to the server at regular intervals. This sending process is performed in the background so as not to interfere with user operations. HTTP / HTTPS is used as the protocol. Specifically, logs are sent to the server in batches at regular volume or time intervals (e.g., every 5 seconds).

[0116] Server receives and analyzes logs

[0117] The server analyzes the received operation logs. For analysis, a database server (e.g., MySQL, PostgreSQL) and data processing scripts (e.g., Python, R) are used. Specifically, the log data is stored in a database and error messages and specific operation patterns are detected.

[0118] For example, if the server receives a "file upload failed" error message, it analyzes the logs to determine that the cause is an excessive file size. The results of this analysis are passed on to the next step.

[0119] Extracting appropriate support information (server)

[0120] Based on the analysis results, the server retrieves the appropriate support information from a database that stores FAQs, guidelines, troubleshooting procedures, etc. SQL queries are used to retrieve the support information.

[0121] For example, for the error message "File size exceeded," you will get support information about "Maximum file size," which will give the user information to understand the specific cause of the problem.

[0122] Solution text generation (server)

[0123] The server generates a solution message using a generative AI model (e.g., OpenAI GPT-4) based on the acquired support information. The generative AI model receives the support information and the cause of the problem as input, and outputs a message that provides a solution to the user in an easy-to-understand format.

[0124] The generated solution text will be specific, for example, "The file may be too large. The maximum file size is 10MB."

[0125] Presentation of the resolution text (terminal)

[0126] The generated solution text is sent to the device and displayed in the user interface using UI elements such as a pop-up window or a notification bar, allowing the user to instantly receive a concrete solution instead of an error message.

[0127] Prompt Sentence Examples

[0128] The prompt provided to the generative AI model looks something like this:

[0129] If the upload failure is due to excessive file size, please provide a solution such as "The file may be too large. The maximum file size is 10MB."

[0130] Using such detailed prompts allows the AI ​​model to generate more accurate and useful solutions.

[0131] This system will enable users to quickly identify the cause of a problem and immediately obtain appropriate support information and solutions, which is expected to significantly improve the experience of using IT services.

[0132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0133] The flow of this system's program processing

[0134] Step 1: Collect user operation logs

[0135] Users operate systems and applications.

[0136] The device monitors each user action (click, input, error message, etc.) in real time and creates a time-stamped operation log.

[0137] Specifically, when a user clicks the "Upload button," the click event is recorded, and if the upload fails, an error message stating "File size too large" is also recorded.

[0138] Input: User actions (clicks, inputs, error messages, etc.)

[0139] Output: Operation log with timestamp

[0140] Step 2: Send the operation log

[0141] The terminal sends the collected operation logs to the server at regular intervals.

[0142] This sending process is done in the background and does not interfere with user operations. For example, a batch of operation logs is sent to the server every 5 seconds.

[0143] Input: Timestamp operation log

[0144] Output: Batch log data to be sent

[0145] Step 3: Receiving and saving the operation log

[0146] The server receives the operation log sent from the terminal and stores it in a database.

[0147] Specifically, the server opens the received log data and inserts each log entry into the appropriate table in the database.

[0148] Input: Batch log data to be sent

[0149] Output: Operation log saved in the database

[0150] Step 4: Analyze the operation log

[0151] The server analyzes the operation logs stored in the database.

[0152] In particular, it processes data to identify the cause of a problem by detecting error messages and operation patterns. For example, it searches for "upload failed" events in the log and identifies that the cause is "file size exceeded."

[0153] Input: Operation logs stored in the database

[0154] Output: Identified cause of the problem (e.g. file size exceeded)

[0155] Step 5: Pulling Support Information

[0156] The server retrieves appropriate support information from a database based on the identified problem.

[0157] Specifically, it runs an SQL query to get information about "Maximum File Size".

[0158] Input: Identified cause of the problem

[0159] Output: Retrieved support information (e.g. information about maximum file size)

[0160] Step 6: Generate a solution

[0161] The server generates a solution text using a generative AI model (e.g., OpenAI GPT-4) based on the obtained supporting information and analysis results.

[0162] The generative AI model generates a prompt and then generates a solution for the user, such as "The file may be too large. The maximum file size is 10MB."

[0163] Input: Retrieved support information, identified problem cause, prompt statement

[0164] Output: Generated solution text

[0165] Step 7: Present the solution

[0166] The server sends the generated solution text to the terminal.

[0167] The device will display the received solution text on the user interface, and the user can immediately obtain a specific solution, for example, as a notification bar or a pop-up window.

[0168] Input: Generated solution text

[0169] Output: Resolution text displayed in the user interface

[0170] (Application example 1)

[0171] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0172] When users encounter errors or problems during the payment process with electronic payment services, they must search for solutions themselves, making it difficult to resolve them quickly. This can lead to a poor user experience and a loss of reliability in the service. Furthermore, there is often no system in place to provide real-time solutions, often leaving users frustrated. This invention aims to resolve these issues and make the payment process smoother and more efficient for users.

[0173] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0174] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying a problem the user is facing, means for retrieving appropriate support information for the identified problem from a data storage device, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user device, and means for analyzing details of an error that occurred in a specific payment process based on the user's operation and quickly providing the solution message, thereby enabling users to quickly resolve problems they encounter in the electronic transaction process and improving their service usage experience.

[0175] A "user operation log" is data that records a series of operation events (clicks, inputs, error messages, etc.) performed by a user on an electronic device or system.

[0176] "Means of collecting data in real time" refers to a mechanism that instantly detects user operations and immediately records the operation information as data.

[0177] A "server" is a computer system that receives operation logs sent by users, analyzes the data, and presents solutions to problems.

[0178] The "means for analyzing and identifying the problems the user is facing" is a mechanism for analyzing error messages and operation patterns from the received operation log to detect the problems the user is currently experiencing.

[0179] "Support Information" is information that helps users solve problems they encounter, such as FAQs, guidelines, and troubleshooting procedures.

[0180] A "data storage device" is a device for long-term storage of support information and other important data in a computer system.

[0181] A "generative AI model" is an algorithm or program that uses artificial intelligence to automatically generate solutions or instructions in natural language based on input data.

[0182] A "solution statement" is a document that provides specific solutions or instructions for a problem that a user faces in an easy-to-understand format.

[0183] A "user device" is a device that a user operates, such as a smartphone, computer, or head-mounted display.

[0184] The "means for analyzing the details of an error" is a mechanism for deeply analyzing the error message contained in the operation log and clarifying the cause and details of the error.

[0185] "Means for rapid provision" refers to a mechanism for presenting analysis results and generated solution text to users without delay.

[0186] This invention is a system that collects user operation logs in real time, generates appropriate solutions based on those logs, and presents them to the user. Specifically, it aims to link the server and user terminals to quickly resolve problems users encounter during the electronic payment process.

[0187] Program processing explanation

[0188] Collecting user operation logs (device)

[0189] The user device monitors user actions (clicks, inputs, error messages, etc.) in real time and records them in a log, which is time-stamped and records each operation event in detail.

[0190] Sending operation logs (terminal)

[0191] The collected operation logs are temporarily stored on the user's device and then sent to the server at regular intervals. This process ensures that the server always has the latest operation information.

[0192] Receiving and analyzing operation logs (server)

[0193] The server receives operation logs sent from user terminals. The received logs are analyzed, and error messages and consecutive operation patterns are automatically detected. A log analysis algorithm is used for this analysis.

[0194] Extracting appropriate support information (server)

[0195] For problems identified as a result of the analysis, the server retrieves appropriate support information from its data store, including FAQs, guidelines, troubleshooting procedures, and the like.

[0196] Solution text generation (server)

[0197] Based on the extracted support information, the server generates a solution using a generative AI model. This generative AI model uses a natural language processing algorithm to automatically generate a solution based on the input data. For example, it can use OpenAI's API.

[0198] Presentation of the resolution text (terminal)

[0199] The generated solution text is sent from the server to the user's device and displayed on the user interface, allowing the user to receive specific instructions on how to solve the problem.

[0200] Specific examples

[0201] 1. The user enters their card information on the payment page and clicks the "Payment button."

[0202] 2. The user terminal will log this operation and immediately afterwards also log a "Payment failed" error message.

[0203] 3. Logs are sent to the server at regular intervals.

[0204] 4. The server receives the log, analyzes it, and determines that the cause of the "payment failure" is incorrect card information.

[0205] 5. The server uses a generative AI model (OpenAI API) to generate a solution message that reads, "Your card information may be incorrect. Please re-enter accurate information."

[0206] 6. The user device displays the generated solution text to the user and provides specific instructions for solving the problem.

[0207] Prompt Sentence Examples

[0208] Please provide a solution: Payment failed: Incorrect card details

[0209] This system allows users to quickly resolve any issues they encounter during the payment process, improving their service experience.

[0210] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0211] Step 1:

[0212] A user enters card information on the page of an electronic payment service and clicks the "Payment button." This operation is performed on the user's device. This input generates a user operation log, which includes a timestamp of the card information input.

[0213] Step 2:

[0214] The user device collects user operation logs (clicks, inputs, error messages, etc.) in real time. For example, clicking the payment button and error messages when a payment fails are recorded as logs. This log is time-stamped and accumulates the collected operation events.

[0215] Step 3:

[0216] The collected operation logs are temporarily stored on the user's device and sent to the server at regular intervals. The data entered here is the operation log data, which is then sent to the server. The server obtains the latest operation information in real time, so the system is designed to avoid delays in sending the logs.

[0217] Step 4:

[0218] The server receives operation logs sent from user terminals. It analyzes the received operation logs and automatically detects error messages and consecutive operation patterns in particular. The input here is the operation log sent to the server, and the output is the analysis results, including error messages. A log analysis algorithm is used for this analysis.

[0219] Step 5:

[0220] As a result of the analysis, error messages and specific operation patterns are detected to identify the problem the user is facing. Based on the analysis results, the server retrieves appropriate support information from a data storage device. Here, the input is the analysis results, and the output is the corresponding support information (FAQs, guidelines, troubleshooting procedures, etc.).

[0221] Step 6:

[0222] Based on the extracted support information, the server uses a generative AI model to generate a solution text. The generative AI model (e.g., OpenAI API) creates a solution text in natural language based on the input support information and analysis results. The input is support information and analysis results, and the output is the solution text.

[0223] Step 7:

[0224] The generated solution text is sent from the server to the user terminal. The input here is the solution text, and the output is the text displayed on the user terminal. The user terminal displays this solution text on the user interface and presents a specific solution to the user.

[0225] Prompt Sentence Examples

[0226] Please provide a solution: Payment failed: Incorrect card details

[0227] The above are the specific processing steps of the system program that realizes the application example. Through this series of processes, users can receive solutions to problems in real time and quickly resolve their issues.

[0228] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0229] This invention combines a system that solves problems that arise when users use IT services in real time with an emotion engine that recognizes the user's emotions. This system not only collects user operation logs and generates and presents appropriate solutions based on those logs, but also adjusts the solution text taking the user's emotions into account.

[0230] Collecting user operation logs (device)

[0231] The device monitors user operations (clicks, inputs, error messages, etc.) in real time and records each operation event in a log with a timestamp. The log also includes operation characteristics such as operation speed, click strength, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[0232] Sending operation logs (terminal)

[0233] The collected operation logs are sent to the server at regular intervals. This interval can be adjusted according to the system settings. The sent data includes basic operation information and operation characteristic information.

[0234] Receiving and analyzing operation logs (server)

[0235] The server analyzes the received operation logs. The log data contains various events, and it identifies problems by paying particular attention to error messages and consecutive operation patterns. It also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion.

[0236] Extracting appropriate support information (server)

[0237] The server retrieves appropriate support information for the identified problem from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account emotional information, providing detailed instructions if the user is confused or quick solutions if the user is impatient.

[0238] Solution text generation (server)

[0239] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is adjusted to make it easier for the user to understand the problem. Specifically, the generated solution might say, "The file may be too large. The maximum file size is 10MB." The way the solution is expressed and the level of detail are adjusted depending on the user's feelings.

[0240] Presentation of the resolution text (terminal)

[0241] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[0242] Specific examples

[0243] Example 1: File upload failure

[0244] 1. The user clicks the "File Upload" button.

[0245] 2. The device will log this operation and then log an error message about the failed upload. If the user has tried multiple times, this information will also be included in the log.

[0246] 3. Logs are sent to the server at regular intervals.

[0247] 4. The server receives the log, identifies the "upload failed" error, and realizes that the user is anxious.

[0248] 5. The server retrieves the "information about the maximum file size" from the database and generates a quick solution that takes into account the user's impatience.

[0249] 6. The server uses the generative AI model to generate a solution: "The file may be too large. Maximum file size is 10MB. Please fix it immediately."

[0250] 7. The server sends the generated solution text to the terminal.

[0251] 8. The device will display the solution text in a pop-up window and provide the user with a specific solution.

[0252] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process. Furthermore, by taking the user's emotions into consideration, the user's experience is further improved.

[0253] The processing flow will be explained below.

[0254] Step 1:

[0255] A user performs a specific action on the system, for example clicking a "File Upload" button.

[0256] Step 2:

[0257] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, operation speed, click strength, mouse movement).

[0258] Step 3:

[0259] The terminal sends the collected log data and operation characteristic information to the server at regular intervals (for example, every 60 seconds). This allows the server to always have the latest operation information and respond immediately.

[0260] Step 4:

[0261] The server analyzes the received log data. This analysis includes detecting error messages and consecutive operation patterns, and recognizing the user's emotions using operation characteristics information. For example, rapid consecutive clicks may indicate impatience, while slow operation speed may indicate confusion.

[0262] Step 5:

[0263] The server retrieves the appropriate support information for the identified issue from a database, including FAQs, guidelines, troubleshooting procedures, etc. Additionally, the retrieved information is tailored to the user's perceived sentiment.

[0264] Step 6:

[0265] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model adjusts the expression and level of detail of the message depending on the user's emotions. For example, it might generate a message such as "The file may be too large. The maximum file size is 10MB. Please fix it immediately" for a user who is expressing impatience.

[0266] Step 7:

[0267] The server sends the generated solution text to the device, which includes specific solutions and operational procedures.

[0268] Step 8:

[0269] The device will display the solution text in the user interface, allowing users to instantly receive a specific solution instead of an error message, and users can take action based on this information to quickly resolve the issue.

[0270] Specific examples

[0271] Example: File upload failure

[0272] Step 1:

[0273] The user clicks the "File Upload" button.

[0274] Step 2:

[0275] The device logs the click, then logs the upload failure error message, and if the user retries multiple times, logs information such as the speed of the click and the strength of the click.

[0276] Step 3:

[0277] The terminal sends the collected log data and operational characteristics information to the server every 60 seconds.

[0278] Step 4:

[0279] The server receives the log, identifies the "upload failed" error, and recognizes from the operational characteristics information contained in the log that the user is impatient.

[0280] Step 5:

[0281] The server retrieves the "information about the maximum file size" from the database, and, considering the user's impatience, prioritizes a quick solution.

[0282] Step 6:

[0283] The server uses a generative AI model to generate a message saying, "The file may be too large. The maximum file size is 10MB. Please fix it immediately." Recognizing the user's impatience, the server then concisely emphasizes the solution.

[0284] Step 7:

[0285] The server sends the generated solution text to the terminal.

[0286] Step 8:

[0287] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[0288] The system allows users to receive immediate solutions when problems arise, as well as emotionally sensitive support.

[0289] Example 2

[0290] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0291] In modern IT services, it is extremely important to quickly and appropriately resolve problems users encounter during operation. However, while current systems can collect and analyze user operation logs to identify problems, it is difficult to provide support that takes the user's emotions into consideration. This can result in a failure to increase user satisfaction and a decline in service quality. The objective of this invention is to solve these problems and provide a system that supports users in resolving problems more quickly and accurately.

[0292] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0293] In this invention, the server includes means for analyzing the received operation log and identifying the problem and emotion the user is facing, means for retrieving appropriate support information for the identified problem and emotion from a database, and means for generating a solution message using a generative AI model based on the retrieved support information. This makes it possible to analyze the user's operation log and emotion in real time and provide appropriate support information, thereby increasing user satisfaction.

[0294] An "operation log" is data that records the operation history, including clicks, keystrokes, and error messages, when a user uses an IT service, along with timestamps.

[0295] "Operation characteristics" are data that capture detailed characteristics of operations, such as the speed of a user's operation, the strength of the click, and the movement of the mouse.

[0296] The "analysis means" is a function for analyzing the received operation log and identifying the problem the user is facing and the user's feelings from error messages and consecutive operation patterns.

[0297] "Support Information" means materials such as FAQs, guidelines, troubleshooting procedures, and the like that are provided to help users solve problems they may encounter.

[0298] A "database" is an information accumulation device in which support information is stored and a system for searching and acquiring the contents thereof.

[0299] A "generative AI model" is an artificial intelligence model used to automatically generate a solution text to be presented to the user based on the acquired support information.

[0300] A "solution text" is a specific guide text for solving a problem that is generated by a generative AI model and presented to the user.

[0301] "Emotion recognition" is the process of identifying a user's emotions, such as impatience or confusion, from operational characteristic information.

[0302] The present invention is a system that solves problems that arise when a user uses an IT service in real time. This system incorporates an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[0303] Hardware and software used

[0304] This system is primarily composed of a terminal and a server. The terminal collects user operation logs in real time and sends them to the server at regular intervals. The server, in turn, analyzes the received operation logs, identifies the user's problems and feelings, and provides appropriate support information.

[0305] Terminal: A device that monitors and records user operation logs (clicks, keystrokes, error messages, etc.) in real time. Logs are also collected, including operation characteristics (speed, click strength, mouse movement, etc.).

[0306] Server: A system that receives and analyzes operation logs sent from the device. It retrieves support information from a database and generates a solution using a generative AI model.

[0307] Processing flow and actual operation

[0308] 1. Collecting user operation logs (device)

[0309] When a user uses an IT service, for example by clicking a "file upload" button, the device records this click event in a log with a timestamp. It also records operation characteristics such as click strength, speed, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[0310] 2. Sending operation logs (terminal)

[0311] The terminal sends the collected operation logs to the server at regular intervals (a few seconds to a few minutes). A secure and high-speed communication protocol (e.g., HTTPS) is used for this transmission.

[0312] 3. Receiving and analyzing operation logs (server)

[0313] The server analyzes the received operation logs. For the analysis, it uses log data containing operation characteristics. For example, if the number of operations increases suddenly, it can be determined that the user is in a hurry, and if the speed of operations slows down, it can be determined that the user is confused.

[0314] 4. Extracting appropriate support information (server)

[0315] The server retrieves appropriate support information for the identified problem and emotion from a database that includes FAQs, guidelines, troubleshooting procedures, etc. Taking into account the user's emotional information, the server provides detailed instructions for confused users and quick solutions for impatient users.

[0316] 5. Generation of solution text (server)

[0317] Based on the obtained support information, the server uses a generative AI model (e.g., GPT-4) to generate a solution message. This message is tailored to help users understand the problem. For example, a solution message might be generated: "The file may be too large. The maximum file size is 10 MB."

[0318] Example prompt sentence:

[0319] "A user attempted to upload a file but received an error message. Please provide information about the maximum file size to generate a quick solution."

[0320] 6. Presentation of the resolution text (terminal)

[0321] The generated solution text is sent from the server to the device and displayed on the user interface, allowing the user to receive specific solutions in real time. For example, the device may display a pop-up window telling the user, "The file may be too large. The maximum file size is 10MB."

[0322] In this way, the system can quickly resolve user problems and respond in a way that takes the user's feelings into consideration, thereby increasing user satisfaction and improving the quality of IT services.

[0323] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0324] Step 1: Collecting user operation logs (device)

[0325] The device monitors in real time the operations (clicks, keystrokes, error messages, etc.) performed by the user while using the service and records them as an operation log with a timestamp. It also collects operation characteristics such as operation speed, click strength, and mouse movement. Specifically, the device saves the timestamp of the click event, click strength, and mouse movement pattern in a log.

[0326] Input: User clicks, keystrokes, error messages

[0327] Data processing: Collection of operation logs and operation characteristics

[0328] Output: Operation log with timestamp

[0329] Step 2: Sending operation logs (terminal)

[0330] The terminal sends the collected operation logs to the server at regular intervals. This interval can be adjusted depending on the system settings, and is usually a few seconds to a few minutes. Specifically, the terminal processes the collected log data in batches and sends it using a secure communication protocol (e.g., HTTPS).

[0331] Input: Timestamp operation log

[0332] Data processing: batch processing of log data

[0333] Output: Operation log sent to the server

[0334] Step 3: Receiving and analyzing operation logs (server)

[0335] The server receives and analyzes the operation logs sent to it. It pays particular attention to error messages and consecutive operation patterns to identify the problems and emotions the user is facing. Specifically, it infers emotions such as impatience or confusion from the type and frequency of errors and the user's operation characteristics.

[0336] Input: Submitted operation log

[0337] Data processing: Log data analysis (error identification, emotion estimation)

[0338] Output: Identified issues and emotions

[0339] Step 4: Pull the appropriate support information (server)

[0340] The server retrieves appropriate support information from a database based on the identified problem and emotion. Support information includes FAQs, guidelines, troubleshooting procedures, etc., and extracts information based on the user's emotion. Specifically, the server executes a database query to search and retrieve the relevant support information.

[0341] Input: Identified issues and emotions

[0342] Data processing: information extraction from databases

[0343] Output: Appropriate supporting information

[0344] Step 5: Generate a solution (server)

[0345] The server generates a solution message using a generative AI model (e.g., GPT-4) based on the acquired support information. This message is adjusted to be easy for the user to understand, and may include a solution such as, "The file may be too large. The maximum file size is 10 MB." Specifically, the server sends a prompt to the generative AI model to generate the message.

[0346] Input: Appropriate supporting information

[0347] Data processing: Text generation using generative AI models

[0348] Output: Solution text

[0349] Step 6: Presentation of the solution (terminal)

[0350] The device displays the solution text received from the server on the user interface. The user can check this text and receive specific instructions for resolving the problem. Specifically, the device displays the solution text in a pop-up window or the like, and presents the solution to the user.

[0351] Input: Solution text received from the server

[0352] Data processing: Display of solution text

[0353] Output: Resolution text displayed in the user interface

[0354] (Application example 2)

[0355] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0356] When operating and maintaining factory robots, it is extremely important to quickly and accurately resolve problems that operators encounter. However, with current systems, when operators encounter a problem, it takes time to find a solution, often resulting in stress and confusion during the process. Furthermore, since solutions to problems are not standardized and depend on the skills and experience of each individual operator, it is difficult to provide consistent support. Given this background, there is a need for a system that not only identifies problems based on user operation logs, but also provides appropriate solutions taking into account the operator's emotions.

[0357] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0358] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user terminal, means for recognizing the user's emotions, and means for adjusting the solution based on the emotion information. This makes it possible to quickly identify the problem the operator is facing and provide an appropriate solution that takes emotions into consideration in real time.

[0359] A "user operation log" is data that records all operational events, such as inputs, clicks, and error messages, when an operator operates a factory robot.

[0360] A "server" is a computer system that receives and analyzes user operation logs.

[0361] "Analysis" refers to the process of examining the received operation logs in detail and identifying the problems the user is facing.

[0362] "Support information" is data that includes solutions and procedures for identified problems, and is information that helps the factory run smoothly.

[0363] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions to identified problems based on pre-trained data.

[0364] A "user terminal" is a device used to display the resolution text, and includes factory control panels, smartphones, smart glasses, etc.

[0365] "Means for recognizing emotions" refers to technology that analyzes data such as the operator's facial expressions and voice to identify their emotions at that time.

[0366] "Means for adjusting solutions based on emotional information" refers to a technique for changing the content and expression of the solution text depending on the recognized emotions.

[0367] This invention is a system that quickly resolves problems faced by operators during the operation and maintenance of factory robots. The system provides appropriate solutions in real time based on operation logs and user sentiment.

[0368] The user device collects the operator's operation log in real time. This operation log includes operation events such as clicks, inputs, and error messages. Operation characteristics such as operation speed, click strength, and duration are also collected. This makes it possible to capture changes in the operator's operating tendencies and emotions.

[0369] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information.

[0370] The server analyzes the received operation logs. The log data contains various events, and issues are identified by paying particular attention to error messages and consecutive operation patterns. The server also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion. This emotion recognition is achieved using a camera and voice recognition.

[0371] Once a problem is identified, the server pulls the appropriate support information from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account perceived emotional information, offering detailed instructions if the operator is confused or a quick solution if they are impatient.

[0372] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the operator. This solution message is tailored to make it easier for the operator to understand the problem. For example, a specific solution message such as "The sensor needs cleaning. Here are the steps:..." is generated.

[0373] The generated solution is sent to the user's device and displayed to the operator, who can then take action to quickly resolve the issue.

[0374] For example, if a factory robot encounters an error while picking a part, and the operator cannot find a solution, the system will display a solution such as "The sensor needs cleaning. Here are the steps:..."

[0375] An example of a prompt sentence is, "An error occurred during the part picking process. The sensor may not be working properly. Emotion recognition indicates that the operator is confused. Please provide a solution."

[0376] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0377] Step 1:

[0378] The terminal collects user operation logs in real time. The collected data includes operation events such as clicks, inputs, and error messages. It also collects operation characteristics such as operation speed, click strength, and mouse movement. The input is the user's operation events, and the output is the collected operation log data. This data collection utilizes the terminal's sensor data and interface monitoring software.

[0379] Step 2:

[0380] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information. The input is the collected operation log data, and the output is the operation log data transferred to the server. This data is sent using an HTTP POST request.

[0381] Step 3:

[0382] The operation log received by the server is analyzed. The log data contains various events, and problems are identified by paying particular attention to error messages and consecutive operation patterns. Furthermore, operation characteristic information is used to recognize user emotions. The input is the operation log data transferred to the server, and the output is the identified problems and emotion recognition results. A log analysis algorithm and an emotion recognition algorithm are used for the analysis.

[0383] Step 4:

[0384] The server retrieves appropriate support information for the identified problem from the database. Based on the identified problem and the user's emotion, relevant support information is obtained. The input is the identified problem and emotion recognition results, and the output is the support information. This data retrieval is performed using SQL queries.

[0385] Step 5:

[0386] A generative AI model is used to generate a solution message based on the support information retrieved by the server. The solution message is adjusted to make it easier for the operator to understand the problem. The input is the support information and emotion recognition results, and the output is the generated solution message. GPT-3 or a similar model is used as the generative AI model for generation.

[0387] Step 6:

[0388] The server sends the generated solution text to the terminal. The input is the generated solution text, and the output is the solution text transferred to the terminal. This data transmission uses WebSocket or HTTP POST request.

[0389] Step 7:

[0390] The device displays the solution text in the user interface. The user can then take action based on the presented solution. The input is the solution text transferred from the server, and the output is the solution text displayed to the user. This display is done using a pop-up window or notification system on the device.

[0391] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0392] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0393] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0394] [Second embodiment]

[0395] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0396] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0397] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0398] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0399] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0400] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0401] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0402] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0403] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0404] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0405] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0406] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0407] This invention relates to a system for solving problems that occur when users use IT services in real time. This system collects user operation logs, generates appropriate solutions based on those logs, and presents them to the users.

[0408] Collecting user operation logs (device)

[0409] The device has the ability to monitor user operations (clicks, inputs, error messages, etc.) in real time. Each operation event is logged with a timestamp. For example, if a user clicks the "Upload File" button but the upload fails, the operation and error message are recorded in the log.

[0410] Sending operation logs (terminal)

[0411] The collected operation logs are sent to the server at regular intervals, so that the server always has the latest operation information and can respond quickly if a problem occurs.

[0412] Receiving and analyzing operation logs (server)

[0413] The server has the ability to analyze the received operation logs. This analysis detects error messages and consecutive operation patterns, among other things, to identify problems the user may be facing. For example, if a file upload fails, the error message can be analyzed to identify the cause of the problem (e.g., file size exceeded).

[0414] Extracting appropriate support information (server)

[0415] The server has the ability to retrieve appropriate support information from the database for the identified problem, including FAQs, guidelines, troubleshooting procedures, etc. For example, in response to a file size exceeding error message, it retrieves "information about the maximum allowed file size."

[0416] Solution text generation (server)

[0417] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is tailored to help the user understand the problem. Specifically, a solution such as "The file may be too large. The maximum file size is 10MB" is generated.

[0418] Presentation of the resolution text (terminal)

[0419] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[0420] Specific examples

[0421] Example 1:

[0422] Here's what happens if a user tries to upload a file but fails:

[0423] 1. The user clicks the "upload button."

[0424] 2. The device will log this operation and also log a subsequent "upload failed" error.

[0425] 3. Logs are sent to the server at regular intervals.

[0426] 4. The server receives the log, analyzes it, and determines that the cause of the "upload failure" is excessive file size.

[0427] 5. The server retrieves the support information about the "maximum file size" from the database.

[0428] 6. Based on the information obtained, the generative AI model generates a solution message: "The file may be too large. The maximum file size is 10MB."

[0429] 7. The device displays the generated solution to the user and provides specific instructions for resolving the problem.

[0430] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process, resulting in faster problem resolution and a better user experience.

[0431] The processing flow will be explained below.

[0432] Step 1:

[0433] A user performs a specific action on the system, for example clicking a "File Upload" button.

[0434] Step 2:

[0435] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, etc.).

[0436] Step 3:

[0437] The terminal sends the collected log data to the server at regular intervals, which can be adjusted according to the system settings.

[0438] Step 4:

[0439] Analyze the log data received by the server. The log data contains various events, and identify problems by paying particular attention to error messages and consecutive operation patterns.

[0440] Step 5:

[0441] The server retrieves the appropriate support information for the identified problem from its database, which may include FAQs, guidelines, troubleshooting procedures, etc.

[0442] Step 6:

[0443] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model then adjusts the message appropriately to make it easier for the user to understand the problem.

[0444] Step 7:

[0445] The server generates a solution message and sends it to the device, which includes specific solutions and operational procedures.

[0446] Step 8:

[0447] The device will display the solution text in the user interface, allowing the user to take appropriate action to resolve the issue.

[0448] Specific examples

[0449] Example: File upload failure

[0450] Step 1:

[0451] The user clicks the "File Upload" button.

[0452] Step 2:

[0453] The device will log the click and then log the upload failure error message.

[0454] Step 3:

[0455] The terminal sends the collected log data to the server every 60 seconds.

[0456] Step 4:

[0457] Analyze the logs received by the server and identify the "upload failed" error.

[0458] Step 5:

[0459] The server retrieves support information about "file size exceeded" from the database.

[0460] Step 6:

[0461] The server uses the generative AI model to generate a solution message: "The file may be too large. The maximum file size is 10MB."

[0462] Step 7:

[0463] The server sends the generated solution text to the terminal.

[0464] Step 8:

[0465] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[0466] Example 1

[0467] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0468] Traditional IT services lacked the means to instantly resolve problems users faced. As a result, users took time to identify the cause of the problem and had to use multiple support channels to find the appropriate solution. In particular, when an error message occurred, the process of understanding the message and finding a solution was cumbersome, resulting in a poor user experience. To solve these problems, a system that automatically identifies problems based on user operation logs and provides appropriate support information is needed.

[0469] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0470] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for providing a prompt message to the generative AI model based on the cause of the problem, and means for displaying the generated solution message on a user interface. This enables the user to quickly identify the cause of the problem and immediately obtain appropriate support information and a solution.

[0471] A "user operation log" is an operation history that includes clicks, inputs, error messages, etc. that is generated when a user uses a system or application.

[0472] "Means of collecting data in real time" refers to the function of instantly capturing the operations performed by the user and recording that information as a log.

[0473] "Means for sending to a server" refers to the function of transferring collected operation logs to a server via a network such as the Internet periodically or based on certain conditions.

[0474] "Means for analyzing received operation logs" refers to the function of processing the data in the operation logs received by the server to detect error messages and operation patterns contained in the logs and identify the cause of the problem.

[0475] "Means for identifying problems faced by users" refers to the function of identifying errors or problems that users are currently facing from the analyzed operation logs.

[0476] "Means for retrieving appropriate support information from the database" refers to the function of searching and retrieving support information such as related solutions, FAQs, guidelines, etc. in the database according to the identified problem.

[0477] "Means for generating a solution message using a generative AI model" refers to the function of utilizing a generative AI model to create a solution message that is easy for users to understand based on the acquired support information and the cause of the problem.

[0478] "Means for providing prompt sentences" refers to the function of supplying sentences that provide the generative AI model with background information and instructions for generating an appropriate solution sentence.

[0479] "Means for displaying on the user interface" refers to the function of displaying the generated solution text on the screen of the user's terminal and providing it visually.

[0480] MODE FOR CARRYING OUT THE INVENTION

[0481] This invention relates to a system for solving problems that occur when a user uses an IT service in real time. This section describes the hardware and software required to configure this system, as well as its operating method.

[0482] Collecting user operation logs (device)

[0483] The device monitors user actions in real time. Events such as buttons clicked by the user, text entered, and error messages are logged with a timestamp. This typically involves sensors and monitoring software (e.g., keyloggers, click-monitoring tools).

[0484] For example, if a user clicks the "Upload button", if this operation is successful, a success event will be logged. Conversely, if the operation fails and an error message such as "File size too large" is displayed, this error information will also be logged.

[0485] Sending operation logs (terminal)

[0486] The collected operation logs are sent to the server at regular intervals. This sending process is performed in the background so as not to interfere with user operations. HTTP / HTTPS is used as the protocol. Specifically, logs are sent to the server in batches at regular volume or time intervals (e.g., every 5 seconds).

[0487] Server receives and analyzes logs

[0488] The server analyzes the received operation logs. For analysis, a database server (e.g., MySQL, PostgreSQL) and data processing scripts (e.g., Python, R) are used. Specifically, the log data is stored in a database and error messages and specific operation patterns are detected.

[0489] For example, if the server receives a "file upload failed" error message, it analyzes the logs to determine that the cause is an excessive file size. The results of this analysis are passed on to the next step.

[0490] Extracting appropriate support information (server)

[0491] Based on the analysis results, the server retrieves the appropriate support information from a database that stores FAQs, guidelines, troubleshooting procedures, etc. SQL queries are used to retrieve the support information.

[0492] For example, for the error message "File size exceeded," you will get support information about "Maximum file size," which will give the user information to understand the specific cause of the problem.

[0493] Solution text generation (server)

[0494] The server generates a solution message using a generative AI model (e.g., OpenAI GPT-4) based on the acquired support information. The generative AI model receives the support information and the cause of the problem as input, and outputs a message that provides a solution to the user in an easy-to-understand format.

[0495] The generated solution text will be specific, for example, "The file may be too large. The maximum file size is 10MB."

[0496] Presentation of the resolution text (terminal)

[0497] The generated solution text is sent to the device and displayed in the user interface using UI elements such as a pop-up window or a notification bar, allowing the user to instantly receive a concrete solution instead of an error message.

[0498] Prompt Sentence Examples

[0499] The prompt provided to the generative AI model looks something like this:

[0500] If the upload failure is due to excessive file size, please provide a solution such as "The file may be too large. The maximum file size is 10MB."

[0501] Using such detailed prompts allows the AI ​​model to generate more accurate and useful solutions.

[0502] This system will enable users to quickly identify the cause of a problem and immediately obtain appropriate support information and solutions, which is expected to significantly improve the experience of using IT services.

[0503] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0504] The flow of this system's program processing

[0505] Step 1: Collect user operation logs

[0506] Users operate systems and applications.

[0507] The device monitors each user action (click, input, error message, etc.) in real time and creates a time-stamped operation log.

[0508] Specifically, when a user clicks the "Upload button," the click event is recorded, and if the upload fails, an error message stating "File size too large" is also recorded.

[0509] Input: User actions (clicks, inputs, error messages, etc.)

[0510] Output: Operation log with timestamp

[0511] Step 2: Send the operation log

[0512] The terminal sends the collected operation logs to the server at regular intervals.

[0513] This sending process is done in the background and does not interfere with user operations. For example, a batch of operation logs is sent to the server every 5 seconds.

[0514] Input: Timestamp operation log

[0515] Output: Batch log data to be sent

[0516] Step 3: Receiving and saving the operation log

[0517] The server receives the operation log sent from the terminal and stores it in a database.

[0518] Specifically, the server opens the received log data and inserts each log entry into the appropriate table in the database.

[0519] Input: Batch log data to be sent

[0520] Output: Operation log saved in the database

[0521] Step 4: Analyze the operation log

[0522] The server analyzes the operation logs stored in the database.

[0523] In particular, it processes data to identify the cause of a problem by detecting error messages and operation patterns. For example, it searches for "upload failed" events in the log and identifies that the cause is "file size exceeded."

[0524] Input: Operation logs stored in the database

[0525] Output: Identified cause of the problem (e.g. file size exceeded)

[0526] Step 5: Pulling Support Information

[0527] The server retrieves appropriate support information from a database based on the identified problem.

[0528] Specifically, it runs an SQL query to get information about "Maximum File Size".

[0529] Input: Identified cause of the problem

[0530] Output: Retrieved support information (e.g. information about maximum file size)

[0531] Step 6: Generate a solution

[0532] The server generates a solution text using a generative AI model (e.g., OpenAI GPT-4) based on the obtained supporting information and analysis results.

[0533] The generative AI model generates a prompt and then generates a solution for the user, such as "The file may be too large. The maximum file size is 10MB."

[0534] Input: Retrieved support information, identified problem cause, prompt statement

[0535] Output: Generated solution text

[0536] Step 7: Present the solution

[0537] The server sends the generated solution text to the terminal.

[0538] The device will display the received solution text on the user interface, and the user can immediately obtain a specific solution, for example, as a notification bar or a pop-up window.

[0539] Input: Generated solution text

[0540] Output: Resolution text displayed in the user interface

[0541] (Application example 1)

[0542] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0543] When users encounter errors or problems during the payment process with electronic payment services, they must search for solutions themselves, making it difficult to resolve them quickly. This can lead to a poor user experience and a loss of reliability in the service. Furthermore, there is often no system in place to provide real-time solutions, often leaving users frustrated. This invention aims to resolve these issues and make the payment process smoother and more efficient for users.

[0544] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0545] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying a problem the user is facing, means for retrieving appropriate support information for the identified problem from a data storage device, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user device, and means for analyzing details of an error that occurred in a specific payment process based on the user's operation and quickly providing the solution message, thereby enabling users to quickly resolve problems they encounter in the electronic transaction process and improving their service usage experience.

[0546] A "user operation log" is data that records a series of operation events (clicks, inputs, error messages, etc.) performed by a user on an electronic device or system.

[0547] "Means of collecting data in real time" refers to a mechanism that instantly detects user operations and immediately records the operation information as data.

[0548] A "server" is a computer system that receives operation logs sent by users, analyzes the data, and presents solutions to problems.

[0549] The "means for analyzing and identifying the problems the user is facing" is a mechanism for analyzing error messages and operation patterns from the received operation log to detect the problems the user is currently experiencing.

[0550] "Support Information" is information that helps users solve problems they encounter, such as FAQs, guidelines, and troubleshooting procedures.

[0551] A "data storage device" is a device for long-term storage of support information and other important data in a computer system.

[0552] A "generative AI model" is an algorithm or program that uses artificial intelligence to automatically generate solutions or instructions in natural language based on input data.

[0553] A "solution statement" is a document that provides specific solutions or instructions for a problem that a user faces in an easy-to-understand format.

[0554] A "user device" is a device that a user operates, such as a smartphone, computer, or head-mounted display.

[0555] The "means for analyzing the details of an error" is a mechanism for deeply analyzing the error message contained in the operation log and clarifying the cause and details of the error.

[0556] "Means for rapid provision" refers to a mechanism for presenting analysis results and generated solution text to users without delay.

[0557] This invention is a system that collects user operation logs in real time, generates appropriate solutions based on those logs, and presents them to the user. Specifically, it aims to link the server and user terminals to quickly resolve problems users encounter during the electronic payment process.

[0558] Program processing explanation

[0559] Collecting user operation logs (device)

[0560] The user device monitors user actions (clicks, inputs, error messages, etc.) in real time and records them in a log, which is time-stamped and records each operation event in detail.

[0561] Sending operation logs (terminal)

[0562] The collected operation logs are temporarily stored on the user's device and then sent to the server at regular intervals. This process ensures that the server always has the latest operation information.

[0563] Receiving and analyzing operation logs (server)

[0564] The server receives operation logs sent from user terminals. The received logs are analyzed, and error messages and consecutive operation patterns are automatically detected. A log analysis algorithm is used for this analysis.

[0565] Extracting appropriate support information (server)

[0566] For problems identified as a result of the analysis, the server retrieves appropriate support information from its data store, including FAQs, guidelines, troubleshooting procedures, and the like.

[0567] Solution text generation (server)

[0568] Based on the extracted support information, the server generates a solution using a generative AI model. This generative AI model uses a natural language processing algorithm to automatically generate a solution based on the input data. For example, it can use OpenAI's API.

[0569] Presentation of the resolution text (terminal)

[0570] The generated solution text is sent from the server to the user's device and displayed on the user interface, allowing the user to receive specific instructions on how to solve the problem.

[0571] Specific examples

[0572] 1. The user enters their card information on the payment page and clicks the "Payment button."

[0573] 2. The user terminal will log this operation and immediately afterwards also log a "Payment failed" error message.

[0574] 3. Logs are sent to the server at regular intervals.

[0575] 4. The server receives the log, analyzes it, and determines that the cause of the "payment failure" is incorrect card information.

[0576] 5. The server uses a generative AI model (OpenAI API) to generate a solution message that reads, "Your card information may be incorrect. Please re-enter accurate information."

[0577] 6. The user device displays the generated solution text to the user and provides specific instructions for solving the problem.

[0578] Prompt Sentence Examples

[0579] Please provide a solution: Payment failed: Incorrect card details

[0580] This system allows users to quickly resolve any issues they encounter during the payment process, improving their service experience.

[0581] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0582] Step 1:

[0583] A user enters card information on the page of an electronic payment service and clicks the "Payment button." This operation is performed on the user's device. This input generates a user operation log, which includes a timestamp of the card information input.

[0584] Step 2:

[0585] The user device collects user operation logs (clicks, inputs, error messages, etc.) in real time. For example, clicking the payment button and error messages when a payment fails are recorded as logs. This log is time-stamped and accumulates the collected operation events.

[0586] Step 3:

[0587] The collected operation logs are temporarily stored on the user's device and sent to the server at regular intervals. The data entered here is the operation log data, which is then sent to the server. The server obtains the latest operation information in real time, so the system is designed to avoid delays in sending the logs.

[0588] Step 4:

[0589] The server receives operation logs sent from user terminals. It analyzes the received operation logs and automatically detects error messages and consecutive operation patterns in particular. The input here is the operation log sent to the server, and the output is the analysis results, including error messages. A log analysis algorithm is used for this analysis.

[0590] Step 5:

[0591] As a result of the analysis, error messages and specific operation patterns are detected to identify the problem the user is facing. Based on the analysis results, the server retrieves appropriate support information from a data storage device. Here, the input is the analysis results, and the output is the corresponding support information (FAQs, guidelines, troubleshooting procedures, etc.).

[0592] Step 6:

[0593] Based on the extracted support information, the server uses a generative AI model to generate a solution text. The generative AI model (e.g., OpenAI API) creates a solution text in natural language based on the input support information and analysis results. The input is support information and analysis results, and the output is the solution text.

[0594] Step 7:

[0595] The generated solution text is sent from the server to the user terminal. The input here is the solution text, and the output is the text displayed on the user terminal. The user terminal displays this solution text on the user interface and presents a specific solution to the user.

[0596] Prompt Sentence Examples

[0597] Please provide a solution: Payment failed: Incorrect card details

[0598] The above are the specific processing steps of the system program that realizes the application example. Through this series of processes, users can receive solutions to problems in real time and quickly resolve their issues.

[0599] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0600] This invention combines a system that solves problems that arise when users use IT services in real time with an emotion engine that recognizes the user's emotions. This system not only collects user operation logs and generates and presents appropriate solutions based on those logs, but also adjusts the solution text taking the user's emotions into account.

[0601] Collecting user operation logs (device)

[0602] The device monitors user operations (clicks, inputs, error messages, etc.) in real time and records each operation event in a log with a timestamp. The log also includes operation characteristics such as operation speed, click strength, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[0603] Sending operation logs (terminal)

[0604] The collected operation logs are sent to the server at regular intervals. This interval can be adjusted according to the system settings. The sent data includes basic operation information and operation characteristic information.

[0605] Receiving and analyzing operation logs (server)

[0606] The server analyzes the received operation logs. The log data contains various events, and it identifies problems by paying particular attention to error messages and consecutive operation patterns. It also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion.

[0607] Extracting appropriate support information (server)

[0608] The server retrieves appropriate support information for the identified problem from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account emotional information, providing detailed instructions if the user is confused or quick solutions if the user is impatient.

[0609] Solution text generation (server)

[0610] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is adjusted to make it easier for the user to understand the problem. Specifically, the generated solution might say, "The file may be too large. The maximum file size is 10MB." The way the solution is expressed and the level of detail are adjusted depending on the user's feelings.

[0611] Presentation of the resolution text (terminal)

[0612] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[0613] Specific examples

[0614] Example 1: File upload failure

[0615] 1. The user clicks the "File Upload" button.

[0616] 2. The device will log this operation and then log an error message about the failed upload. If the user has tried multiple times, this information will also be included in the log.

[0617] 3. Logs are sent to the server at regular intervals.

[0618] 4. The server receives the log, identifies the "upload failed" error, and realizes that the user is anxious.

[0619] 5. The server retrieves the "information about the maximum file size" from the database and generates a quick solution that takes into account the user's impatience.

[0620] 6. The server uses the generative AI model to generate a solution: "The file may be too large. Maximum file size is 10MB. Please fix it immediately."

[0621] 7. The server sends the generated solution text to the terminal.

[0622] 8. The device will display the solution text in a pop-up window and provide the user with a specific solution.

[0623] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process. Furthermore, by taking the user's emotions into consideration, the user's experience is further improved.

[0624] The processing flow will be explained below.

[0625] Step 1:

[0626] A user performs a specific action on the system, for example clicking a "File Upload" button.

[0627] Step 2:

[0628] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, operation speed, click strength, mouse movement).

[0629] Step 3:

[0630] The terminal sends the collected log data and operation characteristic information to the server at regular intervals (for example, every 60 seconds). This allows the server to always have the latest operation information and respond immediately.

[0631] Step 4:

[0632] The server analyzes the received log data. This analysis includes detecting error messages and consecutive operation patterns, and recognizing the user's emotions using operation characteristics information. For example, rapid consecutive clicks may indicate impatience, while slow operation speed may indicate confusion.

[0633] Step 5:

[0634] The server retrieves the appropriate support information for the identified issue from a database, including FAQs, guidelines, troubleshooting procedures, etc. Additionally, the retrieved information is tailored to the user's perceived sentiment.

[0635] Step 6:

[0636] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model adjusts the expression and level of detail of the message depending on the user's emotions. For example, it might generate a message such as "The file may be too large. The maximum file size is 10MB. Please fix it immediately" for a user who is expressing impatience.

[0637] Step 7:

[0638] The server sends the generated solution text to the device, which includes specific solutions and operational procedures.

[0639] Step 8:

[0640] The device will display the solution text in the user interface, allowing users to instantly receive a specific solution instead of an error message, and users can take action based on this information to quickly resolve the issue.

[0641] Specific examples

[0642] Example: File upload failure

[0643] Step 1:

[0644] The user clicks the "File Upload" button.

[0645] Step 2:

[0646] The device logs the click, then logs the upload failure error message, and if the user retries multiple times, logs information such as the speed of the click and the strength of the click.

[0647] Step 3:

[0648] The terminal sends the collected log data and operational characteristics information to the server every 60 seconds.

[0649] Step 4:

[0650] The server receives the log, identifies the "upload failed" error, and recognizes from the operational characteristics information contained in the log that the user is impatient.

[0651] Step 5:

[0652] The server retrieves the "information about the maximum file size" from the database, and, considering the user's impatience, prioritizes a quick solution.

[0653] Step 6:

[0654] The server uses a generative AI model to generate a message saying, "The file may be too large. The maximum file size is 10MB. Please fix it immediately." Recognizing the user's impatience, the server then concisely emphasizes the solution.

[0655] Step 7:

[0656] The server sends the generated solution text to the terminal.

[0657] Step 8:

[0658] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[0659] The system allows users to receive immediate solutions when problems arise, as well as emotionally sensitive support.

[0660] Example 2

[0661] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0662] In modern IT services, it is extremely important to quickly and appropriately resolve problems users encounter during operation. However, while current systems can collect and analyze user operation logs to identify problems, it is difficult to provide support that takes the user's emotions into consideration. This can result in a failure to increase user satisfaction and a decline in service quality. The objective of this invention is to solve these problems and provide a system that supports users in resolving problems more quickly and accurately.

[0663] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0664] In this invention, the server includes means for analyzing the received operation log and identifying the problem and emotion the user is facing, means for retrieving appropriate support information for the identified problem and emotion from a database, and means for generating a solution message using a generative AI model based on the retrieved support information. This makes it possible to analyze the user's operation log and emotion in real time and provide appropriate support information, thereby increasing user satisfaction.

[0665] An "operation log" is data that records the operation history, including clicks, keystrokes, and error messages, when a user uses an IT service, along with timestamps.

[0666] "Operation characteristics" are data that capture detailed characteristics of operations, such as the speed of a user's operation, the strength of the click, and the movement of the mouse.

[0667] The "analysis means" is a function for analyzing the received operation log and identifying the problem the user is facing and the user's feelings from error messages and consecutive operation patterns.

[0668] "Support Information" means materials such as FAQs, guidelines, troubleshooting procedures, and the like that are provided to help users solve problems they may encounter.

[0669] A "database" is an information accumulation device in which support information is stored and a system for searching and acquiring the contents thereof.

[0670] A "generative AI model" is an artificial intelligence model used to automatically generate a solution text to be presented to the user based on the acquired support information.

[0671] A "solution text" is a specific guide text for solving a problem that is generated by a generative AI model and presented to the user.

[0672] "Emotion recognition" is the process of identifying a user's emotions, such as impatience or confusion, from operational characteristic information.

[0673] The present invention is a system that solves problems that arise when a user uses an IT service in real time. This system incorporates an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[0674] Hardware and software used

[0675] This system is primarily composed of a terminal and a server. The terminal collects user operation logs in real time and sends them to the server at regular intervals. The server, in turn, analyzes the received operation logs, identifies the user's problems and feelings, and provides appropriate support information.

[0676] Terminal: A device that monitors and records user operation logs (clicks, keystrokes, error messages, etc.) in real time. Logs are also collected, including operation characteristics (speed, click strength, mouse movement, etc.).

[0677] Server: A system that receives and analyzes operation logs sent from the device. It retrieves support information from a database and generates a solution using a generative AI model.

[0678] Processing flow and actual operation

[0679] 1. Collecting user operation logs (device)

[0680] When a user uses an IT service, for example by clicking a "file upload" button, the device records this click event in a log with a timestamp. It also records operation characteristics such as click strength, speed, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[0681] 2. Sending operation logs (terminal)

[0682] The terminal sends the collected operation logs to the server at regular intervals (a few seconds to a few minutes). A secure and high-speed communication protocol (e.g., HTTPS) is used for this transmission.

[0683] 3. Receiving and analyzing operation logs (server)

[0684] The server analyzes the received operation logs. For the analysis, it uses log data containing operation characteristics. For example, if the number of operations increases suddenly, it can be determined that the user is in a hurry, and if the speed of operations slows down, it can be determined that the user is confused.

[0685] 4. Extracting appropriate support information (server)

[0686] The server retrieves appropriate support information for the identified problem and emotion from a database that includes FAQs, guidelines, troubleshooting procedures, etc. Taking into account the user's emotional information, the server provides detailed instructions for confused users and quick solutions for impatient users.

[0687] 5. Generation of solution text (server)

[0688] Based on the obtained support information, the server uses a generative AI model (e.g., GPT-4) to generate a solution message. This message is tailored to help users understand the problem. For example, a solution message might be generated: "The file may be too large. The maximum file size is 10 MB."

[0689] Example prompt sentence:

[0690] "A user attempted to upload a file but received an error message. Please provide information about the maximum file size to generate a quick solution."

[0691] 6. Presentation of the resolution text (terminal)

[0692] The generated solution text is sent from the server to the device and displayed on the user interface, allowing the user to receive specific solutions in real time. For example, the device may display a pop-up window telling the user, "The file may be too large. The maximum file size is 10MB."

[0693] In this way, the system can quickly resolve user problems and respond in a way that takes the user's feelings into consideration, thereby increasing user satisfaction and improving the quality of IT services.

[0694] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0695] Step 1: Collecting user operation logs (device)

[0696] The device monitors in real time the operations (clicks, keystrokes, error messages, etc.) performed by the user while using the service and records them as an operation log with a timestamp. It also collects operation characteristics such as operation speed, click strength, and mouse movement. Specifically, the device saves the timestamp of the click event, click strength, and mouse movement pattern in a log.

[0697] Input: User clicks, keystrokes, error messages

[0698] Data processing: Collection of operation logs and operation characteristics

[0699] Output: Operation log with timestamp

[0700] Step 2: Sending operation logs (terminal)

[0701] The terminal sends the collected operation logs to the server at regular intervals. This interval can be adjusted depending on the system settings, and is usually a few seconds to a few minutes. Specifically, the terminal processes the collected log data in batches and sends it using a secure communication protocol (e.g., HTTPS).

[0702] Input: Timestamp operation log

[0703] Data processing: batch processing of log data

[0704] Output: Operation log sent to the server

[0705] Step 3: Receiving and analyzing operation logs (server)

[0706] The server receives and analyzes the operation logs sent to it. It pays particular attention to error messages and consecutive operation patterns to identify the problems and emotions the user is facing. Specifically, it infers emotions such as impatience or confusion from the type and frequency of errors and the user's operation characteristics.

[0707] Input: Submitted operation log

[0708] Data processing: Log data analysis (error identification, emotion estimation)

[0709] Output: Identified issues and emotions

[0710] Step 4: Pull the appropriate support information (server)

[0711] The server retrieves appropriate support information from a database based on the identified problem and emotion. Support information includes FAQs, guidelines, troubleshooting procedures, etc., and extracts information based on the user's emotion. Specifically, the server executes a database query to search and retrieve the relevant support information.

[0712] Input: Identified issues and emotions

[0713] Data processing: information extraction from databases

[0714] Output: Appropriate supporting information

[0715] Step 5: Generate a solution (server)

[0716] The server generates a solution message using a generative AI model (e.g., GPT-4) based on the acquired support information. This message is adjusted to be easy for the user to understand, and may include a solution such as, "The file may be too large. The maximum file size is 10 MB." Specifically, the server sends a prompt to the generative AI model to generate the message.

[0717] Input: Appropriate supporting information

[0718] Data processing: Text generation using generative AI models

[0719] Output: Solution text

[0720] Step 6: Presentation of the solution (terminal)

[0721] The device displays the solution text received from the server on the user interface. The user can check this text and receive specific instructions for resolving the problem. Specifically, the device displays the solution text in a pop-up window or the like, and presents the solution to the user.

[0722] Input: Solution text received from the server

[0723] Data processing: Display of solution text

[0724] Output: Resolution text displayed in the user interface

[0725] (Application example 2)

[0726] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0727] When operating and maintaining factory robots, it is extremely important to quickly and accurately resolve problems that operators encounter. However, with current systems, when operators encounter a problem, it takes time to find a solution, often resulting in stress and confusion during the process. Furthermore, since solutions to problems are not standardized and depend on the skills and experience of each individual operator, it is difficult to provide consistent support. Given this background, there is a need for a system that not only identifies problems based on user operation logs, but also provides appropriate solutions taking into account the operator's emotions.

[0728] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0729] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user terminal, means for recognizing the user's emotions, and means for adjusting the solution based on the emotion information. This makes it possible to quickly identify the problem the operator is facing and provide an appropriate solution that takes emotions into consideration in real time.

[0730] A "user operation log" is data that records all operational events, such as inputs, clicks, and error messages, when an operator operates a factory robot.

[0731] A "server" is a computer system that receives and analyzes user operation logs.

[0732] "Analysis" refers to the process of examining the received operation logs in detail and identifying the problems the user is facing.

[0733] "Support information" is data that includes solutions and procedures for identified problems, and is information that helps the factory run smoothly.

[0734] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions to identified problems based on pre-trained data.

[0735] A "user terminal" is a device used to display the resolution text, and includes factory control panels, smartphones, smart glasses, etc.

[0736] "Means for recognizing emotions" refers to technology that analyzes data such as the operator's facial expressions and voice to identify their emotions at that time.

[0737] "Means for adjusting solutions based on emotional information" refers to a technique for changing the content and expression of the solution text depending on the recognized emotions.

[0738] This invention is a system that quickly resolves problems faced by operators during the operation and maintenance of factory robots. The system provides appropriate solutions in real time based on operation logs and user sentiment.

[0739] The user device collects the operator's operation log in real time. This operation log includes operation events such as clicks, inputs, and error messages. Operation characteristics such as operation speed, click strength, and duration are also collected. This makes it possible to capture changes in the operator's operating tendencies and emotions.

[0740] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information.

[0741] The server analyzes the received operation logs. The log data contains various events, and issues are identified by paying particular attention to error messages and consecutive operation patterns. The server also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion. This emotion recognition is achieved using a camera and voice recognition.

[0742] Once a problem is identified, the server pulls the appropriate support information from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account perceived emotional information, offering detailed instructions if the operator is confused or a quick solution if they are impatient.

[0743] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the operator. This solution message is tailored to make it easier for the operator to understand the problem. For example, a specific solution message such as "The sensor needs cleaning. Here are the steps:..." is generated.

[0744] The generated solution is sent to the user's device and displayed to the operator, who can then take action to quickly resolve the issue.

[0745] For example, if a factory robot encounters an error while picking a part, and the operator cannot find a solution, the system will display a solution such as "The sensor needs cleaning. Here are the steps:..."

[0746] An example of a prompt sentence is, "An error occurred during the part picking process. The sensor may not be working properly. Emotion recognition indicates that the operator is confused. Please provide a solution."

[0747] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0748] Step 1:

[0749] The terminal collects user operation logs in real time. The collected data includes operation events such as clicks, inputs, and error messages. It also collects operation characteristics such as operation speed, click strength, and mouse movement. The input is the user's operation events, and the output is the collected operation log data. This data collection utilizes the terminal's sensor data and interface monitoring software.

[0750] Step 2:

[0751] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information. The input is the collected operation log data, and the output is the operation log data transferred to the server. This data is sent using an HTTP POST request.

[0752] Step 3:

[0753] The operation log received by the server is analyzed. The log data contains various events, and problems are identified by paying particular attention to error messages and consecutive operation patterns. Furthermore, operation characteristic information is used to recognize user emotions. The input is the operation log data transferred to the server, and the output is the identified problems and emotion recognition results. A log analysis algorithm and an emotion recognition algorithm are used for the analysis.

[0754] Step 4:

[0755] The server retrieves appropriate support information for the identified problem from the database. Based on the identified problem and the user's emotion, relevant support information is obtained. The input is the identified problem and emotion recognition results, and the output is the support information. This data retrieval is performed using SQL queries.

[0756] Step 5:

[0757] A generative AI model is used to generate a solution message based on the support information retrieved by the server. The solution message is adjusted to make it easier for the operator to understand the problem. The input is the support information and emotion recognition results, and the output is the generated solution message. GPT-3 or a similar model is used as the generative AI model for generation.

[0758] Step 6:

[0759] The server sends the generated solution text to the terminal. The input is the generated solution text, and the output is the solution text transferred to the terminal. This data transmission uses WebSocket or HTTP POST request.

[0760] Step 7:

[0761] The device displays the solution text in the user interface. The user can then take action based on the presented solution. The input is the solution text transferred from the server, and the output is the solution text displayed to the user. This display is done using a pop-up window or notification system on the device.

[0762] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0763] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0764] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0765] [Third embodiment]

[0766] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0767] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0768] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0769] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0770] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0771] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0772] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0773] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0774] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0775] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0776] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0777] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0778] This invention relates to a system for solving problems that occur when users use IT services in real time. This system collects user operation logs, generates appropriate solutions based on those logs, and presents them to the users.

[0779] Collecting user operation logs (device)

[0780] The device has the ability to monitor user operations (clicks, inputs, error messages, etc.) in real time. Each operation event is logged with a timestamp. For example, if a user clicks the "Upload File" button but the upload fails, the operation and error message are recorded in the log.

[0781] Sending operation logs (terminal)

[0782] The collected operation logs are sent to the server at regular intervals, so that the server always has the latest operation information and can respond quickly if a problem occurs.

[0783] Receiving and analyzing operation logs (server)

[0784] The server has the ability to analyze the received operation logs. This analysis detects error messages and consecutive operation patterns, among other things, to identify problems the user may be facing. For example, if a file upload fails, the error message can be analyzed to identify the cause of the problem (e.g., file size exceeded).

[0785] Extracting appropriate support information (server)

[0786] The server has the ability to retrieve appropriate support information from the database for the identified problem, including FAQs, guidelines, troubleshooting procedures, etc. For example, in response to a file size exceeding error message, it retrieves "information about the maximum allowed file size."

[0787] Solution text generation (server)

[0788] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is tailored to help the user understand the problem. Specifically, a solution such as "The file may be too large. The maximum file size is 10MB" is generated.

[0789] Presentation of the resolution text (terminal)

[0790] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[0791] Specific examples

[0792] Example 1:

[0793] Here's what happens if a user tries to upload a file but fails:

[0794] 1. The user clicks the "upload button."

[0795] 2. The device will log this operation and also log a subsequent "upload failed" error.

[0796] 3. Logs are sent to the server at regular intervals.

[0797] 4. The server receives the log, analyzes it, and determines that the cause of the "upload failure" is excessive file size.

[0798] 5. The server retrieves the support information about the "maximum file size" from the database.

[0799] 6. Based on the information obtained, the generative AI model generates a solution message: "The file may be too large. The maximum file size is 10MB."

[0800] 7. The device displays the generated solution to the user and provides specific instructions for resolving the problem.

[0801] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process, resulting in faster problem resolution and a better user experience.

[0802] The processing flow will be explained below.

[0803] Step 1:

[0804] A user performs a specific action on the system, for example clicking a "File Upload" button.

[0805] Step 2:

[0806] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, etc.).

[0807] Step 3:

[0808] The terminal sends the collected log data to the server at regular intervals, which can be adjusted according to the system settings.

[0809] Step 4:

[0810] Analyze the log data received by the server. The log data contains various events, and identify problems by paying particular attention to error messages and consecutive operation patterns.

[0811] Step 5:

[0812] The server retrieves the appropriate support information for the identified problem from its database, which may include FAQs, guidelines, troubleshooting procedures, etc.

[0813] Step 6:

[0814] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model then adjusts the message appropriately to make it easier for the user to understand the problem.

[0815] Step 7:

[0816] The server generates a solution message and sends it to the device, which includes specific solutions and operational procedures.

[0817] Step 8:

[0818] The device will display the solution text in the user interface, allowing the user to take appropriate action to resolve the issue.

[0819] Specific examples

[0820] Example: File upload failure

[0821] Step 1:

[0822] The user clicks the "File Upload" button.

[0823] Step 2:

[0824] The device will log the click and then log the upload failure error message.

[0825] Step 3:

[0826] The terminal sends the collected log data to the server every 60 seconds.

[0827] Step 4:

[0828] Analyze the logs received by the server and identify the "upload failed" error.

[0829] Step 5:

[0830] The server retrieves support information about "file size exceeded" from the database.

[0831] Step 6:

[0832] The server uses the generative AI model to generate a solution message: "The file may be too large. The maximum file size is 10MB."

[0833] Step 7:

[0834] The server sends the generated solution text to the terminal.

[0835] Step 8:

[0836] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[0837] Example 1

[0838] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0839] Traditional IT services lacked the means to instantly resolve problems users faced. As a result, users took time to identify the cause of the problem and had to use multiple support channels to find the appropriate solution. In particular, when an error message occurred, the process of understanding the message and finding a solution was cumbersome, resulting in a poor user experience. To solve these problems, a system that automatically identifies problems based on user operation logs and provides appropriate support information is needed.

[0840] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0841] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for providing a prompt message to the generative AI model based on the cause of the problem, and means for displaying the generated solution message on a user interface. This enables the user to quickly identify the cause of the problem and immediately obtain appropriate support information and a solution.

[0842] A "user operation log" is an operation history that includes clicks, inputs, error messages, etc. that is generated when a user uses a system or application.

[0843] "Means of collecting data in real time" refers to the function of instantly capturing the operations performed by the user and recording that information as a log.

[0844] "Means for sending to a server" refers to the function of transferring collected operation logs to a server via a network such as the Internet periodically or based on certain conditions.

[0845] "Means for analyzing received operation logs" refers to the function of processing the data in the operation logs received by the server to detect error messages and operation patterns contained in the logs and identify the cause of the problem.

[0846] "Means for identifying problems faced by users" refers to the function of identifying errors or problems that users are currently facing from the analyzed operation logs.

[0847] "Means for retrieving appropriate support information from the database" refers to the function of searching and retrieving support information such as related solutions, FAQs, guidelines, etc. in the database according to the identified problem.

[0848] "Means for generating a solution message using a generative AI model" refers to the function of utilizing a generative AI model to create a solution message that is easy for users to understand based on the acquired support information and the cause of the problem.

[0849] "Means for providing prompt sentences" refers to the function of supplying sentences that provide the generative AI model with background information and instructions for generating an appropriate solution sentence.

[0850] "Means for displaying on the user interface" refers to the function of displaying the generated solution text on the screen of the user's terminal and providing it visually.

[0851] MODE FOR CARRYING OUT THE INVENTION

[0852] This invention relates to a system for solving problems that occur when a user uses an IT service in real time. This section describes the hardware and software required to configure this system, as well as its operating method.

[0853] Collecting user operation logs (device)

[0854] The device monitors user actions in real time. Events such as buttons clicked by the user, text entered, and error messages are logged with a timestamp. This typically involves sensors and monitoring software (e.g., keyloggers, click-monitoring tools).

[0855] For example, if a user clicks the "Upload button", if this operation is successful, a success event will be logged. Conversely, if the operation fails and an error message such as "File size too large" is displayed, this error information will also be logged.

[0856] Sending operation logs (terminal)

[0857] The collected operation logs are sent to the server at regular intervals. This sending process is performed in the background so as not to interfere with user operations. HTTP / HTTPS is used as the protocol. Specifically, logs are sent to the server in batches at regular volume or time intervals (e.g., every 5 seconds).

[0858] Server receives and analyzes logs

[0859] The server analyzes the received operation logs. For analysis, a database server (e.g., MySQL, PostgreSQL) and data processing scripts (e.g., Python, R) are used. Specifically, the log data is stored in a database and error messages and specific operation patterns are detected.

[0860] For example, if the server receives a "file upload failed" error message, it analyzes the logs to determine that the cause is an excessive file size. The results of this analysis are passed on to the next step.

[0861] Extracting appropriate support information (server)

[0862] Based on the analysis results, the server retrieves the appropriate support information from a database that stores FAQs, guidelines, troubleshooting procedures, etc. SQL queries are used to retrieve the support information.

[0863] For example, for the error message "File size exceeded," you will get support information about "Maximum file size," which will give the user information to understand the specific cause of the problem.

[0864] Solution text generation (server)

[0865] The server generates a solution message using a generative AI model (e.g., OpenAI GPT-4) based on the acquired support information. The generative AI model receives the support information and the cause of the problem as input, and outputs a message that provides a solution to the user in an easy-to-understand format.

[0866] The generated solution text will be specific, for example, "The file may be too large. The maximum file size is 10MB."

[0867] Presentation of the resolution text (terminal)

[0868] The generated solution text is sent to the device and displayed in the user interface using UI elements such as a pop-up window or a notification bar, allowing the user to instantly receive a concrete solution instead of an error message.

[0869] Prompt Sentence Examples

[0870] The prompt provided to the generative AI model looks something like this:

[0871] If the upload failure is due to excessive file size, please provide a solution such as "The file may be too large. The maximum file size is 10MB."

[0872] Using such detailed prompts allows the AI ​​model to generate more accurate and useful solutions.

[0873] This system will enable users to quickly identify the cause of a problem and immediately obtain appropriate support information and solutions, which is expected to significantly improve the experience of using IT services.

[0874] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0875] The flow of this system's program processing

[0876] Step 1: Collect user operation logs

[0877] Users operate systems and applications.

[0878] The device monitors each user action (click, input, error message, etc.) in real time and creates a time-stamped operation log.

[0879] Specifically, when a user clicks the "Upload button," the click event is recorded, and if the upload fails, an error message stating "File size too large" is also recorded.

[0880] Input: User actions (clicks, inputs, error messages, etc.)

[0881] Output: Operation log with timestamp

[0882] Step 2: Send the operation log

[0883] The terminal sends the collected operation logs to the server at regular intervals.

[0884] This sending process is done in the background and does not interfere with user operations. For example, a batch of operation logs is sent to the server every 5 seconds.

[0885] Input: Timestamp operation log

[0886] Output: Batch log data to be sent

[0887] Step 3: Receiving and saving the operation log

[0888] The server receives the operation log sent from the terminal and stores it in a database.

[0889] Specifically, the server opens the received log data and inserts each log entry into the appropriate table in the database.

[0890] Input: Batch log data to be sent

[0891] Output: Operation log saved in the database

[0892] Step 4: Analyze the operation log

[0893] The server analyzes the operation logs stored in the database.

[0894] In particular, it processes data to identify the cause of a problem by detecting error messages and operation patterns. For example, it searches for "upload failed" events in the log and identifies that the cause is "file size exceeded."

[0895] Input: Operation logs stored in the database

[0896] Output: Identified cause of the problem (e.g. file size exceeded)

[0897] Step 5: Pulling Support Information

[0898] The server retrieves appropriate support information from a database based on the identified problem.

[0899] Specifically, it runs an SQL query to get information about "Maximum File Size".

[0900] Input: Identified cause of the problem

[0901] Output: Retrieved support information (e.g. information about maximum file size)

[0902] Step 6: Generate a solution

[0903] The server generates a solution text using a generative AI model (e.g., OpenAI GPT-4) based on the obtained supporting information and analysis results.

[0904] The generative AI model generates a prompt and then generates a solution for the user, such as "The file may be too large. The maximum file size is 10MB."

[0905] Input: Retrieved support information, identified problem cause, prompt statement

[0906] Output: Generated solution text

[0907] Step 7: Present the solution

[0908] The server sends the generated solution text to the terminal.

[0909] The device will display the received solution text on the user interface, and the user can immediately obtain a specific solution, for example, as a notification bar or a pop-up window.

[0910] Input: Generated solution text

[0911] Output: Resolution text displayed in the user interface

[0912] (Application example 1)

[0913] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0914] When users encounter errors or problems during the payment process with electronic payment services, they must search for solutions themselves, making it difficult to resolve them quickly. This can lead to a poor user experience and a loss of reliability in the service. Furthermore, there is often no system in place to provide real-time solutions, often leaving users frustrated. This invention aims to resolve these issues and make the payment process smoother and more efficient for users.

[0915] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0916] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying a problem the user is facing, means for retrieving appropriate support information for the identified problem from a data storage device, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user device, and means for analyzing details of an error that occurred in a specific payment process based on the user's operation and quickly providing the solution message, thereby enabling users to quickly resolve problems they encounter in the electronic transaction process and improving their service usage experience.

[0917] A "user operation log" is data that records a series of operation events (clicks, inputs, error messages, etc.) performed by a user on an electronic device or system.

[0918] "Means of collecting data in real time" refers to a mechanism that instantly detects user operations and immediately records the operation information as data.

[0919] A "server" is a computer system that receives operation logs sent by users, analyzes the data, and presents solutions to problems.

[0920] The "means for analyzing and identifying the problems the user is facing" is a mechanism for analyzing error messages and operation patterns from the received operation log to detect the problems the user is currently experiencing.

[0921] "Support Information" is information that helps users solve problems they encounter, such as FAQs, guidelines, and troubleshooting procedures.

[0922] A "data storage device" is a device for long-term storage of support information and other important data in a computer system.

[0923] A "generative AI model" is an algorithm or program that uses artificial intelligence to automatically generate solutions or instructions in natural language based on input data.

[0924] A "solution statement" is a document that provides specific solutions or instructions for a problem that a user faces in an easy-to-understand format.

[0925] A "user device" is a device that a user operates, such as a smartphone, computer, or head-mounted display.

[0926] The "means for analyzing the details of an error" is a mechanism for deeply analyzing the error message contained in the operation log and clarifying the cause and details of the error.

[0927] "Means for rapid provision" refers to a mechanism for presenting analysis results and generated solution text to users without delay.

[0928] This invention is a system that collects user operation logs in real time, generates appropriate solutions based on those logs, and presents them to the user. Specifically, it aims to link the server and user terminals to quickly resolve problems users encounter during the electronic payment process.

[0929] Program processing explanation

[0930] Collecting user operation logs (device)

[0931] The user device monitors user actions (clicks, inputs, error messages, etc.) in real time and records them in a log, which is time-stamped and records each operation event in detail.

[0932] Sending operation logs (terminal)

[0933] The collected operation logs are temporarily stored on the user's device and then sent to the server at regular intervals. This process ensures that the server always has the latest operation information.

[0934] Receiving and analyzing operation logs (server)

[0935] The server receives operation logs sent from user terminals. The received logs are analyzed, and error messages and consecutive operation patterns are automatically detected. A log analysis algorithm is used for this analysis.

[0936] Extracting appropriate support information (server)

[0937] For problems identified as a result of the analysis, the server retrieves appropriate support information from its data store, including FAQs, guidelines, troubleshooting procedures, and the like.

[0938] Solution text generation (server)

[0939] Based on the extracted support information, the server generates a solution using a generative AI model. This generative AI model uses a natural language processing algorithm to automatically generate a solution based on the input data. For example, it can use OpenAI's API.

[0940] Presentation of the resolution text (terminal)

[0941] The generated solution text is sent from the server to the user's device and displayed on the user interface, allowing the user to receive specific instructions on how to solve the problem.

[0942] Specific examples

[0943] 1. The user enters their card information on the payment page and clicks the "Payment button."

[0944] 2. The user terminal will log this operation and immediately afterwards also log a "Payment failed" error message.

[0945] 3. Logs are sent to the server at regular intervals.

[0946] 4. The server receives the log, analyzes it, and determines that the cause of the "payment failure" is incorrect card information.

[0947] 5. The server uses a generative AI model (OpenAI API) to generate a solution message that reads, "Your card information may be incorrect. Please re-enter accurate information."

[0948] 6. The user device displays the generated solution text to the user and provides specific instructions for solving the problem.

[0949] Prompt Sentence Examples

[0950] Please provide a solution: Payment failed: Incorrect card details

[0951] This system allows users to quickly resolve any issues they encounter during the payment process, improving their service experience.

[0952] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0953] Step 1:

[0954] A user enters card information on the page of an electronic payment service and clicks the "Payment button." This operation is performed on the user's device. This input generates a user operation log, which includes a timestamp of the card information input.

[0955] Step 2:

[0956] The user device collects user operation logs (clicks, inputs, error messages, etc.) in real time. For example, clicking the payment button and error messages when a payment fails are recorded as logs. This log is time-stamped and accumulates the collected operation events.

[0957] Step 3:

[0958] The collected operation logs are temporarily stored on the user's device and sent to the server at regular intervals. The data entered here is the operation log data, which is then sent to the server. The server obtains the latest operation information in real time, so the system is designed to avoid delays in sending the logs.

[0959] Step 4:

[0960] The server receives operation logs sent from user terminals. It analyzes the received operation logs and automatically detects error messages and consecutive operation patterns in particular. The input here is the operation log sent to the server, and the output is the analysis results, including error messages. A log analysis algorithm is used for this analysis.

[0961] Step 5:

[0962] As a result of the analysis, error messages and specific operation patterns are detected to identify the problem the user is facing. Based on the analysis results, the server retrieves appropriate support information from a data storage device. Here, the input is the analysis results, and the output is the corresponding support information (FAQs, guidelines, troubleshooting procedures, etc.).

[0963] Step 6:

[0964] Based on the extracted support information, the server uses a generative AI model to generate a solution text. The generative AI model (e.g., OpenAI API) creates a solution text in natural language based on the input support information and analysis results. The input is support information and analysis results, and the output is the solution text.

[0965] Step 7:

[0966] The generated solution text is sent from the server to the user terminal. The input here is the solution text, and the output is the text displayed on the user terminal. The user terminal displays this solution text on the user interface and presents a specific solution to the user.

[0967] Prompt Sentence Examples

[0968] Please provide a solution: Payment failed: Incorrect card details

[0969] The above are the specific processing steps of the system program that realizes the application example. Through this series of processes, users can receive solutions to problems in real time and quickly resolve their issues.

[0970] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0971] This invention combines a system that solves problems that arise when users use IT services in real time with an emotion engine that recognizes the user's emotions. This system not only collects user operation logs and generates and presents appropriate solutions based on those logs, but also adjusts the solution text taking the user's emotions into account.

[0972] Collecting user operation logs (device)

[0973] The device monitors user operations (clicks, inputs, error messages, etc.) in real time and records each operation event in a log with a timestamp. The log also includes operation characteristics such as operation speed, click strength, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[0974] Sending operation logs (terminal)

[0975] The collected operation logs are sent to the server at regular intervals. This interval can be adjusted according to the system settings. The sent data includes basic operation information and operation characteristic information.

[0976] Receiving and analyzing operation logs (server)

[0977] The server analyzes the received operation logs. The log data contains various events, and it identifies problems by paying particular attention to error messages and consecutive operation patterns. It also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion.

[0978] Extracting appropriate support information (server)

[0979] The server retrieves appropriate support information for the identified problem from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account emotional information, providing detailed instructions if the user is confused or quick solutions if the user is impatient.

[0980] Solution text generation (server)

[0981] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is adjusted to make it easier for the user to understand the problem. Specifically, the generated solution might say, "The file may be too large. The maximum file size is 10MB." The way the solution is expressed and the level of detail are adjusted depending on the user's feelings.

[0982] Presentation of the resolution text (terminal)

[0983] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[0984] Specific examples

[0985] Example 1: File upload failure

[0986] 1. The user clicks the "File Upload" button.

[0987] 2. The device will log this operation and then log an error message about the failed upload. If the user has tried multiple times, this information will also be included in the log.

[0988] 3. Logs are sent to the server at regular intervals.

[0989] 4. The server receives the log, identifies the "upload failed" error, and realizes that the user is anxious.

[0990] 5. The server retrieves the "information about the maximum file size" from the database and generates a quick solution that takes into account the user's impatience.

[0991] 6. The server uses the generative AI model to generate a solution: "The file may be too large. Maximum file size is 10MB. Please fix it immediately."

[0992] 7. The server sends the generated solution text to the terminal.

[0993] 8. The device will display the solution text in a pop-up window and provide the user with a specific solution.

[0994] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process. Furthermore, by taking the user's emotions into consideration, the user's experience is further improved.

[0995] The processing flow will be explained below.

[0996] Step 1:

[0997] A user performs a specific action on the system, for example clicking a "File Upload" button.

[0998] Step 2:

[0999] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, operation speed, click strength, mouse movement).

[1000] Step 3:

[1001] The terminal sends the collected log data and operation characteristic information to the server at regular intervals (for example, every 60 seconds). This allows the server to always have the latest operation information and respond immediately.

[1002] Step 4:

[1003] The server analyzes the received log data. This analysis includes detecting error messages and consecutive operation patterns, and recognizing the user's emotions using operation characteristics information. For example, rapid consecutive clicks may indicate impatience, while slow operation speed may indicate confusion.

[1004] Step 5:

[1005] The server retrieves the appropriate support information for the identified issue from a database, including FAQs, guidelines, troubleshooting procedures, etc. Additionally, the retrieved information is tailored to the user's perceived sentiment.

[1006] Step 6:

[1007] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model adjusts the expression and level of detail of the message depending on the user's emotions. For example, it might generate a message such as "The file may be too large. The maximum file size is 10MB. Please fix it immediately" for a user who is expressing impatience.

[1008] Step 7:

[1009] The server sends the generated solution text to the device, which includes specific solutions and operational procedures.

[1010] Step 8:

[1011] The device will display the solution text in the user interface, allowing users to instantly receive a specific solution instead of an error message, and users can take action based on this information to quickly resolve the issue.

[1012] Specific examples

[1013] Example: File upload failure

[1014] Step 1:

[1015] The user clicks the "File Upload" button.

[1016] Step 2:

[1017] The device logs the click, then logs the upload failure error message, and if the user retries multiple times, logs information such as the speed of the click and the strength of the click.

[1018] Step 3:

[1019] The terminal sends the collected log data and operational characteristics information to the server every 60 seconds.

[1020] Step 4:

[1021] The server receives the log, identifies the "upload failed" error, and recognizes from the operational characteristics information contained in the log that the user is impatient.

[1022] Step 5:

[1023] The server retrieves the "information about the maximum file size" from the database, and, considering the user's impatience, prioritizes a quick solution.

[1024] Step 6:

[1025] The server uses a generative AI model to generate a message saying, "The file may be too large. The maximum file size is 10MB. Please fix it immediately." Recognizing the user's impatience, the server then concisely emphasizes the solution.

[1026] Step 7:

[1027] The server sends the generated solution text to the terminal.

[1028] Step 8:

[1029] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[1030] The system allows users to receive immediate solutions when problems arise, as well as emotionally sensitive support.

[1031] Example 2

[1032] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1033] In modern IT services, it is extremely important to quickly and appropriately resolve problems users encounter during operation. However, while current systems can collect and analyze user operation logs to identify problems, it is difficult to provide support that takes the user's emotions into consideration. This can result in a failure to increase user satisfaction and a decline in service quality. The objective of this invention is to solve these problems and provide a system that supports users in resolving problems more quickly and accurately.

[1034] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1035] In this invention, the server includes means for analyzing the received operation log and identifying the problem and emotion the user is facing, means for retrieving appropriate support information for the identified problem and emotion from a database, and means for generating a solution message using a generative AI model based on the retrieved support information. This makes it possible to analyze the user's operation log and emotion in real time and provide appropriate support information, thereby increasing user satisfaction.

[1036] An "operation log" is data that records the operation history, including clicks, keystrokes, and error messages, when a user uses an IT service, along with timestamps.

[1037] "Operation characteristics" are data that capture detailed characteristics of operations, such as the speed of a user's operation, the strength of the click, and the movement of the mouse.

[1038] The "analysis means" is a function for analyzing the received operation log and identifying the problem the user is facing and the user's feelings from error messages and consecutive operation patterns.

[1039] "Support Information" means materials such as FAQs, guidelines, troubleshooting procedures, and the like that are provided to help users solve problems they may encounter.

[1040] A "database" is an information accumulation device in which support information is stored and a system for searching and acquiring the contents thereof.

[1041] A "generative AI model" is an artificial intelligence model used to automatically generate a solution text to be presented to the user based on the acquired support information.

[1042] A "solution text" is a specific guide text for solving a problem that is generated by a generative AI model and presented to the user.

[1043] "Emotion recognition" is the process of identifying a user's emotions, such as impatience or confusion, from operational characteristic information.

[1044] The present invention is a system that solves problems that arise when a user uses an IT service in real time. This system incorporates an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[1045] Hardware and software used

[1046] This system is primarily composed of a terminal and a server. The terminal collects user operation logs in real time and sends them to the server at regular intervals. The server, in turn, analyzes the received operation logs, identifies the user's problems and feelings, and provides appropriate support information.

[1047] Terminal: A device that monitors and records user operation logs (clicks, keystrokes, error messages, etc.) in real time. Logs are also collected, including operation characteristics (speed, click strength, mouse movement, etc.).

[1048] Server: A system that receives and analyzes operation logs sent from the device. It retrieves support information from a database and generates a solution using a generative AI model.

[1049] Processing flow and actual operation

[1050] 1. Collecting user operation logs (device)

[1051] When a user uses an IT service, for example by clicking a "file upload" button, the device records this click event in a log with a timestamp. It also records operation characteristics such as click strength, speed, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[1052] 2. Sending operation logs (terminal)

[1053] The terminal sends the collected operation logs to the server at regular intervals (a few seconds to a few minutes). A secure and high-speed communication protocol (e.g., HTTPS) is used for this transmission.

[1054] 3. Receiving and analyzing operation logs (server)

[1055] The server analyzes the received operation logs. For the analysis, it uses log data containing operation characteristics. For example, if the number of operations increases suddenly, it can be determined that the user is in a hurry, and if the speed of operations slows down, it can be determined that the user is confused.

[1056] 4. Extracting appropriate support information (server)

[1057] The server retrieves appropriate support information for the identified problem and emotion from a database that includes FAQs, guidelines, troubleshooting procedures, etc. Taking into account the user's emotional information, the server provides detailed instructions for confused users and quick solutions for impatient users.

[1058] 5. Generation of solution text (server)

[1059] Based on the obtained support information, the server uses a generative AI model (e.g., GPT-4) to generate a solution message. This message is tailored to help users understand the problem. For example, a solution message might be generated: "The file may be too large. The maximum file size is 10 MB."

[1060] Example prompt sentence:

[1061] "A user attempted to upload a file but received an error message. Please provide information about the maximum file size to generate a quick solution."

[1062] 6. Presentation of the resolution text (terminal)

[1063] The generated solution text is sent from the server to the device and displayed on the user interface, allowing the user to receive specific solutions in real time. For example, the device may display a pop-up window telling the user, "The file may be too large. The maximum file size is 10MB."

[1064] In this way, the system can quickly resolve user problems and respond in a way that takes the user's feelings into consideration, thereby increasing user satisfaction and improving the quality of IT services.

[1065] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1066] Step 1: Collecting user operation logs (device)

[1067] The device monitors in real time the operations (clicks, keystrokes, error messages, etc.) performed by the user while using the service and records them as an operation log with a timestamp. It also collects operation characteristics such as operation speed, click strength, and mouse movement. Specifically, the device saves the timestamp of the click event, click strength, and mouse movement pattern in a log.

[1068] Input: User clicks, keystrokes, error messages

[1069] Data processing: Collection of operation logs and operation characteristics

[1070] Output: Operation log with timestamp

[1071] Step 2: Sending operation logs (terminal)

[1072] The terminal sends the collected operation logs to the server at regular intervals. This interval can be adjusted depending on the system settings, and is usually a few seconds to a few minutes. Specifically, the terminal processes the collected log data in batches and sends it using a secure communication protocol (e.g., HTTPS).

[1073] Input: Timestamp operation log

[1074] Data processing: batch processing of log data

[1075] Output: Operation log sent to the server

[1076] Step 3: Receiving and analyzing operation logs (server)

[1077] The server receives and analyzes the operation logs sent to it. It pays particular attention to error messages and consecutive operation patterns to identify the problems and emotions the user is facing. Specifically, it infers emotions such as impatience or confusion from the type and frequency of errors and the user's operation characteristics.

[1078] Input: Submitted operation log

[1079] Data processing: Log data analysis (error identification, emotion estimation)

[1080] Output: Identified issues and emotions

[1081] Step 4: Pull the appropriate support information (server)

[1082] The server retrieves appropriate support information from a database based on the identified problem and emotion. Support information includes FAQs, guidelines, troubleshooting procedures, etc., and extracts information based on the user's emotion. Specifically, the server executes a database query to search and retrieve the relevant support information.

[1083] Input: Identified issues and emotions

[1084] Data processing: information extraction from databases

[1085] Output: Appropriate supporting information

[1086] Step 5: Generate a solution (server)

[1087] The server generates a solution message using a generative AI model (e.g., GPT-4) based on the acquired support information. This message is adjusted to be easy for the user to understand, and may include a solution such as, "The file may be too large. The maximum file size is 10 MB." Specifically, the server sends a prompt to the generative AI model to generate the message.

[1088] Input: Appropriate supporting information

[1089] Data processing: Text generation using generative AI models

[1090] Output: Solution text

[1091] Step 6: Presentation of the solution (terminal)

[1092] The device displays the solution text received from the server on the user interface. The user can check this text and receive specific instructions for resolving the problem. Specifically, the device displays the solution text in a pop-up window or the like, and presents the solution to the user.

[1093] Input: Solution text received from the server

[1094] Data processing: Display of solution text

[1095] Output: Resolution text displayed in the user interface

[1096] (Application example 2)

[1097] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1098] When operating and maintaining factory robots, it is extremely important to quickly and accurately resolve problems that operators encounter. However, with current systems, when operators encounter a problem, it takes time to find a solution, often resulting in stress and confusion during the process. Furthermore, since solutions to problems are not standardized and depend on the skills and experience of each individual operator, it is difficult to provide consistent support. Given this background, there is a need for a system that not only identifies problems based on user operation logs, but also provides appropriate solutions taking into account the operator's emotions.

[1099] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1100] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user terminal, means for recognizing the user's emotions, and means for adjusting the solution based on the emotion information. This makes it possible to quickly identify the problem the operator is facing and provide an appropriate solution that takes emotions into consideration in real time.

[1101] A "user operation log" is data that records all operational events, such as inputs, clicks, and error messages, when an operator operates a factory robot.

[1102] A "server" is a computer system that receives and analyzes user operation logs.

[1103] "Analysis" refers to the process of examining the received operation logs in detail and identifying the problems the user is facing.

[1104] "Support information" is data that includes solutions and procedures for identified problems, and is information that helps the factory run smoothly.

[1105] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions to identified problems based on pre-trained data.

[1106] A "user terminal" is a device used to display the resolution text, and includes factory control panels, smartphones, smart glasses, etc.

[1107] "Means for recognizing emotions" refers to technology that analyzes data such as the operator's facial expressions and voice to identify their emotions at that time.

[1108] "Means for adjusting solutions based on emotional information" refers to a technique for changing the content and expression of the solution text depending on the recognized emotions.

[1109] This invention is a system that quickly resolves problems faced by operators during the operation and maintenance of factory robots. The system provides appropriate solutions in real time based on operation logs and user sentiment.

[1110] The user device collects the operator's operation log in real time. This operation log includes operation events such as clicks, inputs, and error messages. Operation characteristics such as operation speed, click strength, and duration are also collected. This makes it possible to capture changes in the operator's operating tendencies and emotions.

[1111] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information.

[1112] The server analyzes the received operation logs. The log data contains various events, and issues are identified by paying particular attention to error messages and consecutive operation patterns. The server also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion. This emotion recognition is achieved using a camera and voice recognition.

[1113] Once a problem is identified, the server pulls the appropriate support information from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account perceived emotional information, offering detailed instructions if the operator is confused or a quick solution if they are impatient.

[1114] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the operator. This solution message is tailored to make it easier for the operator to understand the problem. For example, a specific solution message such as "The sensor needs cleaning. Here are the steps:..." is generated.

[1115] The generated solution is sent to the user's device and displayed to the operator, who can then take action to quickly resolve the issue.

[1116] For example, if a factory robot encounters an error while picking a part, and the operator cannot find a solution, the system will display a solution such as "The sensor needs cleaning. Here are the steps:..."

[1117] An example of a prompt sentence is, "An error occurred during the part picking process. The sensor may not be working properly. Emotion recognition indicates that the operator is confused. Please provide a solution."

[1118] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1119] Step 1:

[1120] The terminal collects user operation logs in real time. The collected data includes operation events such as clicks, inputs, and error messages. It also collects operation characteristics such as operation speed, click strength, and mouse movement. The input is the user's operation events, and the output is the collected operation log data. This data collection utilizes the terminal's sensor data and interface monitoring software.

[1121] Step 2:

[1122] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information. The input is the collected operation log data, and the output is the operation log data transferred to the server. This data is sent using an HTTP POST request.

[1123] Step 3:

[1124] The operation log received by the server is analyzed. The log data contains various events, and problems are identified by paying particular attention to error messages and consecutive operation patterns. Furthermore, operation characteristic information is used to recognize user emotions. The input is the operation log data transferred to the server, and the output is the identified problems and emotion recognition results. A log analysis algorithm and an emotion recognition algorithm are used for the analysis.

[1125] Step 4:

[1126] The server retrieves appropriate support information for the identified problem from the database. Based on the identified problem and the user's emotion, relevant support information is obtained. The input is the identified problem and emotion recognition results, and the output is the support information. This data retrieval is performed using SQL queries.

[1127] Step 5:

[1128] A generative AI model is used to generate a solution message based on the support information retrieved by the server. The solution message is adjusted to make it easier for the operator to understand the problem. The input is the support information and emotion recognition results, and the output is the generated solution message. GPT-3 or a similar model is used as the generative AI model for generation.

[1129] Step 6:

[1130] The server sends the generated solution text to the terminal. The input is the generated solution text, and the output is the solution text transferred to the terminal. This data transmission uses WebSocket or HTTP POST request.

[1131] Step 7:

[1132] The device displays the solution text in the user interface. The user can then take action based on the presented solution. The input is the solution text transferred from the server, and the output is the solution text displayed to the user. This display is done using a pop-up window or notification system on the device.

[1133] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1134] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1135] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1136] [Fourth embodiment]

[1137] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1138] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1139] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1140] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1141] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1142] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1143] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1144] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1145] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1146] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1147] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1148] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1149] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1150] This invention relates to a system for solving problems that occur when users use IT services in real time. This system collects user operation logs, generates appropriate solutions based on those logs, and presents them to the users.

[1151] Collecting user operation logs (device)

[1152] The device has the ability to monitor user operations (clicks, inputs, error messages, etc.) in real time. Each operation event is logged with a timestamp. For example, if a user clicks the "Upload File" button but the upload fails, the operation and error message are recorded in the log.

[1153] Sending operation logs (terminal)

[1154] The collected operation logs are sent to the server at regular intervals, so that the server always has the latest operation information and can respond quickly if a problem occurs.

[1155] Receiving and analyzing operation logs (server)

[1156] The server has the ability to analyze the received operation logs. This analysis detects error messages and consecutive operation patterns, among other things, to identify problems the user may be facing. For example, if a file upload fails, the error message can be analyzed to identify the cause of the problem (e.g., file size exceeded).

[1157] Extracting appropriate support information (server)

[1158] The server has the ability to retrieve appropriate support information from the database for the identified problem, including FAQs, guidelines, troubleshooting procedures, etc. For example, in response to a file size exceeding error message, it retrieves "information about the maximum allowed file size."

[1159] Solution text generation (server)

[1160] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is tailored to help the user understand the problem. Specifically, a solution such as "The file may be too large. The maximum file size is 10MB" is generated.

[1161] Presentation of the resolution text (terminal)

[1162] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[1163] Specific examples

[1164] Example 1:

[1165] Here's what happens if a user tries to upload a file but fails:

[1166] 1. The user clicks the "upload button."

[1167] 2. The device will log this operation and also log a subsequent "upload failed" error.

[1168] 3. Logs are sent to the server at regular intervals.

[1169] 4. The server receives the log, analyzes it, and determines that the cause of the "upload failure" is excessive file size.

[1170] 5. The server retrieves the support information about the "maximum file size" from the database.

[1171] 6. Based on the information obtained, the generative AI model generates a solution message: "The file may be too large. The maximum file size is 10MB."

[1172] 7. The device displays the generated solution to the user and provides specific instructions for resolving the problem.

[1173] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process, resulting in faster problem resolution and a better user experience.

[1174] The processing flow will be explained below.

[1175] Step 1:

[1176] A user performs a specific action on the system, for example clicking a "File Upload" button.

[1177] Step 2:

[1178] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, etc.).

[1179] Step 3:

[1180] The terminal sends the collected log data to the server at regular intervals, which can be adjusted according to the system settings.

[1181] Step 4:

[1182] Analyze the log data received by the server. The log data contains various events, and identify problems by paying particular attention to error messages and consecutive operation patterns.

[1183] Step 5:

[1184] The server retrieves the appropriate support information for the identified problem from its database, which may include FAQs, guidelines, troubleshooting procedures, etc.

[1185] Step 6:

[1186] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model then adjusts the message appropriately to make it easier for the user to understand the problem.

[1187] Step 7:

[1188] The server generates a solution message and sends it to the device, which includes specific solutions and operational procedures.

[1189] Step 8:

[1190] The device will display the solution text in the user interface, allowing the user to take appropriate action to resolve the issue.

[1191] Specific examples

[1192] Example: File upload failure

[1193] Step 1:

[1194] The user clicks the "File Upload" button.

[1195] Step 2:

[1196] The device will log the click and then log the upload failure error message.

[1197] Step 3:

[1198] The terminal sends the collected log data to the server every 60 seconds.

[1199] Step 4:

[1200] Analyze the logs received by the server and identify the "upload failed" error.

[1201] Step 5:

[1202] The server retrieves support information about "file size exceeded" from the database.

[1203] Step 6:

[1204] The server uses the generative AI model to generate a solution message: "The file may be too large. The maximum file size is 10MB."

[1205] Step 7:

[1206] The server sends the generated solution text to the terminal.

[1207] Step 8:

[1208] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[1209] Example 1

[1210] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1211] Traditional IT services lacked the means to instantly resolve problems users faced. As a result, users took time to identify the cause of the problem and had to use multiple support channels to find the appropriate solution. In particular, when an error message occurred, the process of understanding the message and finding a solution was cumbersome, resulting in a poor user experience. To solve these problems, a system that automatically identifies problems based on user operation logs and provides appropriate support information is needed.

[1212] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1213] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for providing a prompt message to the generative AI model based on the cause of the problem, and means for displaying the generated solution message on a user interface. This enables the user to quickly identify the cause of the problem and immediately obtain appropriate support information and a solution.

[1214] A "user operation log" is an operation history that includes clicks, inputs, error messages, etc. that is generated when a user uses a system or application.

[1215] "Means of collecting data in real time" refers to the function of instantly capturing the operations performed by the user and recording that information as a log.

[1216] "Means for sending to a server" refers to the function of transferring collected operation logs to a server via a network such as the Internet periodically or based on certain conditions.

[1217] "Means for analyzing received operation logs" refers to the function of processing the data in the operation logs received by the server to detect error messages and operation patterns contained in the logs and identify the cause of the problem.

[1218] "Means for identifying problems faced by users" refers to the function of identifying errors or problems that users are currently facing from the analyzed operation logs.

[1219] "Means for retrieving appropriate support information from the database" refers to the function of searching and retrieving support information such as related solutions, FAQs, guidelines, etc. in the database according to the identified problem.

[1220] "Means for generating a solution message using a generative AI model" refers to the function of utilizing a generative AI model to create a solution message that is easy for users to understand based on the acquired support information and the cause of the problem.

[1221] "Means for providing prompt sentences" refers to the function of supplying sentences that provide the generative AI model with background information and instructions for generating an appropriate solution sentence.

[1222] "Means for displaying on the user interface" refers to the function of displaying the generated solution text on the screen of the user's terminal and providing it visually.

[1223] MODE FOR CARRYING OUT THE INVENTION

[1224] This invention relates to a system for solving problems that occur when a user uses an IT service in real time. This section describes the hardware and software required to configure this system, as well as its operating method.

[1225] Collecting user operation logs (device)

[1226] The device monitors user actions in real time. Events such as buttons clicked by the user, text entered, and error messages are logged with a timestamp. This typically involves sensors and monitoring software (e.g., keyloggers, click-monitoring tools).

[1227] For example, if a user clicks the "Upload button", if this operation is successful, a success event will be logged. Conversely, if the operation fails and an error message such as "File size too large" is displayed, this error information will also be logged.

[1228] Sending operation logs (terminal)

[1229] The collected operation logs are sent to the server at regular intervals. This sending process is performed in the background so as not to interfere with user operations. HTTP / HTTPS is used as the protocol. Specifically, logs are sent to the server in batches at regular volume or time intervals (e.g., every 5 seconds).

[1230] Server receives and analyzes logs

[1231] The server analyzes the received operation logs. For analysis, a database server (e.g., MySQL, PostgreSQL) and data processing scripts (e.g., Python, R) are used. Specifically, the log data is stored in a database and error messages and specific operation patterns are detected.

[1232] For example, if the server receives a "file upload failed" error message, it analyzes the logs to determine that the cause is an excessive file size. The results of this analysis are passed on to the next step.

[1233] Extracting appropriate support information (server)

[1234] Based on the analysis results, the server retrieves the appropriate support information from a database that stores FAQs, guidelines, troubleshooting procedures, etc. SQL queries are used to retrieve the support information.

[1235] For example, for the error message "File size exceeded," you will get support information about "Maximum file size," which will give the user information to understand the specific cause of the problem.

[1236] Solution text generation (server)

[1237] The server generates a solution message using a generative AI model (e.g., OpenAI GPT-4) based on the acquired support information. The generative AI model receives the support information and the cause of the problem as input, and outputs a message that provides a solution to the user in an easy-to-understand format.

[1238] The generated solution text will be specific, for example, "The file may be too large. The maximum file size is 10MB."

[1239] Presentation of the resolution text (terminal)

[1240] The generated solution text is sent to the device and displayed in the user interface using UI elements such as a pop-up window or a notification bar, allowing the user to instantly receive a concrete solution instead of an error message.

[1241] Prompt Sentence Examples

[1242] The prompt provided to the generative AI model looks something like this:

[1243] If the upload failure is due to excessive file size, please provide a solution such as "The file may be too large. The maximum file size is 10MB."

[1244] Using such detailed prompts allows the AI ​​model to generate more accurate and useful solutions.

[1245] This system will enable users to quickly identify the cause of a problem and immediately obtain appropriate support information and solutions, which is expected to significantly improve the experience of using IT services.

[1246] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1247] The flow of this system's program processing

[1248] Step 1: Collect user operation logs

[1249] Users operate systems and applications.

[1250] The device monitors each user action (click, input, error message, etc.) in real time and creates a time-stamped operation log.

[1251] Specifically, when a user clicks the "Upload button," the click event is recorded, and if the upload fails, an error message stating "File size too large" is also recorded.

[1252] Input: User actions (clicks, inputs, error messages, etc.)

[1253] Output: Operation log with timestamp

[1254] Step 2: Send the operation log

[1255] The terminal sends the collected operation logs to the server at regular intervals.

[1256] This sending process is done in the background and does not interfere with user operations. For example, a batch of operation logs is sent to the server every 5 seconds.

[1257] Input: Timestamp operation log

[1258] Output: Batch log data to be sent

[1259] Step 3: Receiving and saving the operation log

[1260] The server receives the operation log sent from the terminal and stores it in a database.

[1261] Specifically, the server opens the received log data and inserts each log entry into the appropriate table in the database.

[1262] Input: Batch log data to be sent

[1263] Output: Operation log saved in the database

[1264] Step 4: Analyze the operation log

[1265] The server analyzes the operation logs stored in the database.

[1266] In particular, it processes data to identify the cause of a problem by detecting error messages and operation patterns. For example, it searches for "upload failed" events in the log and identifies that the cause is "file size exceeded."

[1267] Input: Operation logs stored in the database

[1268] Output: Identified cause of the problem (e.g. file size exceeded)

[1269] Step 5: Pulling Support Information

[1270] The server retrieves appropriate support information from a database based on the identified problem.

[1271] Specifically, it runs an SQL query to get information about "Maximum File Size".

[1272] Input: Identified cause of the problem

[1273] Output: Retrieved support information (e.g. information about maximum file size)

[1274] Step 6: Generate a solution

[1275] The server generates a solution text using a generative AI model (e.g., OpenAI GPT-4) based on the obtained supporting information and analysis results.

[1276] The generative AI model generates a prompt and then generates a solution for the user, such as "The file may be too large. The maximum file size is 10MB."

[1277] Input: Retrieved support information, identified problem cause, prompt statement

[1278] Output: Generated solution text

[1279] Step 7: Present the solution

[1280] The server sends the generated solution text to the terminal.

[1281] The device will display the received solution text on the user interface, and the user can immediately obtain a specific solution, for example, as a notification bar or a pop-up window.

[1282] Input: Generated solution text

[1283] Output: Resolution text displayed in the user interface

[1284] (Application example 1)

[1285] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1286] When users encounter errors or problems during the payment process with electronic payment services, they must search for solutions themselves, making it difficult to resolve them quickly. This can lead to a poor user experience and a loss of reliability in the service. Furthermore, there is often no system in place to provide real-time solutions, often leaving users frustrated. This invention aims to resolve these issues and make the payment process smoother and more efficient for users.

[1287] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1288] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying a problem the user is facing, means for retrieving appropriate support information for the identified problem from a data storage device, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user device, and means for analyzing details of an error that occurred in a specific payment process based on the user's operation and quickly providing the solution message, thereby enabling users to quickly resolve problems they encounter in the electronic transaction process and improving their service usage experience.

[1289] The "user operation log" refers to data that records a series of operation events (such as clicks, inputs, error messages, etc.) performed by a user on an electronic device or system.

[1290] The "means for collecting in real time" refers to a mechanism that instantaneously detects a user's operation and immediately records the operation information as data.

[1291] The "server" refers to a computer system that receives the operation logs sent by a user and performs data analysis and presentation of problem-solving solutions.

[1292] The "means for analyzing and identifying the problems faced by the user" refers to a mechanism that analyzes error messages and operation patterns from the received operation logs and detects the problems that the user is currently facing.

[1293] The "support information" refers to information that helps solve the problems encountered by the user and includes FAQs, guidelines, troubleshooting procedures, etc.

[1294] The "data storage device" refers to a device in a computer system for long-term storage of support information and other important data.

[1295] The "generative AI model" refers to an algorithm or program that uses artificial intelligence to automatically generate solutions and instructions in natural language based on input data.

[1296] The "solution text" refers to a text that describes specific solutions and instructions in an easy-to-understand format for the problems faced by the user.

[1297] The "user device" refers to a device for a user to perform operations, such as a smartphone, computer, head-mounted display, etc.

[1298] The "means for analyzing the details of an error" is a mechanism for deeply analyzing the error message contained in the operation log and clarifying the cause and details of the error.

[1299] "Means for rapid provision" refers to a mechanism for presenting analysis results and generated solution text to users without delay.

[1300] This invention is a system that collects user operation logs in real time, generates appropriate solutions based on those logs, and presents them to the user. Specifically, it aims to link the server and user terminals to quickly resolve problems users encounter during the electronic payment process.

[1301] Program processing explanation

[1302] Collecting user operation logs (device)

[1303] The user device monitors user actions (clicks, inputs, error messages, etc.) in real time and records them in a log, which is time-stamped and records each operation event in detail.

[1304] Sending operation logs (terminal)

[1305] The collected operation logs are temporarily stored on the user's device and then sent to the server at regular intervals. This process ensures that the server always has the latest operation information.

[1306] Receiving and analyzing operation logs (server)

[1307] The server receives operation logs sent from user terminals. The received logs are analyzed, and error messages and consecutive operation patterns are automatically detected. A log analysis algorithm is used for this analysis.

[1308] Extracting appropriate support information (server)

[1309] For problems identified as a result of the analysis, the server retrieves appropriate support information from its data store, including FAQs, guidelines, troubleshooting procedures, and the like.

[1310] Solution text generation (server)

[1311] Based on the extracted support information, the server generates a solution using a generative AI model. This generative AI model uses a natural language processing algorithm to automatically generate a solution based on the input data. For example, it can use OpenAI's API.

[1312] Presentation of the resolution text (terminal)

[1313] The generated solution text is sent from the server to the user's device and displayed on the user interface, allowing the user to receive specific instructions on how to solve the problem.

[1314] Specific examples

[1315] 1. The user enters their card information on the payment page and clicks the "Payment button."

[1316] 2. The user terminal will log this operation and immediately afterwards also log a "Payment failed" error message.

[1317] 3. Logs are sent to the server at regular intervals.

[1318] 4. The server receives the log, analyzes it, and determines that the cause of the "payment failure" is incorrect card information.

[1319] 5. The server uses a generative AI model (OpenAI API) to generate a solution message that reads, "Your card information may be incorrect. Please re-enter accurate information."

[1320] 6. The user device displays the generated solution text to the user and provides specific instructions for solving the problem.

[1321] Prompt Sentence Examples

[1322] Please provide a solution: Payment failed: Incorrect card details

[1323] This system allows users to quickly resolve any issues they encounter during the payment process, improving their service experience.

[1324] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1325] Step 1:

[1326] A user enters card information on the page of an electronic payment service and clicks the "Payment button." This operation is performed on the user's device. This input generates a user operation log, which includes a timestamp of the card information input.

[1327] Step 2:

[1328] The user device collects user operation logs (clicks, inputs, error messages, etc.) in real time. For example, clicking the payment button and error messages when a payment fails are recorded as logs. This log is time-stamped and accumulates the collected operation events.

[1329] Step 3:

[1330] The collected operation logs are temporarily stored on the user's device and sent to the server at regular intervals. The data entered here is the operation log data, which is then sent to the server. The server obtains the latest operation information in real time, so the system is designed to avoid delays in sending the logs.

[1331] Step 4:

[1332] The server receives operation logs sent from user terminals. It analyzes the received operation logs and automatically detects error messages and consecutive operation patterns in particular. The input here is the operation log sent to the server, and the output is the analysis results, including error messages. A log analysis algorithm is used for this analysis.

[1333] Step 5:

[1334] As a result of the analysis, error messages and specific operation patterns are detected to identify the problem the user is facing. Based on the analysis results, the server retrieves appropriate support information from a data storage device. Here, the input is the analysis results, and the output is the corresponding support information (FAQs, guidelines, troubleshooting procedures, etc.).

[1335] Step 6:

[1336] Based on the extracted support information, the server uses a generative AI model to generate a solution text. The generative AI model (e.g., OpenAI API) creates a solution text in natural language based on the input support information and analysis results. The input is support information and analysis results, and the output is the solution text.

[1337] Step 7:

[1338] The generated solution text is sent from the server to the user terminal. The input here is the solution text, and the output is the text displayed on the user terminal. The user terminal displays this solution text on the user interface and presents a specific solution to the user.

[1339] Prompt Sentence Examples

[1340] Please provide a solution: Payment failed: Incorrect card details

[1341] The above are the specific processing steps of the system program that realizes the application example. Through this series of processes, users can receive solutions to problems in real time and quickly resolve their issues.

[1342] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1343] This invention combines a system that solves problems that arise when users use IT services in real time with an emotion engine that recognizes the user's emotions. This system not only collects user operation logs and generates and presents appropriate solutions based on those logs, but also adjusts the solution text taking the user's emotions into account.

[1344] Collecting user operation logs (device)

[1345] The device monitors user operations (clicks, inputs, error messages, etc.) in real time and records each operation event in a log with a timestamp. The log also includes operation characteristics such as operation speed, click strength, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[1346] Sending operation logs (terminal)

[1347] The collected operation logs are sent to the server at regular intervals. This interval can be adjusted according to the system settings. The sent data includes basic operation information and operation characteristic information.

[1348] Receiving and analyzing operation logs (server)

[1349] The server analyzes the received operation logs. The log data contains various events, and it identifies problems by paying particular attention to error messages and consecutive operation patterns. It also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion.

[1350] Extracting appropriate support information (server)

[1351] The server retrieves appropriate support information for the identified problem from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account emotional information, providing detailed instructions if the user is confused or quick solutions if the user is impatient.

[1352] Solution text generation (server)

[1353] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the user. This message is adjusted to make it easier for the user to understand the problem. Specifically, the generated solution might say, "The file may be too large. The maximum file size is 10MB." The way the solution is expressed and the level of detail are adjusted depending on the user's feelings.

[1354] Presentation of the resolution text (terminal)

[1355] The generated solution text is sent to the device and displayed in the user interface, allowing the user to instantly receive a concrete solution instead of an error message, and to take action to quickly resolve the problem.

[1356] Specific examples

[1357] Example 1: File upload failure

[1358] 1. The user clicks the "File Upload" button.

[1359] 2. The device will log this operation and then log an error message about the failed upload. If the user has tried multiple times, this information will also be included in the log.

[1360] 3. Logs are sent to the server at regular intervals.

[1361] 4. The server receives the log, identifies the "upload failed" error, and realizes that the user is anxious.

[1362] 5. The server retrieves the "information about the maximum file size" from the database and generates a quick solution that takes into account the user's impatience.

[1363] 6. The server uses the generative AI model to generate a solution: "The file may be too large. Maximum file size is 10MB. Please fix it immediately."

[1364] 7. The server sends the generated solution text to the terminal.

[1365] 8. The device will display the solution text in a pop-up window and provide the user with a specific solution.

[1366] In this way, the present invention allows users to receive appropriate support on the spot without interrupting the process. Furthermore, by taking the user's emotions into consideration, the user's experience is further improved.

[1367] The processing flow will be explained below.

[1368] Step 1:

[1369] A user performs a specific action on the system, for example clicking a "File Upload" button.

[1370] Step 2:

[1371] The device will record this operation event in a real-time log, which will include the event type, timestamp, and related details (e.g., button ID, error message, operation speed, click strength, mouse movement).

[1372] Step 3:

[1373] The terminal sends the collected log data and operation characteristic information to the server at regular intervals (for example, every 60 seconds). This allows the server to always have the latest operation information and respond immediately.

[1374] Step 4:

[1375] The server analyzes the received log data. This analysis includes detecting error messages and consecutive operation patterns, and recognizing the user's emotions using operation characteristics information. For example, rapid consecutive clicks may indicate impatience, while slow operation speed may indicate confusion.

[1376] Step 5:

[1377] The server retrieves the appropriate support information for the identified issue from a database, including FAQs, guidelines, troubleshooting procedures, etc. Additionally, the retrieved information is tailored to the user's perceived sentiment.

[1378] Step 6:

[1379] Based on the support information acquired by the server, a generative AI model is used to generate a solution message to be presented to the user. The generative AI model adjusts the expression and level of detail of the message depending on the user's emotions. For example, it might generate a message such as "The file may be too large. The maximum file size is 10MB. Please fix it immediately" for a user who is expressing impatience.

[1380] Step 7:

[1381] The server sends the generated solution text to the device, which includes specific solutions and operational procedures.

[1382] Step 8:

[1383] The device will display the solution text in the user interface, allowing users to instantly receive a specific solution instead of an error message, and users can take action based on this information to quickly resolve the issue.

[1384] Specific examples

[1385] Example: File upload failure

[1386] Step 1:

[1387] The user clicks the "File Upload" button.

[1388] Step 2:

[1389] The device logs the click, then logs the upload failure error message, and if the user retries multiple times, logs information such as the speed of the click and the strength of the click.

[1390] Step 3:

[1391] The terminal sends the collected log data and operational characteristics information to the server every 60 seconds.

[1392] Step 4:

[1393] The server receives the log, identifies the "upload failed" error, and recognizes from the operational characteristics information contained in the log that the user is impatient.

[1394] Step 5:

[1395] The server retrieves the "information about the maximum file size" from the database, and, considering the user's impatience, prioritizes a quick solution.

[1396] Step 6:

[1397] The server uses a generative AI model to generate a message saying, "The file may be too large. The maximum file size is 10MB. Please fix it immediately." Recognizing the user's impatience, the server then concisely emphasizes the solution.

[1398] Step 7:

[1399] The server sends the generated solution text to the terminal.

[1400] Step 8:

[1401] The device will display the solution text in a pop-up window, providing the user with a specific solution.

[1402] The system allows users to receive immediate solutions when problems arise, as well as emotionally sensitive support.

[1403] Example 2

[1404] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1405] In modern IT services, it is extremely important to quickly and appropriately resolve problems users encounter during operation. However, while current systems can collect and analyze user operation logs to identify problems, it is difficult to provide support that takes the user's emotions into consideration. This can result in a failure to increase user satisfaction and a decline in service quality. The objective of this invention is to solve these problems and provide a system that supports users in resolving problems more quickly and accurately.

[1406] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1407] In this invention, the server includes means for analyzing the received operation log and identifying the problem and emotion the user is facing, means for retrieving appropriate support information for the identified problem and emotion from a database, and means for generating a solution message using a generative AI model based on the retrieved support information. This makes it possible to analyze the user's operation log and emotion in real time and provide appropriate support information, thereby increasing user satisfaction.

[1408] An "operation log" is data that records the operation history, including clicks, keystrokes, and error messages, when a user uses an IT service, along with timestamps.

[1409] "Operation characteristics" are data that capture detailed characteristics of operations, such as the speed of a user's operation, the strength of the click, and the movement of the mouse.

[1410] The "analysis means" is a function for analyzing the received operation log and identifying the problem the user is facing and the user's feelings from error messages and consecutive operation patterns.

[1411] "Support Information" means materials such as FAQs, guidelines, troubleshooting procedures, and the like that are provided to help users solve problems they may encounter.

[1412] A "database" is an information accumulation device in which support information is stored and a system for searching and acquiring the contents thereof.

[1413] A "generative AI model" is an artificial intelligence model used to automatically generate a solution text to be presented to the user based on the acquired support information.

[1414] A "solution text" is a specific guide text for solving a problem that is generated by a generative AI model and presented to the user.

[1415] "Emotion recognition" is the process of identifying a user's emotions, such as impatience or confusion, from operational characteristic information.

[1416] The present invention is a system that solves problems that arise when a user uses an IT service in real time. This system incorporates an emotion engine that recognizes the user's emotions. A specific embodiment of this system will be described below.

[1417] Hardware and software used

[1418] This system is primarily composed of a terminal and a server. The terminal collects user operation logs in real time and sends them to the server at regular intervals. The server, in turn, analyzes the received operation logs, identifies the user's problems and feelings, and provides appropriate support information.

[1419] Terminal: A device that monitors and records user operation logs (clicks, keystrokes, error messages, etc.) in real time. Logs are also collected, including operation characteristics (speed, click strength, mouse movement, etc.).

[1420] Server: A system that receives and analyzes operation logs sent from the device. It retrieves support information from a database and generates a solution using a generative AI model.

[1421] Processing flow and actual operation

[1422] 1. Collecting user operation logs (device)

[1423] When a user uses an IT service, for example by clicking a "file upload" button, the device records this click event in a log with a timestamp. It also records operation characteristics such as click strength, speed, and mouse movement. This makes it possible to capture changes in the user's operation trends and emotions.

[1424] 2. Sending operation logs (terminal)

[1425] The terminal sends the collected operation logs to the server at regular intervals (a few seconds to a few minutes). A secure and high-speed communication protocol (e.g., HTTPS) is used for this transmission.

[1426] 3. Receiving and analyzing operation logs (server)

[1427] The server analyzes the received operation logs. For the analysis, it uses log data containing operation characteristics. For example, if the number of operations increases suddenly, it can be determined that the user is in a hurry, and if the speed of operations slows down, it can be determined that the user is confused.

[1428] 4. Extracting appropriate support information (server)

[1429] The server retrieves appropriate support information for the identified problem and emotion from a database that includes FAQs, guidelines, troubleshooting procedures, etc. Taking into account the user's emotional information, the server provides detailed instructions for confused users and quick solutions for impatient users.

[1430] 5. Generation of solution text (server)

[1431] Based on the obtained support information, the server uses a generative AI model (e.g., GPT-4) to generate a solution message. This message is tailored to help users understand the problem. For example, a solution message might be generated: "The file may be too large. The maximum file size is 10 MB."

[1432] Example prompt sentence:

[1433] "A user attempted to upload a file but received an error message. Please provide information about the maximum file size to generate a quick solution."

[1434] 6. Presentation of the resolution text (terminal)

[1435] The generated solution text is sent from the server to the device and displayed on the user interface, allowing the user to receive specific solutions in real time. For example, the device may display a pop-up window telling the user, "The file may be too large. The maximum file size is 10MB."

[1436] In this way, the system can quickly resolve user problems and respond in a way that takes the user's feelings into consideration, thereby increasing user satisfaction and improving the quality of IT services.

[1437] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1438] Step 1: Collecting user operation logs (device)

[1439] The device monitors in real time the operations (clicks, keystrokes, error messages, etc.) performed by the user while using the service and records them as an operation log with a timestamp. It also collects operation characteristics such as operation speed, click strength, and mouse movement. Specifically, the device saves the timestamp of the click event, click strength, and mouse movement pattern in a log.

[1440] Input: User clicks, keystrokes, error messages

[1441] Data processing: Collection of operation logs and operation characteristics

[1442] Output: Operation log with timestamp

[1443] Step 2: Sending operation logs (terminal)

[1444] The terminal sends the collected operation logs to the server at regular intervals. This interval can be adjusted depending on the system settings, and is usually a few seconds to a few minutes. Specifically, the terminal processes the collected log data in batches and sends it using a secure communication protocol (e.g., HTTPS).

[1445] Input: Timestamp operation log

[1446] Data processing: batch processing of log data

[1447] Output: Operation log sent to the server

[1448] Step 3: Receiving and analyzing operation logs (server)

[1449] The server receives and analyzes the operation logs sent to it. It pays particular attention to error messages and consecutive operation patterns to identify the problems and emotions the user is facing. Specifically, it infers emotions such as impatience or confusion from the type and frequency of errors and the user's operation characteristics.

[1450] Input: Submitted operation log

[1451] Data processing: Log data analysis (error identification, emotion estimation)

[1452] Output: Identified issues and emotions

[1453] Step 4: Pull the appropriate support information (server)

[1454] The server retrieves appropriate support information from a database based on the identified problem and emotion. Support information includes FAQs, guidelines, troubleshooting procedures, etc., and extracts information based on the user's emotion. Specifically, the server executes a database query to search and retrieve the relevant support information.

[1455] Input: Identified issues and emotions

[1456] Data processing: information extraction from databases

[1457] Output: Appropriate supporting information

[1458] Step 5: Generate a solution (server)

[1459] The server generates a solution message using a generative AI model (e.g., GPT-4) based on the acquired support information. This message is adjusted to be easy for the user to understand, and may include a solution such as, "The file may be too large. The maximum file size is 10 MB." Specifically, the server sends a prompt to the generative AI model to generate the message.

[1460] Input: Appropriate supporting information

[1461] Data processing: Text generation using generative AI models

[1462] Output: Solution text

[1463] Step 6: Presentation of the solution (terminal)

[1464] The device displays the solution text received from the server on the user interface. The user can check this text and receive specific instructions for resolving the problem. Specifically, the device displays the solution text in a pop-up window or the like, and presents the solution to the user.

[1465] Input: Solution text received from the server

[1466] Data processing: Display of solution text

[1467] Output: Resolution text displayed in the user interface

[1468] (Application example 2)

[1469] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1470] When operating and maintaining factory robots, it is extremely important to quickly and accurately resolve problems that operators encounter. However, with current systems, when operators encounter a problem, it takes time to find a solution, often resulting in stress and confusion during the process. Furthermore, since solutions to problems are not standardized and depend on the skills and experience of each individual operator, it is difficult to provide consistent support. Given this background, there is a need for a system that not only identifies problems based on user operation logs, but also provides appropriate solutions taking into account the operator's emotions.

[1471] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1472] In this invention, the server includes means for collecting user operation logs in real time, means for transmitting the collected operation logs to the server, means for analyzing the received operation logs and identifying the problem the user is facing, means for retrieving appropriate support information for the identified problem from a database, means for generating a solution message using a generative AI model based on the retrieved support information, means for displaying the generated solution message on the user terminal, means for recognizing the user's emotions, and means for adjusting the solution based on the emotion information. This makes it possible to quickly identify the problem the operator is facing and provide an appropriate solution that takes emotions into consideration in real time.

[1473] A "user operation log" is data that records all operational events, such as inputs, clicks, and error messages, when an operator operates a factory robot.

[1474] A "server" is a computer system that receives and analyzes user operation logs.

[1475] "Analysis" refers to the process of examining the received operation logs in detail and identifying the problems the user is facing.

[1476] "Support information" is data that includes solutions and procedures for identified problems, and is information that helps the factory run smoothly.

[1477] A "generative AI model" is an artificial intelligence algorithm that generates optimal solutions to identified problems based on pre-trained data.

[1478] A "user terminal" is a device used to display the resolution text, and includes factory control panels, smartphones, smart glasses, etc.

[1479] "Means for recognizing emotions" refers to technology that analyzes data such as the operator's facial expressions and voice to identify their emotions at that time.

[1480] "Means for adjusting solutions based on emotional information" refers to a technique for changing the content and expression of the solution text depending on the recognized emotions.

[1481] This invention is a system that quickly resolves problems faced by operators during the operation and maintenance of factory robots. The system provides appropriate solutions in real time based on operation logs and user sentiment.

[1482] The user device collects the operator's operation log in real time. This operation log includes operation events such as clicks, inputs, and error messages. Operation characteristics such as operation speed, click strength, and duration are also collected. This makes it possible to capture changes in the operator's operating tendencies and emotions.

[1483] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information.

[1484] The server analyzes the received operation logs. The log data contains various events, and issues are identified by paying particular attention to error messages and consecutive operation patterns. The server also uses operation characteristic information to recognize the user's emotions. For example, a sudden increase in operations may indicate impatience, while slow operation speed may indicate confusion. This emotion recognition is achieved using a camera and voice recognition.

[1485] Once a problem is identified, the server pulls the appropriate support information from its database, including FAQs, guidelines, troubleshooting steps, etc. It also takes into account perceived emotional information, offering detailed instructions if the operator is confused or a quick solution if they are impatient.

[1486] Based on the acquired support information, the server uses a generative AI model to generate a solution message to present to the operator. This solution message is tailored to make it easier for the operator to understand the problem. For example, a specific solution message such as "The sensor needs cleaning. Here are the steps:..." is generated.

[1487] The generated solution is sent to the user's device and displayed to the operator, who can then take action to quickly resolve the issue.

[1488] For example, if a factory robot encounters an error while picking a part, and the operator cannot find a solution, the system will display a solution such as "The sensor needs cleaning. Here are the steps:..."

[1489] An example of a prompt sentence is, "An error occurred during the part picking process. The sensor may not be working properly. Emotion recognition indicates that the operator is confused. Please provide a solution."

[1490] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1491] Step 1:

[1492] The terminal collects user operation logs in real time. The collected data includes operation events such as clicks, inputs, and error messages. It also collects operation characteristics such as operation speed, click strength, and mouse movement. The input is the user's operation events, and the output is the collected operation log data. This data collection utilizes the terminal's sensor data and interface monitoring software.

[1493] Step 2:

[1494] The collected operation logs are sent to the server at regular intervals. The sent data includes basic operation information and operation characteristic information. The input is the collected operation log data, and the output is the operation log data transferred to the server. This data is sent using an HTTP POST request.

[1495] Step 3:

[1496] The operation log received by the server is analyzed. The log data contains various events, and problems are identified by paying particular attention to error messages and consecutive operation patterns. Furthermore, operation characteristic information is used to recognize user emotions. The input is the operation log data transferred to the server, and the output is the identified problems and emotion recognition results. A log analysis algorithm and an emotion recognition algorithm are used for the analysis.

[1497] Step 4:

[1498] The server retrieves appropriate support information for the identified problem from the database. Based on the identified problem and the user's emotion, relevant support information is obtained. The input is the identified problem and emotion recognition results, and the output is the support information. This data retrieval is performed using SQL queries.

[1499] Step 5:

[1500] A generative AI model is used to generate a solution message based on the support information retrieved by the server. The solution message is adjusted to make it easier for the operator to understand the problem. The input is the support information and emotion recognition results, and the output is the generated solution message. GPT-3 or a similar model is used as the generative AI model for generation.

[1501] Step 6:

[1502] The server sends the generated solution text to the terminal. The input is the generated solution text, and the output is the solution text transferred to the terminal. This data transmission uses WebSocket or HTTP POST request.

[1503] Step 7:

[1504] The device displays the solution text in the user interface. The user can then take action based on the presented solution. The input is the solution text transferred from the server, and the output is the solution text displayed to the user. This display is done using a pop-up window or notification system on the device.

[1505] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1506] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1507] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1508] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1509] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1510] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1511] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1512] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1513] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1514] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1515] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1516] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1517] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1518] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1519] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1520] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1521] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1522] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1523] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1524] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1525] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1526] The following is further disclosed regarding the above embodiment.

[1527] (Claim 1)

[1528] A means of collecting user operation logs in real time,

[1529] A means for transmitting the collected operation logs to a server;

[1530] A means of analyzing the received operation logs and identifying the problems the user is facing;

[1531] a means of retrieving appropriate supporting information from the database for the identified problem;

[1532] A means for generating a solution statement using a generative AI model based on the extracted support information;

[1533] The system includes means for displaying the generated solution text on a user terminal.

[1534] (Claim 2)

[1535] 2. The system according to claim 1, wherein the analysis means identifies an operation log containing an error message and analyzes the details of the error.

[1536] (Claim 3)

[1537] 2. The system according to claim 1, wherein the solution text generation means adjusts the solution so that the user can easily understand the problem.

[1538] "Example 1"

[1539] (Claim 1)

[1540] A means of collecting user operation logs in real time,

[1541] A means for transmitting the collected operation logs to a server;

[1542] A means of analyzing the received operation logs and identifying the problems the user is facing;

[1543] a means of retrieving appropriate supporting information from the database for the identified problem;

[1544] A means for generating a solution statement using a generative AI model based on the extracted support information;

[1545] a means for displaying the generated solution text on a user terminal;

[1546] a means for providing a prompt to the generative AI model based on the cause of the problem;

[1547] The system includes means for displaying the generated solution text in a user interface.

[1548] (Claim 2)

[1549] 2. The system according to claim 1, wherein the analysis means identifies an operation log containing an error message and analyzes the details of the error.

[1550] (Claim 3)

[1551] 2. The system according to claim 1, wherein the solution text generation means adjusts the solution so that the user can easily understand the problem.

[1552] "Application Example 1"

[1553] (Claim 1)

[1554] A means of collecting user operation logs in real time,

[1555] A means for transmitting the collected operation logs to a server;

[1556] A means of analyzing the received operation logs and identifying the problems the user is facing;

[1557] means for retrieving appropriate support information for the identified problem from a data store;

[1558] A means for generating a solution statement using a generative AI model based on the extracted support information;

[1559] means for displaying the generated solution statement on a user device;

[1560] A system that includes a means for analyzing the details of errors that occur within a specific payment process based on user operations and quickly providing a resolution document.

[1561] (Claim 2)

[1562] 2. The system according to claim 1, wherein the analysis means identifies an operation log containing an error message and analyzes the error message.

[1563] (Claim 3)

[1564] 2. The system of claim 1, wherein the solution generation means adjusts the solution to make it easier for a user to understand the problem within the electronic trading process.

[1565] "Example 2: Combining Emotion Engines"

[1566] (Claim 1)

[1567] A means of collecting user operation logs in real time,

[1568] means for transmitting an operation log including the collected operation characteristics to a server;

[1569] A means for analyzing the received operation logs to identify the problems and emotions faced by users;

[1570] a means of retrieving appropriate support information from a database for the identified problems and feelings;

[1571] A means for generating a solution statement using a generative AI model based on the extracted support information;

[1572] The system includes means for displaying the generated solution text on a user terminal.

[1573] (Claim 2)

[1574] 2. The system according to claim 1, wherein the analysis means identifies an operation log containing an error message and analyzes the details of the error and the user's feelings.

[1575] (Claim 3)

[1576] 2. The system according to claim 1, wherein the solution text generation means adjusts the solution according to the user's feelings and generates text that makes the problem easy to understand.

[1577] "Application example 2 when combining emotion engines"

[1578] (Claim 1)

[1579] A means of collecting user operation logs in real time,

[1580] A means for transmitting the collected operation logs to a server;

[1581] A means of analyzing the received operation logs and identifying the problems the user is facing;

[1582] a means of retrieving appropriate supporting information from the database for the identified problem;

[1583] A means for generating a solution statement using a generative AI model based on the extracted support information;

[1584] a means for displaying the generated solution text on a user terminal;

[1585] a means of recognizing a user's emotions;

[1586] and a means for adjusting the solution based on the emotional information.

[1587] (Claim 2)

[1588] 2. The system according to claim 1, wherein the analysis means identifies an operation log containing an error message and analyzes the details of the error.

[1589] (Claim 3)

[1590] 2. The system according to claim 1, wherein the solution text generation means adjusts the solution so that the user can easily understand the problem. [Explanation of symbols]

[1591] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting user operation logs in real time, A means for transmitting the collected operation logs to a server; A means of analyzing the received operation logs and identifying the problems the user is facing; a means of retrieving appropriate supporting information from the database for the identified problem; A means for generating a solution statement using a generative AI model based on the extracted support information; The system includes means for displaying the generated solution text on a user terminal.

2. 2. The system according to claim 1, wherein the analysis means identifies an operation log containing an error message and analyzes the details of the error.

3. 2. The system according to claim 1, wherein the solution text generation means adjusts the solution so that the user can easily understand the problem.

Images