system

Abstract

Provide a system. 【Solution means】 Means for receiving inquiry information, Means for analyzing the received inquiry to identify problems, Means for determining automatic countermeasures for problems, Means for executing the determined countermeasures, Means for monitoring the system state after execution, Means for escalating to human operation personnel as necessary, Means for visualizing the operation status of the system in real time through physical sensors, Means for providing the visualized information to a display device, A system including the above.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In modern enterprises, system operation for 24 hours a day, 365 days a year is required, and it is becoming difficult to secure personnel. In particular, inquiry response and trouble response in system operation require a great deal of labor and time, and despite the large number of routine operations, they rely on human resources. For this reason, there is a problem that it is difficult to solve problems efficiently and quickly in a situation of manpower shortage.

Means for Solving the Problems

[0005] This invention utilizes AI-powered system automation technology to automate a series of processes, including receiving inquiry information, analyzing its content and identifying problems, determining and executing automated countermeasures, monitoring system status, and escalating issues to humans as needed. This streamlines routine operational tasks and enables 24 / 7 system operation without relying on human intervention.

[0006] "Inquiry information" refers to data such as questions and issues provided to the system by users or other systems.

[0007] "Means of receiving information" refers to functions that capture inquiry information with maximum efficiency and accuracy.

[0008] "Means of analysis" refers to the function of processing information received to understand its meaning and the core of the problem, and to identify relevant data.

[0009] "Automated corrective actions" refer to corrective or adjustment work that a system performs spontaneously according to defined procedures.

[0010] "Means of implementation" refers to the mechanisms that support the process of activating decided countermeasures into concrete actions.

[0011] "Monitoring means" refer to functions for continuously checking the system's status and measuring abnormalities or changes.

[0012] "Escalation" refers to the process of handing over problems that cannot be resolved through normal automated processing to humans. [Brief explanation of the drawing]

[0013] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3]It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0015] First, the terms used in the following description will be explained.

[0016] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0017] In the following embodiments, the numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0018] In the following embodiments, the numbered storage is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0019] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0020] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0021] [First Embodiment]

[0022] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0023] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0024] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0025] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0026] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0027] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0028] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0029] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0030] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0031] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0032] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0033] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0034] This invention is a system that uses an AI agent to automate system operation inquiries and troubleshooting. This system mainly consists of a server, terminals, and users.

[0035] When a user enters a system-related inquiry or problem from their terminal, the server receives it. The server analyzes the inquiry information through an AI agent to identify the problem. For example, if a user reports that "the website is not displaying," the server checks the network connection status and the web server's operational status.

[0036] Based on the identified problem, the AI ​​agent determines an automated course of action. If necessary, the server implements corrective measures. For example, the server might terminate abnormal processes to free up resources and restore website performance.

[0037] Once processing is complete, the server monitors the system status and reports any changes to the user. This report allows the user to confirm that the problem has been resolved. If the AI ​​agent determines that it cannot handle the issue automatically, the server escalates it to a human operations staff member for further assistance.

[0038] This invention automates routine operational tasks, enabling rapid problem solving. This reduces the burden of system operation and contributes to addressing labor shortages.

[0039] The following describes the processing flow.

[0040] Step 1:

[0041] Users enter system-related problems or inquiries through their devices. This input is done via forms or chat interfaces and sent to the server.

[0042] Step 2:

[0043] The server receives inquiry information from the user and passes it to the AI ​​agent. The AI ​​agent uses natural language processing to analyze the content of the inquiry and identifies the problem by referring to log data and related information.

[0044] Step 3:

[0045] Based on the server's identification of the problem, an AI agent automatically determines the appropriate course of action. These actions may include configuration changes, process restarts, and resource reallocation.

[0046] Step 4:

[0047] The server executes the predetermined automated response. For example, if the server is overloaded, it will reallocate resources and stop unnecessary processes to reduce the load.

[0048] Step 5:

[0049] The server monitors the system status after execution to check for any improvements. If necessary, the corrective measures will be implemented again.

[0050] Step 6:

[0051] The server provides feedback to the user regarding the problem resolution status. If the problem is resolved, the user is notified accordingly. If the AI ​​agent cannot resolve the issue automatically, the server escalates it to the operations team.

[0052] (Example 1)

[0053] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0054] In the field of information processing, with the increasing number of system-related inquiries and troubleshooting, there is a growing need to resolve problems quickly and effectively. However, traditional methods lack sufficient automation and rely heavily on human resources, making efficient responses difficult. Furthermore, delays in providing feedback to users have led to decreased user satisfaction.

[0055] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0056] In this invention, the server includes means for acquiring query information, means for analyzing the acquired query to identify a problem, and means for determining an automatic solution based on the identified problem. This enables efficient automation of the query and fault response process and allows for rapid feedback to the user.

[0057] "Means for obtaining inquiry information" refers to a function for receiving inquiries from users regarding the system.

[0058] "Means for analyzing acquired inquiries to identify problems" refers to a function that analyzes received inquiry information and identifies specific problems based on its content.

[0059] "Means for automatically determining solutions based on identified problems" refers to a function that automatically selects and determines appropriate countermeasures for identified problems.

[0060] "Means of implementing the decided solution" refers to the function of actually putting the selected solution into action.

[0061] "Means for checking the system's operational status after implementation" refers to a function that monitors the system's state after the countermeasure has been implemented and verifies whether it is functioning correctly.

[0062] The "means of notifying operations personnel of a problem" refer to a function for escalating a problem to human operations personnel when automatic resolution is difficult.

[0063] "Means for analyzing inquiry information using natural language" refers to a function that converts inquiries entered by users in natural language into a format that a computer can understand and then analyzes them.

[0064] "Means of notifying users of system improvements resulting from implemented solutions" refers to a function that reports to users the improved state of the system as a result of the implemented measures.

[0065] This invention illustrates a specific embodiment of an automated inquiry processing system utilizing an AI agent. This system primarily consists of a server, a terminal, and a user. The user inputs problems and inquiries about the system via a terminal, which is done through an interface. Typical terminals include personal computers and smartphones.

[0066] The server is responsible for receiving inquiries from users. On the server, generative AI models and natural language processing (NLP) techniques are used to analyze the content of the inquiries and identify specific problems. To do this, the server accesses various databases and log files to extract information related to the inquiries.

[0067] Once a problem is identified, the AI ​​agent automatically determines a course of action based on historical data and pre-configured rules. The server then executes appropriate automated scripts and corrective actions according to the nature of the problem. For example, if the server's resource load is high, it can stabilize the system by terminating unnecessary processes.

[0068] As a concrete example, let's explain what happens when a user reports that they "cannot connect to the website." The server immediately checks the web server logs and network connection status, and as soon as it detects a specific connection problem, it executes the process recommended by the AI ​​agent to resolve the issue.

[0069] This system allows the use of the following prompt statements:

[0070] "Question for AI agent: Please explain the steps in the process of analyzing and resolving a problem where I cannot connect to a website."

[0071] As described above, this invention aims to efficiently automate the handling of inquiries related to system operation and improve the user experience.

[0072] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0073] Step 1:

[0074] The user enters system-related inquiries through the terminal interface. This involves entering specific problems or situations into text boxes and pressing the "Send" button. At this point, the terminal sends the inquiry data to the server in string format.

[0075] Step 2:

[0076] The server receives query data sent by the user. The server analyzes this data using natural language processing techniques to identify the query content. At this stage, a generative AI model operates to analyze the intent of the query. The output of the analysis includes the type of problem and related data points.

[0077] Step 3:

[0078] The server identifies the problem based on the analysis results. The server checks log files and system status to clarify what the actual problem is. For example, if there are communication errors or connection problems, the server investigates the network status and service operation. This process outputs specific data regarding the cause of the problem.

[0079] Step 4:

[0080] The server uses an AI model to determine an automated solution to the identified problem. The AI ​​agent refers to historical data and the system's rule set to select the optimal response. The selected solution is output in script or command format and ready for immediate execution.

[0081] Step 5:

[0082] The server will execute the determined solution. Specifically, it will run scripts on the server to restart necessary processes and change configurations. The goal of this execution is to restore the system to a normal state. As a result of the execution, the corrected system status will be output.

[0083] Step 6:

[0084] The server monitors the system status after execution. It continuously checks the system's operation using monitoring tools to confirm that the problem has been resolved. If additional intervention is needed, it detects this and prepares to implement corrective measures again.

[0085] Step 7:

[0086] The server notifies the user that the problem has been completely resolved. The resolution is displayed on the terminal screen, allowing the user to confirm that the problem has been fixed. This feedback also includes a summary of the actions taken.

[0087] Step 8:

[0088] If the server determines that it cannot automatically resolve the problem, it will escalate the issue to the operations team. Escalation sends a notification to the team so that they can quickly address any issues requiring further analysis and action. This notification includes details of the detected problem and recommended actions.

[0089] (Application Example 1)

[0090] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0091] System management and troubleshooting in data centers need to be performed quickly and accurately in real time. However, currently, this process relies on manual processes, making rapid problem resolution difficult and increasing the operational burden. Furthermore, insufficient visualization of abnormal conditions and system improvements leads to delays in communicating information to administrators.

[0092] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0093] In this invention, the server includes means for receiving query information, means for analyzing the query information to identify problems, and means for visualizing the system's operating status in real time through physical sensors. This enables system administrators in a data center to quickly detect failures, automatically execute problem-solving measures, and visualize the management status.

[0094] "Inquiry information" refers to data that includes problems and questions reported by users regarding the system.

[0095] "Analysis" is the process of thoroughly evaluating the received inquiry information and identifying specific problems.

[0096] An "automated response measure" is a solution that a system automatically implements in response to an identified problem.

[0097] "Execution" refers to the act of concretely starting the decided countermeasures within the system.

[0098] "Monitoring the system status" refers to the activity of detecting abnormalities early by continuously observing the system's operating status and performance.

[0099] "Escalation" is the process of requesting a human operations manager to conduct a detailed investigation and take action regarding a problem that the AI ​​agent cannot resolve.

[0100] A "physical sensor" is a device used to acquire system operating status and environmental data.

[0101] "Real-time visualization" refers to displaying the system's status and changes in a format that can be instantly and intuitively understood.

[0102] A "display device" is a device or apparatus used to physically provide monitoring data or information.

[0103] This system is designed to provide real-time, rapid, and accurate system management and fault response in data centers. The following is a specific implementation example.

[0104] The server uses a network communication module to receive query information. When a user reports a system problem using a terminal, this information is sent to the server. This received query information is analyzed using generative AI models such as Google Cloud AI and Microsoft Azure AI. If the analysis identifies a specific problem, the system automatically determines a corrective action.

[0105] The execution of automated response measures utilizes APIs to adjust system resources managed by the server. For example, this may include restarting the processes of web services where an anomaly has been detected. This process uses data collected from various sensor devices to visualize the system's operational status in real time. This visualization information is provided to display devices used by administrators, specifically smartphones and head-mounted displays.

[0106] In response to specific conditions or events, the server sends a generated status report to the user. This notification allows administrators to immediately understand the system's progress. For example, if the system goes down, a detailed prompt message might be sent stating, "Network connectivity issues, specific hardware in the server room is unresponsive."

[0107] This invention enables rapid response to failures and reduces operational burden. Furthermore, it improves data center management efficiency by monitoring and visualizing the system status in real time.

[0108] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0109] Step 1:

[0110] The user uses a terminal to input inquiry information about the system. This information is sent to the server via the network. The input data is the user's inquiry in text format, and the output is the raw data received by the server.

[0111] Step 2:

[0112] The server uses a generative AI model to analyze the received query information. It analyzes the input query information to identify specific problems. Data processing involves text analysis using natural language processing, and the resulting output is a list of identified problems.

[0113] Step 3:

[0114] The server determines the optimal automated solution based on the identified problem. It utilizes a generative AI model to automatically generate solutions tailored to the type of problem. The input is a list of identified problems, and the output is the determined solution. This process references a database of past solutions and their results.

[0115] Step 4:

[0116] The server then implements the determined countermeasures. Specifically, it adjusts system resources and restarts processes via the API. The input is the automated countermeasure, and the output is the result of the implemented countermeasure.

[0117] Step 5:

[0118] After implementation, the server uses physical sensors to monitor the system's operational status in real time. The input is operational status data obtained from the sensors, and after analysis, it verifies whether the problem has been resolved. This result is recorded as a log.

[0119] Step 6:

[0120] The server generates a status report based on the monitored information and notifies the user. The input is monitoring data and the problem resolution status, and the output is a detailed report sent to the user. Specifically, the status is updated in real time on the smartphone display device.

[0121] Step 7:

[0122] If necessary, the server escalates unresolved issues or incidents that the AI ​​cannot handle to human operators. The input is information about unresolved issues, and the output is a list of escalated issues. In this step, the operators are notified and instructed to take further manual action if necessary.

[0123] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0124] This invention is a system automation technology that combines an AI agent and an emotion engine, and highly automates inquiry handling and fault resolution in system operations. This system consists of a server, terminals, and users.

[0125] When a user using a terminal enters an inquiry or problem with the system, the server receives that information. The server uses an AI agent to analyze the inquiry information and identifies the problem by referring to log data and other relevant information. Furthermore, an emotion engine recognizes emotions from the user's input. For example, if a user reports "the payment screen isn't working" with feelings of frustration, the emotion engine analyzes that emotional state and provides feedback to the AI ​​agent.

[0126] Considering the identified problem and emotional state, the AI ​​agent determines the optimal automated response. The server then executes that response, taking actions such as restarting the relevant service.

[0127] After execution, the server monitors the system status to check for improvements. It sends feedback to the user and reports on the status of problem resolution. If the problem cannot be resolved through the automated process, it escalates detailed information, including the results of the sentiment engine, to the operations team. This allows for quick and appropriate human intervention if the user is dissatisfied or highly stressed.

[0128] This system can not only improve the efficiency of system operations but also contribute to increased user satisfaction.

[0129] The following describes the processing flow.

[0130] Step 1:

[0131] Users enter inquiries or reports of problems with the system in text format from their devices. Users then submit details using forms or chat widgets.

[0132] Step 2:

[0133] The server receives inquiry information from the user and passes it to the AI ​​agent. Simultaneously, the emotion engine analyzes the user's emotions based on their input.

[0134] Step 3:

[0135] The AI ​​agent analyzes the received inquiry, checks relevant log data and performance information to identify the problem. For example, it can detect an issue where a particular function is not responding.

[0136] Step 4:

[0137] Based on the results analyzed by the emotion engine, the AI ​​agent is notified of the user's emotional state. If emotions such as "stress" or "dissatisfaction" are detected, processing will be prioritized.

[0138] Step 5:

[0139] The server executes automated countermeasures based on instructions from the AI ​​agent. For example, it may restart system processes or adjust settings.

[0140] Step 6:

[0141] The server will verify the execution results and monitor system performance, evaluating whether improvements are being made. Further action will be taken as needed.

[0142] Step 7:

[0143] The server sends feedback to the user about the problem resolution status and the measures taken. The report is written in a way that quickly soothes the user's emotions.

[0144] Step 8:

[0145] If the issue is difficult to resolve, the server escalates the problem to the operations team along with the results of the emotion engine's analysis. This allows the operations team to take appropriate action based on the emotional situation.

[0146] (Example 2)

[0147] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0148] In modern information systems, promptly and accurately addressing user inquiries and problem reports is a crucial challenge. In particular, a lack of consideration for emotional aspects can lead to decreased user satisfaction and misunderstandings. Therefore, automated responses that consider user emotions, along with analysis of inquiry content, are required.

[0149] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0150] In this invention, the server includes means for receiving inquiry information, means for analyzing the received information in natural language to identify a problem, means for recognizing the user's emotional state, means for determining an automated response based on the identified problem and emotional state, means for executing the determined response, means for monitoring the state after execution to confirm improvement, means for reporting the improvement to the user, and means for escalating the information to the operations manager as necessary. This enables a swift and accurate response that takes the user's emotions into consideration.

[0151] "Inquiry information" refers to information about problems or questions that users enter into the system.

[0152] "Natural language" refers to the language that humans use on a daily basis, and is a form of unstructured, free-form text data.

[0153] "Emotional state" refers to the state of the user's psychological reactions and feelings as evaluated during input.

[0154] "Automated response measures" refer to solutions or actions that a system mechanically determines in response to a specific problem.

[0155] "Post-execution state" refers to the overall operating status and performance of the system after the system has implemented automated countermeasures.

[0156] "Monitoring" is the process of continuously checking the operation and status of a system and comparing it to expected standards.

[0157] "Escalation" is the process of handing over information to a human operations manager and requesting them to address a problem if it cannot be resolved automatically.

[0158] This system begins with the user entering a query to the system via a terminal. The terminal can be a digital device such as a personal computer or smartphone. When the user makes a query in natural language, that information is sent to the server.

[0159] The server receives this inquiry information and processes it using an AI agent and an emotion engine. The AI ​​agent uses natural language processing techniques to analyze the content of the inquiry. This analysis identifies the problem included in the user's inquiry. For example, if the user enters "the screen froze," the server will refer to log data to identify that problem.

[0160] Furthermore, the server uses an emotion engine to identify the emotional state the user is experiencing. For example, if a user's message is filled with frustration or anxiety, the server analyzes that emotional state and provides feedback to the AI ​​agent.

[0161] The AI ​​agent determines the optimal automated response based on the identified problem and the user's emotional state. This may involve actions such as restarting relevant software services. The server then monitors the system status to check for improvements. It provides feedback to the user regarding the improvement status and notifies them that the problem has been resolved.

[0162] For example, if a user reports a problem such as "the webpage won't display," the server uses an AI agent to check the web server logs and identify the connection problem. When the emotion engine detects the user's frustration, it prioritizes taking corrective action based on that information.

[0163] When analyzing problems through a generative AI model, the prompt "Please suggest an automated solution for when a user emotionally inputs 'the payment screen isn't working'" is used. In this way, the system can achieve problem-solving that takes user emotions into account, leading to increased efficiency in responses and improved user satisfaction.

[0164] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0165] Step 1:

[0166] The user enters inquiry information through their device. The device accepts input in natural language text format and sends the content to the server. The input includes specific information about the problem, such as "I cannot log in."

[0167] Step 2:

[0168] The server receives inquiry information from the terminal. It takes the received text data as input and prepares to pass it on to the AI ​​agent.

[0169] Step 3:

[0170] The server activates an AI agent to analyze the received query information. The AI ​​agent uses natural language processing techniques to analyze the text data and perform data calculations to identify the problem from the query. Specifically, it extracts problematic lines and error messages from relevant logs. The output generates information about the identified problem.

[0171] Step 4:

[0172] The server uses an emotion engine to analyze the user's emotional state. The input is the user's text message, and the emotion engine analyzes the tone and emotional expressions in the text to recognize the user's psychological state (e.g., irritation or anxiety). The output is the analyzed emotional state data.

[0173] Step 5:

[0174] The server receives problem identification information from the AI ​​agent and emotional state data from the emotion engine as input, and automatically determines a course of action. The AI ​​agent utilizes rule-based models and machine learning to select the optimal solution. This process creates an action list and generates its output.

[0175] Step 6:

[0176] The server will execute the determined course of action. Specifically, it will restart the relevant services within the system or run a script to resolve the problem. During this step, the server's activity history will be logged.

[0177] Step 7:

[0178] The server monitors the system status after execution. It checks performance metrics and log data to see if the problem has been resolved. The input is the data obtained from the previous execution steps, and the output is the evaluation result of the resolution status.

[0179] Step 8:

[0180] The server provides feedback to the user based on the monitoring results. It notifies the user that the system is functioning correctly and reports that the problem has been resolved. Specifically, it sends messages via email or a notification system.

[0181] Step 9:

[0182] If the problem is not resolved, the server will escalate the issue to operations personnel with detailed information, including the results of the sentiment engine. This provides operations staff with the necessary context to manually address the problem.

[0183] (Application Example 2)

[0184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0185] In the operation of information systems, it is crucial to respond to user inquiries and resolve problems quickly and effectively. However, conventional systems have limited automated response capabilities, and the optimization of response measures based on user emotions is insufficient. Furthermore, because inquiries are in natural language, misinterpretations and delays in response can occur. This leads to challenges such as decreased user satisfaction and reduced operational efficiency.

[0186] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0187] In this invention, the server includes means for receiving inquiry information, means for analyzing the received inquiry to identify a problem, means for analyzing user sentiment information, means for optimizing automated response measures based on the analyzed sentiment information, means for determining automated response measures for the problem, means for executing the determined response measures, means for monitoring the system state after execution, and means for escalating to a human operator as necessary. This enables automated inquiry handling and highly accurate problem solving that takes user sentiment into consideration.

[0188] "Means for receiving inquiry information" refers to devices or programs that have the function of receiving data related to inquiries and problems sent from users to the system.

[0189] "Means for analyzing received inquiries and identifying problems" refers to devices or programs that process data based on received inquiry information and have the function of identifying specific problems.

[0190] "Means for determining automated countermeasures" refers to devices or programs that have the function of selecting the optimal countermeasure for an identified problem and formulating a plan for its implementation.

[0191] "Means for implementing the decided countermeasures" refers to devices or programs that have the function of actually putting the decided countermeasures into action and carrying out the set processes.

[0192] "Means for monitoring the system state after execution" refers to devices or programs that have the function of observing the system status after the implementation of countermeasures and confirming the degree of improvement of the problem.

[0193] "Means of escalating to operations personnel" refers to devices or programs that, when a problem cannot be resolved through automated means, have the function of forwarding detailed information to a human administrator and requesting further action.

[0194] "Means for analyzing user emotional information" refers to devices or programs that have the function of processing and analyzing the emotions contained in information received from users in order to understand their content.

[0195] "Means for optimizing automated response measures based on analyzed emotional information" refers to devices or programs that have the function of selecting or adjusting the optimal response measure while taking into account the user's emotional data.

[0196] The system implementing this invention consists of a server, a terminal, and a user. The server receives query information and performs analysis using an AI agent and an emotion engine. Specifically, the server interprets the query information sent by the user using natural language processing technology to identify the problem. Python and TENSORFLOW® are used as the main software environment.

[0197] After analysis, the server uses an emotion engine to evaluate the emotional information contained in the user's input. At this time, data analysis that takes the user's emotions into account is performed, and the optimal automated response to the problem is determined. For example, if a user inputs an expression of frustration such as "payment not completed," the AI ​​agent will use that information to identify the cause and consider whether to reconnect to the service.

[0198] The selected countermeasures are automatically implemented, and the system's improvement status is monitored. If automatic resolution is difficult, detailed information is escalated to the operations team. The server has a function to provide feedback on problem resolution in a way that also takes user feelings into consideration.

[0199] As a concrete example, consider a case where a user reports that "a payment failed, but the charge is still showing up in my account." The server immediately identifies the problem, automates the appropriate action, and sends reassuring feedback to the user saying, "The problem has been resolved. Please try again."

[0200] An example of a prompt using a generative AI model is: "Analyze the user's frustration regarding the payment failure and suggest a solution." This allows the server to perform advanced sentiment analysis and improve the accuracy of automated responses.

[0201] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0202] Step 1:

[0203] The user enters query information using a terminal. This input includes a description of the problem and expressions of sentiment in natural language. The terminal sends this information to the server, which receives the input data. Here, the input is the text data entered by the user, and the output is the query data transferred to the server.

[0204] Step 2:

[0205] The server analyzes the received query data using natural language processing techniques (generative AI models using Python and TensorFlow). The input is query data, which the server analyzes as text to identify the problem. The output of this process is the identified problem and the keywords and information that comprise it. Specific operations include word tokenization and sentiment extraction.

[0206] Step 3:

[0207] The server uses an emotion engine to analyze the user's emotional information. The input at this stage is the user's original text data and analyzed keywords, while the output is information about the type and intensity of the emotion. Based on this, the server evaluates the user's emotional state and feeds this information back to an AI model that generates emotional information. Specific actions include applying emotion analysis algorithms to classify the emotions.

[0208] Step 4:

[0209] The server determines an automated response based on the identified problem and analyzed sentiment information. The input consists of the problem identification information and sentiment information obtained in the preceding steps. The server processes this information and selects an appropriate response. The output is the selected response. Specific operations include referencing an internal database and analyzing similar past cases to determine the appropriate response.

[0210] Step 5:

[0211] The server executes the chosen countermeasure. The input here is the automatically selected countermeasure, and the output is the result of the countermeasure's execution and the change in system state. The server attempts to resolve the problem by calling relevant APIs or launching internal processes. Specific actions may include system restarts or database modifications.

[0212] Step 6:

[0213] The server monitors the system status after execution and checks for improvements. The input is system log data after execution, and the output is the evaluation result of the system's health or abnormality. If the server detects an anomaly during monitoring, it further analyzes it and takes additional actions as needed. Specific actions include inspecting real-time logs and evaluating performance metrics.

[0214] Step 7:

[0215] The system sends feedback to the user and reports on the status of problem resolution. Inputs include an evaluation of the improvement status and the user's emotional information, while output is a feedback message that takes emotional information into consideration. The server sends the feedback and provides a specific and heartfelt message to reassure the user.

[0216] Step 8:

[0217] If the issue cannot be resolved, the server escalates detailed information, including the results of the sentiment engine, to the operations team. Inputs are unresolved issue information and sentiment information, while output is a detailed escalation report received by the operations team. The server generates a detailed report and forwards it to the appropriate person to facilitate a quick human response. Specific actions include organizing detailed log files and automatically generating reports.

[0218] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0219] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0220] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0221] [Second Embodiment]

[0222] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0223] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0224] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0225] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0226] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0227] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0228] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0229] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0230] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0231] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0232] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0233] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0234] This invention is a system that uses an AI agent to automate system operation inquiries and troubleshooting. This system mainly consists of a server, terminals, and users.

[0235] When a user enters a system-related inquiry or problem from their terminal, the server receives it. The server analyzes the inquiry information through an AI agent to identify the problem. For example, if a user reports that "the website is not displaying," the server checks the network connection status and the web server's operational status.

[0236] Based on the identified problem, the AI ​​agent determines an automated course of action. If necessary, the server implements corrective measures. For example, the server might terminate abnormal processes to free up resources and restore website performance.

[0237] Once processing is complete, the server monitors the system status and reports any changes to the user. This report allows the user to confirm that the problem has been resolved. If the AI ​​agent determines that it cannot handle the issue automatically, the server escalates it to a human operations staff member for further assistance.

[0238] This invention automates routine operational tasks, enabling rapid problem solving. This reduces the burden of system operation and contributes to addressing labor shortages.

[0239] The following describes the processing flow.

[0240] Step 1:

[0241] Users enter system-related problems or inquiries through their devices. This input is done via forms or chat interfaces and sent to the server.

[0242] Step 2:

[0243] The server receives inquiry information from the user and passes it to the AI ​​agent. The AI ​​agent uses natural language processing to analyze the content of the inquiry and identifies the problem by referring to log data and related information.

[0244] Step 3:

[0245] Based on the server's identification of the problem, an AI agent automatically determines the appropriate course of action. These actions may include configuration changes, process restarts, and resource reallocation.

[0246] Step 4:

[0247] The server executes the predetermined automated response. For example, if the server is overloaded, it will reallocate resources and stop unnecessary processes to reduce the load.

[0248] Step 5:

[0249] The server monitors the system status after execution to check for any improvements. If necessary, the corrective measures will be implemented again.

[0250] Step 6:

[0251] The server provides feedback to the user regarding the problem resolution status. If the problem is resolved, the user is notified accordingly. If the AI ​​agent cannot resolve the issue automatically, the server escalates it to the operations team.

[0252] (Example 1)

[0253] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0254] In the field of information processing, with the increasing number of system-related inquiries and troubleshooting, there is a growing need to resolve problems quickly and effectively. However, traditional methods lack sufficient automation and rely heavily on human resources, making efficient responses difficult. Furthermore, delays in providing feedback to users have led to decreased user satisfaction.

[0255] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0256] In this invention, the server includes means for acquiring query information, means for analyzing the acquired query to identify a problem, and means for determining an automatic solution based on the identified problem. This enables efficient automation of the query and fault response process and allows for rapid feedback to the user.

[0257] "Means for obtaining inquiry information" refers to a function for receiving inquiries from users regarding the system.

[0258] "Means for analyzing acquired inquiries to identify problems" refers to a function that analyzes received inquiry information and identifies specific problems based on its content.

[0259] "Means for automatically determining solutions based on identified problems" refers to a function that automatically selects and determines appropriate countermeasures for identified problems.

[0260] "Means of implementing the decided solution" refers to the function of actually putting the selected solution into action.

[0261] "Means for checking the system's operational status after implementation" refers to a function that monitors the system's state after the countermeasure has been implemented and verifies whether it is functioning correctly.

[0262] The "means of notifying operations personnel of a problem" refer to a function for escalating a problem to human operations personnel when automatic resolution is difficult.

[0263] "Means for analyzing inquiry information using natural language" refers to a function that converts inquiries entered by users in natural language into a format that a computer can understand and then analyzes them.

[0264] "Means of notifying users of system improvements resulting from implemented solutions" refers to a function that reports to users the improved state of the system as a result of the implemented measures.

[0265] This invention illustrates a specific embodiment of an automated inquiry processing system utilizing an AI agent. This system primarily consists of a server, a terminal, and a user. The user inputs problems and inquiries about the system via a terminal, which is done through an interface. Typical terminals include personal computers and smartphones.

[0266] The server is responsible for receiving inquiries from users. On the server, generative AI models and natural language processing (NLP) techniques are used to analyze the content of the inquiries and identify specific problems. To do this, the server accesses various databases and log files to extract information related to the inquiries.

[0267] Once a problem is identified, the AI ​​agent automatically determines a course of action based on historical data and pre-configured rules. The server then executes appropriate automated scripts and corrective actions according to the nature of the problem. For example, if the server's resource load is high, it can stabilize the system by terminating unnecessary processes.

[0268] As a concrete example, let's explain what happens when a user reports that they "cannot connect to the website." The server immediately checks the web server logs and network connection status, and as soon as it detects a specific connection problem, it executes the process recommended by the AI ​​agent to resolve the issue.

[0269] This system allows the use of the following prompt statements:

[0270] "Question for AI agent: Please explain the steps in the process of analyzing and resolving a problem where I cannot connect to a website."

[0271] As described above, this invention aims to efficiently automate the handling of inquiries related to system operation and improve the user experience.

[0272] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0273] Step 1:

[0274] The user enters system-related inquiries through the terminal interface. This involves entering specific problems or situations into text boxes and pressing the "Send" button. At this point, the terminal sends the inquiry data to the server in string format.

[0275] Step 2:

[0276] The server receives query data sent by the user. The server analyzes this data using natural language processing techniques to identify the query content. At this stage, a generative AI model operates to analyze the intent of the query. The output of the analysis includes the type of problem and related data points.

[0277] Step 3:

[0278] The server identifies the problem based on the analysis results. The server checks log files and system status to clarify what the actual problem is. For example, if there are communication errors or connection problems, the server investigates the network status and service operation. This process outputs specific data regarding the cause of the problem.

[0279] Step 4:

[0280] The server uses an AI model to determine an automated solution to the identified problem. The AI ​​agent refers to historical data and the system's rule set to select the optimal response. The selected solution is output in script or command format and ready for immediate execution.

[0281] Step 5:

[0282] The server executes the determined solution. Specifically, it runs a script on the server to restart necessary processes and make configuration changes, aiming to return the system to a normal state. As a result of the execution, the modified system status is output.

[0283] Step 6:

[0284] The server monitors the system state after execution. It continuously checks the system's operating status using a monitoring tool to confirm that the problem has been solved. If additional intervention is required, it detects it and prepares to implement countermeasures again.

[0285] Step 7:

[0286] The server notifies the user that the problem has been completely solved. It displays the solution result on the terminal screen so that the user can confirm the resolution of the problem. This feedback also includes an overview of what processing was done.

[0287] Step 8:

[0288] If the server determines that it cannot automatically solve the problem, it escalates to the operator. Through escalation, a notification is sent to the operator so that they can quickly respond to problems that require detailed analysis and countermeasures. This notification includes the details of the detected problem and recommended actions.

[0289] (Application Example 1)

[0290] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0291] System management and troubleshooting in data centers need to be performed quickly and accurately in real time. However, currently, this process relies on manual processes, making rapid problem resolution difficult and increasing the operational burden. Furthermore, insufficient visualization of abnormal conditions and system improvements leads to delays in communicating information to administrators.

[0292] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0293] In this invention, the server includes means for receiving query information, means for analyzing the query information to identify problems, and means for visualizing the system's operating status in real time through physical sensors. This enables system administrators in a data center to quickly detect failures, automatically execute problem-solving measures, and visualize the management status.

[0294] "Inquiry information" refers to data that includes problems and questions reported by users regarding the system.

[0295] "Analysis" is the process of thoroughly evaluating the received inquiry information and identifying specific problems.

[0296] An "automated response measure" is a solution that a system automatically implements in response to an identified problem.

[0297] "Execution" refers to the act of concretely starting the decided countermeasures within the system.

[0298] "Monitoring the system status" refers to the activity of detecting abnormalities early by continuously observing the system's operating status and performance.

[0299] "Escalation" is the process of requesting a human operations manager to conduct a detailed investigation and take action regarding a problem that the AI ​​agent cannot resolve.

[0300] A "physical sensor" is a device used to acquire system operating status and environmental data.

[0301] "Real-time visualization" refers to displaying the system's status and changes in a format that can be instantly and intuitively understood.

[0302] A "display device" is a device or apparatus used to physically provide monitoring data or information.

[0303] This system is designed to provide real-time, rapid, and accurate system management and fault response in data centers. The following is a specific implementation example.

[0304] The server uses a network communication module to receive query information. When a user reports a system problem using a terminal, this information is sent to the server. This received query information is analyzed using generative AI models such as Google Cloud AI and Microsoft Azure AI. If the analysis identifies a specific problem, the system automatically determines a corrective action.

[0305] The execution of automated response measures utilizes APIs to adjust system resources managed by the server. For example, this may include restarting the processes of web services where an anomaly has been detected. This process uses data collected from various sensor devices to visualize the system's operational status in real time. This visualization information is provided to display devices used by administrators, specifically smartphones and head-mounted displays.

[0306] According to a specific state or event, the server sends the generated situation report to the user. By notifying this information, the administrator can immediately grasp the improvement status of the system. For example, when the system goes down, a detailed prompt message such as "There is a problem with the network connection, and a specific piece of hardware in the server room is not responding" is sent.

[0307] This invention enables rapid response during failures and reduction of operation load. Also, by monitoring and visualizing the system state in real time, the management efficiency of the data center is improved.

[0308] The flow of the specific process in Application Example 1 will be described using FIG. 12.

[0309] Step 1:

[0310] The user uses the terminal to input inquiry information about the system. This information is sent to the server through the network. The input data is the user's text-based inquiry, and the output is the raw data received by the server.

[0311] Step 2:

[0312] The server utilizes the generated AI model to analyze the received inquiry information. It analyzes the input inquiry information to identify specific problem points. As data processing, text analysis using natural language processing is performed, and the output as a result is a list of identified problems.

[0313] Step 3:

[0314] Based on the identified problems, the server determines the optimal automatic response measures. It utilizes the generated AI model to automatically generate solutions according to the type of problem. The input is the list of identified problems, and the output is the determined response measures. In this process, the database of past response measures and results is referred to.

[0315] Step 4:

[0316] The server then implements the determined countermeasures. Specifically, it adjusts system resources and restarts processes via the API. The input is the automated countermeasure, and the output is the result of the implemented countermeasure.

[0317] Step 5:

[0318] After implementation, the server uses physical sensors to monitor the system's operational status in real time. The input is operational status data obtained from the sensors, and after analysis, it verifies whether the problem has been resolved. This result is recorded as a log.

[0319] Step 6:

[0320] The server generates a status report based on the monitored information and notifies the user. The input is monitoring data and the problem resolution status, and the output is a detailed report sent to the user. Specifically, the status is updated in real time on the smartphone display device.

[0321] Step 7:

[0322] If necessary, the server escalates unresolved issues or incidents that the AI ​​cannot handle to human operators. The input is information about unresolved issues, and the output is a list of escalated issues. In this step, the operators are notified and instructed to take further manual action if necessary.

[0323] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0324] This invention is a system automation technology that combines an AI agent and an emotion engine, and highly automates inquiry handling and fault resolution in system operations. This system consists of a server, terminals, and users.

[0325] When a user using a terminal enters an inquiry or problem with the system, the server receives that information. The server uses an AI agent to analyze the inquiry information and identifies the problem by referring to log data and other relevant information. Furthermore, an emotion engine recognizes emotions from the user's input. For example, if a user reports "the payment screen isn't working" with feelings of frustration, the emotion engine analyzes that emotional state and provides feedback to the AI ​​agent.

[0326] Considering the identified problem and emotional state, the AI ​​agent determines the optimal automated response. The server then executes that response, taking actions such as restarting the relevant service.

[0327] After execution, the server monitors the system status to check for improvements. It sends feedback to the user and reports on the status of problem resolution. If the problem cannot be resolved through the automated process, it escalates detailed information, including the results of the sentiment engine, to the operations team. This allows for quick and appropriate human intervention if the user is dissatisfied or highly stressed.

[0328] This system can not only improve the efficiency of system operations but also contribute to increased user satisfaction.

[0329] The following describes the processing flow.

[0330] Step 1:

[0331] Users enter inquiries or reports of problems with the system in text format from their devices. Users then submit details using forms or chat widgets.

[0332] Step 2:

[0333] The server receives inquiry information from the user and passes it to the AI ​​agent. Simultaneously, the emotion engine analyzes the user's emotions based on their input.

[0334] Step 3:

[0335] The AI ​​agent analyzes the received inquiry, checks relevant log data and performance information to identify the problem. For example, it can detect an issue where a particular function is not responding.

[0336] Step 4:

[0337] Based on the results analyzed by the emotion engine, the AI ​​agent is notified of the user's emotional state. If emotions such as "stress" or "dissatisfaction" are detected, processing will be prioritized.

[0338] Step 5:

[0339] The server executes automated countermeasures based on instructions from the AI ​​agent. For example, it may restart system processes or adjust settings.

[0340] Step 6:

[0341] The server will verify the execution results and monitor system performance, evaluating whether improvements are being made. Further action will be taken as needed.

[0342] Step 7:

[0343] The server sends feedback to the user about the problem resolution status and the measures taken. The report is written in a way that quickly soothes the user's emotions.

[0344] Step 8:

[0345] If the issue is difficult to resolve, the server escalates the problem to the operations team along with the results of the emotion engine's analysis. This allows the operations team to take appropriate action based on the emotional situation.

[0346] (Example 2)

[0347] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0348] In modern information systems, promptly and accurately addressing user inquiries and problem reports is a crucial challenge. In particular, a lack of consideration for emotional aspects can lead to decreased user satisfaction and misunderstandings. Therefore, automated responses that consider user emotions, along with analysis of inquiry content, are required.

[0349] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0350] In this invention, the server includes means for receiving inquiry information, means for analyzing the received information in natural language to identify a problem, means for recognizing the user's emotional state, means for determining an automated response based on the identified problem and emotional state, means for executing the determined response, means for monitoring the state after execution to confirm improvement, means for reporting the improvement to the user, and means for escalating the information to the operations manager as necessary. This enables a swift and accurate response that takes the user's emotions into consideration.

[0351] "Inquiry information" refers to information about problems or questions that users enter into the system.

[0352] "Natural language" refers to the language that humans use on a daily basis, and is a form of unstructured, free-form text data.

[0353] "Emotional state" refers to the state of the user's psychological reactions and feelings as evaluated during input.

[0354] "Automated response measures" refer to solutions or actions that a system mechanically determines in response to a specific problem.

[0355] "Post-execution state" refers to the overall operating status and performance of the system after the system has implemented automated countermeasures.

[0356] "Monitoring" is the process of continuously checking the operation and status of a system and comparing it to expected standards.

[0357] "Escalation" is the process of handing over information to a human operations manager and requesting them to address a problem if it cannot be resolved automatically.

[0358] This system begins with the user entering a query to the system via a terminal. The terminal can be a digital device such as a personal computer or smartphone. When the user makes a query in natural language, that information is sent to the server.

[0359] The server receives this inquiry information and processes it using an AI agent and an emotion engine. The AI ​​agent uses natural language processing techniques to analyze the content of the inquiry. This analysis identifies the problem included in the user's inquiry. For example, if the user enters "the screen froze," the server will refer to log data to identify that problem.

[0360] Furthermore, the server uses an emotion engine to identify the emotional state the user is experiencing. For example, if a user's message is filled with frustration or anxiety, the server analyzes that emotional state and provides feedback to the AI ​​agent.

[0361] The AI ​​agent determines the optimal automated response based on the identified problem and the user's emotional state. This may involve actions such as restarting relevant software services. The server then monitors the system status to check for improvements. It provides feedback to the user regarding the improvement status and notifies them that the problem has been resolved.

[0362] For example, if a user reports a problem such as "the webpage won't display," the server uses an AI agent to check the web server logs and identify the connection problem. When the emotion engine detects the user's frustration, it prioritizes taking corrective action based on that information.

[0363] When analyzing problems through a generative AI model, the prompt "Please suggest an automated solution for when a user emotionally inputs 'the payment screen isn't working'" is used. In this way, the system can achieve problem-solving that takes user emotions into account, leading to increased efficiency in responses and improved user satisfaction.

[0364] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0365] Step 1:

[0366] The user enters inquiry information through their device. The device accepts input in natural language text format and sends the content to the server. The input includes specific information about the problem, such as "I cannot log in."

[0367] Step 2:

[0368] The server receives inquiry information from the terminal. It takes the received text data as input and prepares to pass it on to the AI ​​agent.

[0369] Step 3:

[0370] The server activates an AI agent to analyze the received query information. The AI ​​agent uses natural language processing techniques to analyze the text data and perform data calculations to identify the problem from the query. Specifically, it extracts problematic lines and error messages from relevant logs. The output generates information about the identified problem.

[0371] Step 4:

[0372] The server uses an emotion engine to analyze the user's emotional state. The input is the user's text message, and the emotion engine analyzes the tone and emotional expressions in the text to recognize the user's psychological state (e.g., irritation or anxiety). The output is the analyzed emotional state data.

[0373] Step 5:

[0374] The server receives problem identification information from the AI ​​agent and emotional state data from the emotion engine as input, and automatically determines a course of action. The AI ​​agent utilizes rule-based models and machine learning to select the optimal solution. This process creates an action list and generates its output.

[0375] Step 6:

[0376] The server will execute the determined course of action. Specifically, it will restart the relevant services within the system or run a script to resolve the problem. During this step, the server's activity history will be logged.

[0377] Step 7:

[0378] The server monitors the system status after execution. It checks performance metrics and log data to see if the problem has been resolved. The input is the data obtained from the previous execution steps, and the output is the evaluation result of the resolution status.

[0379] Step 8:

[0380] The server provides feedback to the user based on the monitoring results. It notifies the user that the system is functioning correctly and reports that the problem has been resolved. Specifically, it sends messages via email or a notification system.

[0381] Step 9:

[0382] If the problem is not resolved, the server will escalate the issue to operations personnel with detailed information, including the results of the sentiment engine. This provides operations staff with the necessary context to manually address the problem.

[0383] (Application Example 2)

[0384] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the smart glasses 214 as the "terminal".

[0385] In the operation of information systems, it is crucial to respond to user inquiries and resolve problems quickly and effectively. However, conventional systems have limited automated response capabilities, and the optimization of response measures based on user emotions is insufficient. Furthermore, because inquiries are in natural language, misinterpretations and delays in response can occur. This leads to challenges such as decreased user satisfaction and reduced operational efficiency.

[0386] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0387] In this invention, the server includes means for receiving inquiry information, means for analyzing the received inquiry to identify a problem, means for analyzing user sentiment information, means for optimizing automated response measures based on the analyzed sentiment information, means for determining automated response measures for the problem, means for executing the determined response measures, means for monitoring the system state after execution, and means for escalating to a human operator as necessary. This enables automated inquiry handling and highly accurate problem solving that takes user sentiment into consideration.

[0388] "Means for receiving inquiry information" refers to devices or programs that have the function of receiving data related to inquiries and problems sent from users to the system.

[0389] "Means for analyzing received inquiries and identifying problems" refers to devices or programs that process data based on received inquiry information and have the function of identifying specific problems.

[0390] "Means for determining automated countermeasures" refers to devices or programs that have the function of selecting the optimal countermeasure for an identified problem and formulating a plan for its implementation.

[0391] "Means for implementing the decided countermeasures" refers to devices or programs that have the function of actually putting the decided countermeasures into action and carrying out the set processes.

[0392] "Means for monitoring the system state after execution" refers to devices or programs that have the function of observing the system status after the implementation of countermeasures and confirming the degree of improvement of the problem.

[0393] "Means of escalating to operations personnel" refers to devices or programs that, when a problem cannot be resolved through automated means, have the function of forwarding detailed information to a human administrator and requesting further action.

[0394] "Means for analyzing user emotional information" refers to devices or programs that have the function of processing and analyzing the emotions contained in information received from users in order to understand their content.

[0395] "Means for optimizing automated response measures based on analyzed emotional information" refers to devices or programs that have the function of selecting or adjusting the optimal response measure while taking into account the user's emotional data.

[0396] The system implementing this invention consists of a server, a terminal, and a user. The server receives query information and performs analysis using an AI agent and an emotion engine. Specifically, the server interprets the query information sent by the user using natural language processing techniques to identify the problem. Python and TensorFlow are used as the main software environments.

[0397] After analysis, the server uses an emotion engine to evaluate the emotional information contained in the user's input. At this time, data analysis that takes the user's emotions into account is performed, and the optimal automated response to the problem is determined. For example, if a user inputs an expression of frustration such as "payment not completed," the AI ​​agent will use that information to identify the cause and consider whether to reconnect to the service.

[0398] The selected countermeasures are automatically implemented, and the system's improvement status is monitored. If automatic resolution is difficult, detailed information is escalated to the operations team. The server has a function to provide feedback on problem resolution in a way that also takes user feelings into consideration.

[0399] As a concrete example, consider a case where a user reports that "a payment failed, but the charge is still showing up in my account." The server immediately identifies the problem, automates the appropriate action, and sends reassuring feedback to the user saying, "The problem has been resolved. Please try again."

[0400] An example of a prompt using a generative AI model is: "Analyze the user's frustration regarding the payment failure and suggest a solution." This allows the server to perform advanced sentiment analysis and improve the accuracy of automated responses.

[0401] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0402] Step 1:

[0403] The user enters query information using a terminal. This input includes a description of the problem and expressions of sentiment in natural language. The terminal sends this information to the server, which receives the input data. Here, the input is the text data entered by the user, and the output is the query data transferred to the server.

[0404] Step 2:

[0405] The server analyzes the received query data using natural language processing techniques (generative AI models using Python and TensorFlow). The input is query data, which the server analyzes as text to identify the problem. The output of this process is the identified problem and the keywords and information that comprise it. Specific operations include word tokenization and sentiment extraction.

[0406] Step 3:

[0407] The server uses an emotion engine to analyze the user's emotional information. The input at this stage is the user's original text data and analyzed keywords, while the output is information about the type and intensity of the emotion. Based on this, the server evaluates the user's emotional state and feeds this information back to an AI model that generates emotional information. Specific actions include applying emotion analysis algorithms to classify the emotions.

[0408] Step 4:

[0409] The server determines an automated response based on the identified problem and analyzed sentiment information. The input consists of the problem identification information and sentiment information obtained in the preceding steps. The server processes this information and selects an appropriate response. The output is the selected response. Specific operations include referencing an internal database and analyzing similar past cases to determine the appropriate response.

[0410] Step 5:

[0411] The server executes the chosen countermeasure. The input here is the automatically selected countermeasure, and the output is the result of the countermeasure's execution and the change in system state. The server attempts to resolve the problem by calling relevant APIs or launching internal processes. Specific actions may include system restarts or database modifications.

[0412] Step 6:

[0413] The server monitors the system status after execution and checks for improvements. The input is system log data after execution, and the output is the evaluation result of the system's health or abnormality. If the server detects an anomaly during monitoring, it further analyzes it and takes additional actions as needed. Specific actions include inspecting real-time logs and evaluating performance metrics.

[0414] Step 7:

[0415] The system sends feedback to the user and reports on the status of problem resolution. Inputs include an evaluation of the improvement status and the user's emotional information, while output is a feedback message that takes emotional information into consideration. The server sends the feedback and provides a specific and heartfelt message to reassure the user.

[0416] Step 8:

[0417] If the issue cannot be resolved, the server escalates detailed information, including the results of the sentiment engine, to the operations team. Inputs are unresolved issue information and sentiment information, while output is a detailed escalation report received by the operations team. The server generates a detailed report and forwards it to the appropriate person to facilitate a quick human response. Specific actions include organizing detailed log files and automatically generating reports.

[0418] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0419] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0420] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0421] [Third Embodiment]

[0422] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0423] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0424] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0425] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0426] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0427] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0428] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0429] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0430] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0431] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0432] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0433] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0434] This invention is a system that uses an AI agent to automate system operation inquiries and troubleshooting. This system mainly consists of a server, terminals, and users.

[0435] When a user enters a system-related inquiry or problem from their terminal, the server receives it. The server analyzes the inquiry information through an AI agent to identify the problem. For example, if a user reports that "the website is not displaying," the server checks the network connection status and the web server's operational status.

[0436] Based on the identified problem, the AI ​​agent determines an automated course of action. If necessary, the server implements corrective measures. For example, the server might terminate abnormal processes to free up resources and restore website performance.

[0437] Once processing is complete, the server monitors the system status and reports any changes to the user. This report allows the user to confirm that the problem has been resolved. If the AI ​​agent determines that it cannot handle the issue automatically, the server escalates it to a human operations staff member for further assistance.

[0438] This invention automates routine operational tasks, enabling rapid problem solving. This reduces the burden of system operation and contributes to addressing labor shortages.

[0439] The following describes the processing flow.

[0440] Step 1:

[0441] Users enter system-related problems or inquiries through their devices. This input is done via forms or chat interfaces and sent to the server.

[0442] Step 2:

[0443] The server receives inquiry information from the user and passes it to the AI ​​agent. The AI ​​agent uses natural language processing to analyze the content of the inquiry and identifies the problem by referring to log data and related information.

[0444] Step 3:

[0445] Based on the server's identification of the problem, an AI agent automatically determines the appropriate course of action. These actions may include configuration changes, process restarts, and resource reallocation.

[0446] Step 4:

[0447] The server executes the predetermined automated response. For example, if the server is overloaded, it will reallocate resources and stop unnecessary processes to reduce the load.

[0448] Step 5:

[0449] The server monitors the system status after execution to check for any improvements. If necessary, the corrective measures will be implemented again.

[0450] Step 6:

[0451] The server provides feedback to the user regarding the problem resolution status. If the problem is resolved, the user is notified accordingly. If the AI ​​agent cannot resolve the issue automatically, the server escalates it to the operations team.

[0452] (Example 1)

[0453] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0454] In the field of information processing, with the increasing number of system-related inquiries and troubleshooting, there is a growing need to resolve problems quickly and effectively. However, traditional methods lack sufficient automation and rely heavily on human resources, making efficient responses difficult. Furthermore, delays in providing feedback to users have led to decreased user satisfaction.

[0455] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0456] In this invention, the server includes means for acquiring query information, means for analyzing the acquired query to identify a problem, and means for determining an automatic solution based on the identified problem. This enables efficient automation of the query and fault response process and allows for rapid feedback to the user.

[0457] "Means for obtaining inquiry information" refers to a function for receiving inquiries from users regarding the system.

[0458] "Means for analyzing acquired inquiries to identify problems" refers to a function that analyzes received inquiry information and identifies specific problems based on its content.

[0459] "Means for automatically determining solutions based on identified problems" refers to a function that automatically selects and determines appropriate countermeasures for identified problems.

[0460] "Means of implementing the decided solution" refers to the function of actually putting the selected solution into action.

[0461] "Means for checking the system's operational status after implementation" refers to a function that monitors the system's state after the countermeasure has been implemented and verifies whether it is functioning correctly.

[0462] The "means of notifying operations personnel of a problem" refer to a function for escalating a problem to human operations personnel when automatic resolution is difficult.

[0463] "Means for analyzing inquiry information using natural language" refers to a function that converts inquiries entered by users in natural language into a format that a computer can understand and then analyzes them.

[0464] "Means of notifying users of system improvements resulting from implemented solutions" refers to a function that reports to users the improved state of the system as a result of the implemented measures.

[0465] This invention illustrates a specific embodiment of an automated inquiry processing system utilizing an AI agent. This system primarily consists of a server, a terminal, and a user. The user inputs problems and inquiries about the system via a terminal, which is done through an interface. Typical terminals include personal computers and smartphones.

[0466] The server is responsible for receiving inquiries from users. On the server, generative AI models and natural language processing (NLP) techniques are used to analyze the content of the inquiries and identify specific problems. To do this, the server accesses various databases and log files to extract information related to the inquiries.

[0467] Once a problem is identified, the AI ​​agent automatically determines a course of action based on historical data and pre-configured rules. The server then executes appropriate automated scripts and corrective actions according to the nature of the problem. For example, if the server's resource load is high, it can stabilize the system by terminating unnecessary processes.

[0468] As a concrete example, let's explain what happens when a user reports that they "cannot connect to the website." The server immediately checks the web server logs and network connection status, and as soon as it detects a specific connection problem, it executes the process recommended by the AI ​​agent to resolve the issue.

[0469] This system allows the use of the following prompt statements:

[0470] "Question for AI agent: Please explain the steps in the process of analyzing and resolving a problem where I cannot connect to a website."

[0471] As described above, this invention aims to efficiently automate the handling of inquiries related to system operation and improve the user experience.

[0472] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0473] Step 1:

[0474] The user enters system-related inquiries through the terminal interface. This involves entering specific problems or situations into text boxes and pressing the "Send" button. At this point, the terminal sends the inquiry data to the server in string format.

[0475] Step 2:

[0476] The server receives query data sent by the user. The server analyzes this data using natural language processing techniques to identify the query content. At this stage, a generative AI model operates to analyze the intent of the query. The output of the analysis includes the type of problem and related data points.

[0477] Step 3:

[0478] The server identifies the problem based on the analysis results. The server checks log files and system status to clarify what the actual problem is. For example, if there are communication errors or connection problems, the server investigates the network status and service operation. This process outputs specific data regarding the cause of the problem.

[0479] Step 4:

[0480] The server uses an AI model to determine an automated solution to the identified problem. The AI ​​agent refers to historical data and the system's rule set to select the optimal response. The selected solution is output in script or command format and ready for immediate execution.

[0481] Step 5:

[0482] The server will execute the determined solution. Specifically, it will run scripts on the server to restart necessary processes and change configurations. The goal of this execution is to restore the system to a normal state. As a result of the execution, the corrected system status will be output.

[0483] Step 6:

[0484] The server monitors the system status after execution. It continuously checks the system's operation using monitoring tools to confirm that the problem has been resolved. If additional intervention is needed, it detects this and prepares to implement corrective measures again.

[0485] Step 7:

[0486] The server notifies the user that the problem has been completely resolved. The resolution is displayed on the terminal screen, allowing the user to confirm that the problem has been fixed. This feedback also includes a summary of the actions taken.

[0487] Step 8:

[0488] If the server determines that it cannot automatically resolve the problem, it will escalate the issue to the operations team. Escalation sends a notification to the team so that they can quickly address any issues requiring further analysis and action. This notification includes details of the detected problem and recommended actions.

[0489] (Application Example 1)

[0490] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0491] System management and troubleshooting in data centers need to be performed quickly and accurately in real time. However, currently, this process relies on manual processes, making rapid problem resolution difficult and increasing the operational burden. Furthermore, insufficient visualization of abnormal conditions and system improvements leads to delays in communicating information to administrators.

[0492] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0493] In this invention, the server includes means for receiving query information, means for analyzing the query information to identify problems, and means for visualizing the system's operating status in real time through physical sensors. This enables system administrators in a data center to quickly detect failures, automatically execute problem-solving measures, and visualize the management status.

[0494] "Inquiry information" refers to data that includes problems and questions reported by users regarding the system.

[0495] "Analysis" is the process of thoroughly evaluating the received inquiry information and identifying specific problems.

[0496] An "automated response measure" is a solution that a system automatically implements in response to an identified problem.

[0497] "Execution" refers to the act of concretely starting the decided countermeasures within the system.

[0498] "Monitoring the system status" refers to the activity of detecting abnormalities early by continuously observing the system's operating status and performance.

[0499] "Escalation" is the process of requesting a human operations manager to conduct a detailed investigation and take action regarding a problem that the AI ​​agent cannot resolve.

[0500] A "physical sensor" is a device used to acquire system operating status and environmental data.

[0501] "Real-time visualization" refers to displaying the system's status and changes in a format that can be instantly and intuitively understood.

[0502] A "display device" is a device or apparatus used to physically provide monitoring data or information.

[0503] This system is designed to provide real-time, rapid, and accurate system management and fault response in data centers. The following is a specific implementation example.

[0504] The server uses a network communication module to receive query information. When a user reports a system problem using a terminal, this information is sent to the server. This received query information is analyzed using generative AI models such as Google Cloud AI and Microsoft Azure AI. If the analysis identifies a specific problem, the system automatically determines a corrective action.

[0505] The execution of automated response measures utilizes APIs to adjust system resources managed by the server. For example, this may include restarting the processes of web services where an anomaly has been detected. This process uses data collected from various sensor devices to visualize the system's operational status in real time. This visualization information is provided to display devices used by administrators, specifically smartphones and head-mounted displays.

[0506] In response to specific conditions or events, the server sends a generated status report to the user. This notification allows administrators to immediately understand the system's progress. For example, if the system goes down, a detailed prompt message might be sent stating, "Network connectivity issues, specific hardware in the server room is unresponsive."

[0507] This invention enables rapid response to failures and reduces operational burden. Furthermore, it improves data center management efficiency by monitoring and visualizing the system status in real time.

[0508] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0509] Step 1:

[0510] The user uses a terminal to input inquiry information about the system. This information is sent to the server via the network. The input data is the user's inquiry in text format, and the output is the raw data received by the server.

[0511] Step 2:

[0512] The server uses a generative AI model to analyze the received query information. It analyzes the input query information to identify specific problems. Data processing involves text analysis using natural language processing, and the resulting output is a list of identified problems.

[0513] Step 3:

[0514] The server determines the optimal automated solution based on the identified problem. It utilizes a generative AI model to automatically generate solutions tailored to the type of problem. The input is a list of identified problems, and the output is the determined solution. This process references a database of past solutions and their results.

[0515] Step 4:

[0516] The server then implements the determined countermeasures. Specifically, it adjusts system resources and restarts processes via the API. The input is the automated countermeasure, and the output is the result of the implemented countermeasure.

[0517] Step 5:

[0518] After implementation, the server uses physical sensors to monitor the system's operational status in real time. The input is operational status data obtained from the sensors, and after analysis, it verifies whether the problem has been resolved. This result is recorded as a log.

[0519] Step 6:

[0520] The server generates a status report based on the monitored information and notifies the user. The input is monitoring data and the problem resolution status, and the output is a detailed report sent to the user. Specifically, the status is updated in real time on the smartphone display device.

[0521] Step 7:

[0522] If necessary, the server escalates unresolved issues or incidents that the AI ​​cannot handle to human operators. The input is information about unresolved issues, and the output is a list of escalated issues. In this step, the operators are notified and instructed to take further manual action if necessary.

[0523] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0524] This invention is a system automation technology that combines an AI agent and an emotion engine, and highly automates inquiry handling and fault resolution in system operations. This system consists of a server, terminals, and users.

[0525] When a user using a terminal enters an inquiry or problem with the system, the server receives that information. The server uses an AI agent to analyze the inquiry information and identifies the problem by referring to log data and other relevant information. Furthermore, an emotion engine recognizes emotions from the user's input. For example, if a user reports "the payment screen isn't working" with feelings of frustration, the emotion engine analyzes that emotional state and provides feedback to the AI ​​agent.

[0526] Considering the identified problem and emotional state, the AI ​​agent determines the optimal automated response. The server then executes that response, taking actions such as restarting the relevant service.

[0527] After execution, the server monitors the system status to check for improvements. It sends feedback to the user and reports on the status of problem resolution. If the problem cannot be resolved through the automated process, it escalates detailed information, including the results of the sentiment engine, to the operations team. This allows for quick and appropriate human intervention if the user is dissatisfied or highly stressed.

[0528] This system can not only improve the efficiency of system operations but also contribute to increased user satisfaction.

[0529] The following describes the processing flow.

[0530] Step 1:

[0531] Users enter inquiries or reports of problems with the system in text format from their devices. Users then submit details using forms or chat widgets.

[0532] Step 2:

[0533] The server receives inquiry information from the user and passes it to the AI ​​agent. Simultaneously, the emotion engine analyzes the user's emotions based on their input.

[0534] Step 3:

[0535] The AI ​​agent analyzes the received inquiry, checks relevant log data and performance information to identify the problem. For example, it can detect an issue where a particular function is not responding.

[0536] Step 4:

[0537] Based on the results analyzed by the emotion engine, the AI ​​agent is notified of the user's emotional state. If emotions such as "stress" or "dissatisfaction" are detected, processing will be prioritized.

[0538] Step 5:

[0539] The server executes automated countermeasures based on instructions from the AI ​​agent. For example, it may restart system processes or adjust settings.

[0540] Step 6:

[0541] The server will verify the execution results and monitor system performance, evaluating whether improvements are being made. Further action will be taken as needed.

[0542] Step 7:

[0543] The server sends feedback to the user about the problem resolution status and the measures taken. The report is written in a way that quickly soothes the user's emotions.

[0544] Step 8:

[0545] If the issue is difficult to resolve, the server escalates the problem to the operations team along with the results of the emotion engine's analysis. This allows the operations team to take appropriate action based on the emotional situation.

[0546] (Example 2)

[0547] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0548] In modern information systems, promptly and accurately addressing user inquiries and problem reports is a crucial challenge. In particular, a lack of consideration for emotional aspects can lead to decreased user satisfaction and misunderstandings. Therefore, automated responses that consider user emotions, along with analysis of inquiry content, are required.

[0549] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0550] In this invention, the server includes means for receiving inquiry information, means for analyzing the received information in natural language to identify a problem, means for recognizing the user's emotional state, means for determining an automated response based on the identified problem and emotional state, means for executing the determined response, means for monitoring the state after execution to confirm improvement, means for reporting the improvement to the user, and means for escalating the information to the operations manager as necessary. This enables a swift and accurate response that takes the user's emotions into consideration.

[0551] "Inquiry information" refers to information about problems or questions that users enter into the system.

[0552] "Natural language" refers to the language that humans use on a daily basis, and is a form of unstructured, free-form text data.

[0553] "Emotional state" refers to the state of the user's psychological reactions and feelings as evaluated during input.

[0554] "Automated response measures" refer to solutions or actions that a system mechanically determines in response to a specific problem.

[0555] "Post-execution state" refers to the overall operating status and performance of the system after the system has implemented automated countermeasures.

[0556] "Monitoring" is the process of continuously checking the operation and status of a system and comparing it to expected standards.

[0557] "Escalation" is the process of handing over information to a human operations manager and requesting them to address a problem if it cannot be resolved automatically.

[0558] This system begins with the user entering a query to the system via a terminal. The terminal can be a digital device such as a personal computer or smartphone. When the user makes a query in natural language, that information is sent to the server.

[0559] The server receives this inquiry information and processes it using an AI agent and an emotion engine. The AI ​​agent uses natural language processing techniques to analyze the content of the inquiry. This analysis identifies the problem included in the user's inquiry. For example, if the user enters "the screen froze," the server will refer to log data to identify that problem.

[0560] Furthermore, the server uses an emotion engine to identify the emotional state the user is experiencing. For example, if a user's message is filled with frustration or anxiety, the server analyzes that emotional state and provides feedback to the AI ​​agent.

[0561] The AI ​​agent determines the optimal automated response based on the identified problem and the user's emotional state. This may involve actions such as restarting relevant software services. The server then monitors the system status to check for improvements. It provides feedback to the user regarding the improvement status and notifies them that the problem has been resolved.

[0562] For example, if a user reports a problem such as "the webpage won't display," the server uses an AI agent to check the web server logs and identify the connection problem. When the emotion engine detects the user's frustration, it prioritizes taking corrective action based on that information.

[0563] When analyzing problems through a generative AI model, the prompt "Please suggest an automated solution for when a user emotionally inputs 'the payment screen isn't working'" is used. In this way, the system can achieve problem-solving that takes user emotions into account, leading to increased efficiency in responses and improved user satisfaction.

[0564] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0565] Step 1:

[0566] The user enters inquiry information through their device. The device accepts input in natural language text format and sends the content to the server. The input includes specific information about the problem, such as "I cannot log in."

[0567] Step 2:

[0568] The server receives inquiry information from the terminal. It takes the received text data as input and prepares to pass it on to the AI ​​agent.

[0569] Step 3:

[0570] The server activates an AI agent to analyze the received query information. The AI ​​agent uses natural language processing techniques to analyze the text data and perform data calculations to identify the problem from the query. Specifically, it extracts problematic lines and error messages from relevant logs. The output generates information about the identified problem.

[0571] Step 4:

[0572] The server uses an emotion engine to analyze the user's emotional state. The input is the user's text message, and the emotion engine analyzes the tone and emotional expressions in the text to recognize the user's psychological state (e.g., irritation or anxiety). The output is the analyzed emotional state data.

[0573] Step 5:

[0574] The server receives problem identification information from the AI ​​agent and emotional state data from the emotion engine as input, and automatically determines a course of action. The AI ​​agent utilizes rule-based models and machine learning to select the optimal solution. This process creates an action list and generates its output.

[0575] Step 6:

[0576] The server will execute the determined course of action. Specifically, it will restart the relevant services within the system or run a script to resolve the problem. During this step, the server's activity history will be logged.

[0577] Step 7:

[0578] The server monitors the system status after execution. It checks performance metrics and log data to see if the problem has been resolved. The input is the data obtained from the previous execution steps, and the output is the evaluation result of the resolution status.

[0579] Step 8:

[0580] The server provides feedback to the user based on the monitoring results. It notifies the user that the system is functioning correctly and reports that the problem has been resolved. Specifically, it sends messages via email or a notification system.

[0581] Step 9:

[0582] If the problem is not resolved, the server will escalate the issue to operations personnel with detailed information, including the results of the sentiment engine. This provides operations staff with the necessary context to manually address the problem.

[0583] (Application Example 2)

[0584] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0585] In the operation of information systems, it is crucial to respond to user inquiries and resolve problems quickly and effectively. However, conventional systems have limited automated response capabilities, and the optimization of response measures based on user emotions is insufficient. Furthermore, because inquiries are in natural language, misinterpretations and delays in response can occur. This leads to challenges such as decreased user satisfaction and reduced operational efficiency.

[0586] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0587] In this invention, the server includes means for receiving inquiry information, means for analyzing the received inquiry to identify a problem, means for analyzing user sentiment information, means for optimizing automated response measures based on the analyzed sentiment information, means for determining automated response measures for the problem, means for executing the determined response measures, means for monitoring the system state after execution, and means for escalating to a human operator as necessary. This enables automated inquiry handling and highly accurate problem solving that takes user sentiment into consideration.

[0588] "Means for receiving inquiry information" refers to devices or programs that have the function of receiving data related to inquiries and problems sent from users to the system.

[0589] "Means for analyzing received inquiries and identifying problems" refers to devices or programs that process data based on received inquiry information and have the function of identifying specific problems.

[0590] "Means for determining automated countermeasures" refers to devices or programs that have the function of selecting the optimal countermeasure for an identified problem and formulating a plan for its implementation.

[0591] "Means for implementing the decided countermeasures" refers to devices or programs that have the function of actually putting the decided countermeasures into action and carrying out the set processes.

[0592] "Means for monitoring the system state after execution" refers to devices or programs that have the function of observing the system status after the implementation of countermeasures and confirming the degree of improvement of the problem.

[0593] "Means of escalating to operations personnel" refers to devices or programs that, when a problem cannot be resolved through automated means, have the function of forwarding detailed information to a human administrator and requesting further action.

[0594] "Means for analyzing user emotional information" refers to devices or programs that have the function of processing and analyzing the emotions contained in information received from users in order to understand their content.

[0595] "Means for optimizing automated response measures based on analyzed emotional information" refers to devices or programs that have the function of selecting or adjusting the optimal response measure while taking into account the user's emotional data.

[0596] The system implementing this invention consists of a server, a terminal, and a user. The server receives query information and performs analysis using an AI agent and an emotion engine. Specifically, the server interprets the query information sent by the user using natural language processing techniques to identify the problem. Python and TensorFlow are used as the main software environments.

[0597] After analysis, the server uses an emotion engine to evaluate the emotional information contained in the user's input. At this time, data analysis that takes the user's emotions into account is performed, and the optimal automated response to the problem is determined. For example, if a user inputs an expression of frustration such as "payment not completed," the AI ​​agent will use that information to identify the cause and consider whether to reconnect to the service.

[0598] The selected countermeasures are automatically implemented, and the system's improvement status is monitored. If automatic resolution is difficult, detailed information is escalated to the operations team. The server has a function to provide feedback on problem resolution in a way that also takes user feelings into consideration.

[0599] As a concrete example, consider a case where a user reports that "a payment failed, but the charge is still showing up in my account." The server immediately identifies the problem, automates the appropriate action, and sends reassuring feedback to the user saying, "The problem has been resolved. Please try again."

[0600] An example of a prompt using a generative AI model is: "Analyze the user's frustration regarding the payment failure and suggest a solution." This allows the server to perform advanced sentiment analysis and improve the accuracy of automated responses.

[0601] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0602] Step 1:

[0603] The user enters query information using a terminal. This input includes a description of the problem and expressions of sentiment in natural language. The terminal sends this information to the server, which receives the input data. Here, the input is the text data entered by the user, and the output is the query data transferred to the server.

[0604] Step 2:

[0605] The server analyzes the received query data using natural language processing techniques (generative AI models using Python and TensorFlow). The input is query data, which the server analyzes as text to identify the problem. The output of this process is the identified problem and the keywords and information that comprise it. Specific operations include word tokenization and sentiment extraction.

[0606] Step 3:

[0607] The server uses an emotion engine to analyze the user's emotional information. The input at this stage is the user's original text data and analyzed keywords, while the output is information about the type and intensity of the emotion. Based on this, the server evaluates the user's emotional state and feeds this information back to an AI model that generates emotional information. Specific actions include applying emotion analysis algorithms to classify the emotions.

[0608] Step 4:

[0609] The server determines an automated response based on the identified problem and analyzed sentiment information. The input consists of the problem identification information and sentiment information obtained in the preceding steps. The server processes this information and selects an appropriate response. The output is the selected response. Specific operations include referencing an internal database and analyzing similar past cases to determine the appropriate response.

[0610] Step 5:

[0611] The server executes the chosen countermeasure. The input here is the automatically selected countermeasure, and the output is the result of the countermeasure's execution and the change in system state. The server attempts to resolve the problem by calling relevant APIs or launching internal processes. Specific actions may include system restarts or database modifications.

[0612] Step 6:

[0613] The server monitors the system status after execution and checks for improvements. The input is system log data after execution, and the output is the evaluation result of the system's health or abnormality. If the server detects an anomaly during monitoring, it further analyzes it and takes additional actions as needed. Specific actions include inspecting real-time logs and evaluating performance metrics.

[0614] Step 7:

[0615] The system sends feedback to the user and reports on the status of problem resolution. Inputs include an evaluation of the improvement status and the user's emotional information, while output is a feedback message that takes emotional information into consideration. The server sends the feedback and provides a specific and heartfelt message to reassure the user.

[0616] Step 8:

[0617] If the issue cannot be resolved, the server escalates detailed information, including the results of the sentiment engine, to the operations team. Inputs are unresolved issue information and sentiment information, while output is a detailed escalation report received by the operations team. The server generates a detailed report and forwards it to the appropriate person to facilitate a quick human response. Specific actions include organizing detailed log files and automatically generating reports.

[0618] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0619] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0620] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0621] [Fourth Embodiment]

[0622] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0623] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0624] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0625] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0626] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0627] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0628] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0629] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0630] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0631] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0632] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0633] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0634] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0635] This invention is a system that uses an AI agent to automate system operation inquiries and troubleshooting. This system mainly consists of a server, terminals, and users.

[0636] When a user enters a system-related inquiry or problem from their terminal, the server receives it. The server analyzes the inquiry information through an AI agent to identify the problem. For example, if a user reports that "the website is not displaying," the server checks the network connection status and the web server's operational status.

[0637] Based on the identified problem, the AI ​​agent determines an automated course of action. If necessary, the server implements corrective measures. For example, the server might terminate abnormal processes to free up resources and restore website performance.

[0638] Once processing is complete, the server monitors the system status and reports any changes to the user. This report allows the user to confirm that the problem has been resolved. If the AI ​​agent determines that it cannot handle the issue automatically, the server escalates it to a human operations staff member for further assistance.

[0639] This invention automates routine operational tasks, enabling rapid problem solving. This reduces the burden of system operation and contributes to addressing labor shortages.

[0640] The following describes the processing flow.

[0641] Step 1:

[0642] Users enter system-related problems or inquiries through their devices. This input is done via forms or chat interfaces and sent to the server.

[0643] Step 2:

[0644] The server receives inquiry information from the user and passes it to the AI ​​agent. The AI ​​agent uses natural language processing to analyze the content of the inquiry and identifies the problem by referring to log data and related information.

[0645] Step 3:

[0646] Based on the server's identification of the problem, an AI agent automatically determines the appropriate course of action. These actions may include configuration changes, process restarts, and resource reallocation.

[0647] Step 4:

[0648] The server executes the predetermined automated response. For example, if the server is overloaded, it will reallocate resources and stop unnecessary processes to reduce the load.

[0649] Step 5:

[0650] The server monitors the system status after execution to check for any improvements. If necessary, the corrective measures will be implemented again.

[0651] Step 6:

[0652] The server provides feedback to the user regarding the problem resolution status. If the problem is resolved, the user is notified accordingly. If the AI ​​agent cannot resolve the issue automatically, the server escalates it to the operations team.

[0653] (Example 1)

[0654] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0655] In the field of information processing, with the increasing number of system-related inquiries and troubleshooting, there is a growing need to resolve problems quickly and effectively. However, traditional methods lack sufficient automation and rely heavily on human resources, making efficient responses difficult. Furthermore, delays in providing feedback to users have led to decreased user satisfaction.

[0656] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0657] In this invention, the server includes means for acquiring query information, means for analyzing the acquired query to identify a problem, and means for determining an automatic solution based on the identified problem. This enables efficient automation of the query and fault response process and allows for rapid feedback to the user.

[0658] "Means for obtaining inquiry information" refers to a function for receiving inquiries from users regarding the system.

[0659] "Means for analyzing acquired inquiries to identify problems" refers to a function that analyzes received inquiry information and identifies specific problems based on its content.

[0660] "Means for automatically determining solutions based on identified problems" refers to a function that automatically selects and determines appropriate countermeasures for identified problems.

[0661] "Means of implementing the decided solution" refers to the function of actually putting the selected solution into action.

[0662] "Means for checking the system's operational status after implementation" refers to a function that monitors the system's state after the countermeasure has been implemented and verifies whether it is functioning correctly.

[0663] The "means of notifying operations personnel of a problem" refer to a function for escalating a problem to human operations personnel when automatic resolution is difficult.

[0664] "Means for analyzing inquiry information using natural language" refers to a function that converts inquiries entered by users in natural language into a format that a computer can understand and then analyzes them.

[0665] "Means of notifying users of system improvements resulting from implemented solutions" refers to a function that reports to users the improved state of the system as a result of the implemented measures.

[0666] This invention illustrates a specific embodiment of an automated inquiry processing system utilizing an AI agent. This system primarily consists of a server, a terminal, and a user. The user inputs problems and inquiries about the system via a terminal, which is done through an interface. Typical terminals include personal computers and smartphones.

[0667] The server is responsible for receiving inquiries from users. On the server, generative AI models and natural language processing (NLP) techniques are used to analyze the content of the inquiries and identify specific problems. To do this, the server accesses various databases and log files to extract information related to the inquiries.

[0668] Once a problem is identified, the AI ​​agent automatically determines a course of action based on historical data and pre-configured rules. The server then executes appropriate automated scripts and corrective actions according to the nature of the problem. For example, if the server's resource load is high, it can stabilize the system by terminating unnecessary processes.

[0669] As a concrete example, let's explain what happens when a user reports that they "cannot connect to the website." The server immediately checks the web server logs and network connection status, and as soon as it detects a specific connection problem, it executes the process recommended by the AI ​​agent to resolve the issue.

[0670] This system allows the use of the following prompt statements:

[0671] "Question for AI agent: Please explain the steps in the process of analyzing and resolving a problem where I cannot connect to a website."

[0672] As described above, this invention aims to efficiently automate the handling of inquiries related to system operation and improve the user experience.

[0673] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0674] Step 1:

[0675] The user enters system-related inquiries through the terminal interface. This involves entering specific problems or situations into text boxes and pressing the "Send" button. At this point, the terminal sends the inquiry data to the server in string format.

[0676] Step 2:

[0677] The server receives query data sent by the user. The server analyzes this data using natural language processing techniques to identify the query content. At this stage, a generative AI model operates to analyze the intent of the query. The output of the analysis includes the type of problem and related data points.

[0678] Step 3:

[0679] The server identifies the problem based on the analysis results. The server checks log files and system status to clarify what the actual problem is. For example, if there are communication errors or connection problems, the server investigates the network status and service operation. This process outputs specific data regarding the cause of the problem.

[0680] Step 4:

[0681] The server uses an AI model to determine an automated solution to the identified problem. The AI ​​agent refers to historical data and the system's rule set to select the optimal response. The selected solution is output in script or command format and ready for immediate execution.

[0682] Step 5:

[0683] The server will execute the determined solution. Specifically, it will run scripts on the server to restart necessary processes and change configurations. The goal of this execution is to restore the system to a normal state. As a result of the execution, the corrected system status will be output.

[0684] Step 6:

[0685] The server monitors the system status after execution. It continuously checks the system's operation using monitoring tools to confirm that the problem has been resolved. If additional intervention is needed, it detects this and prepares to implement corrective measures again.

[0686] Step 7:

[0687] The server notifies the user that the problem has been completely resolved. The resolution is displayed on the terminal screen, allowing the user to confirm that the problem has been fixed. This feedback also includes a summary of the actions taken.

[0688] Step 8:

[0689] If the server determines that it cannot automatically resolve the problem, it will escalate the issue to the operations team. Escalation sends a notification to the team so that they can quickly address any issues requiring further analysis and action. This notification includes details of the detected problem and recommended actions.

[0690] (Application Example 1)

[0691] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0692] System management and troubleshooting in data centers need to be performed quickly and accurately in real time. However, currently, this process relies on manual processes, making rapid problem resolution difficult and increasing the operational burden. Furthermore, insufficient visualization of abnormal conditions and system improvements leads to delays in communicating information to administrators.

[0693] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0694] In this invention, the server includes means for receiving query information, means for analyzing the query information to identify problems, and means for visualizing the system's operating status in real time through physical sensors. This enables system administrators in a data center to quickly detect failures, automatically execute problem-solving measures, and visualize the management status.

[0695] "Inquiry information" refers to data that includes problems and questions reported by users regarding the system.

[0696] "Analysis" is the process of thoroughly evaluating the received inquiry information and identifying specific problems.

[0697] An "automated response measure" is a solution that a system automatically implements in response to an identified problem.

[0698] "Execution" refers to the act of concretely starting the decided countermeasures within the system.

[0699] "Monitoring the system status" refers to the activity of detecting abnormalities early by continuously observing the system's operating status and performance.

[0700] "Escalation" is the process of requesting a human operations manager to conduct a detailed investigation and take action regarding a problem that the AI ​​agent cannot resolve.

[0701] A "physical sensor" is a device used to acquire system operating status and environmental data.

[0702] "Real-time visualization" refers to displaying the system's status and changes in a format that can be instantly and intuitively understood.

[0703] A "display device" is a device or apparatus used to physically provide monitoring data or information.

[0704] This system is designed to provide real-time, rapid, and accurate system management and fault response in data centers. The following is a specific implementation example.

[0705] The server uses a network communication module to receive query information. When a user reports a system problem using a terminal, this information is sent to the server. This received query information is analyzed using generative AI models such as Google Cloud AI and Microsoft Azure AI. If the analysis identifies a specific problem, the system automatically determines a corrective action.

[0706] The execution of automated response measures utilizes APIs to adjust system resources managed by the server. For example, this may include restarting the processes of web services where an anomaly has been detected. This process uses data collected from various sensor devices to visualize the system's operational status in real time. This visualization information is provided to display devices used by administrators, specifically smartphones and head-mounted displays.

[0707] In response to specific conditions or events, the server sends a generated status report to the user. This notification allows administrators to immediately understand the system's progress. For example, if the system goes down, a detailed prompt message might be sent stating, "Network connectivity issues, specific hardware in the server room is unresponsive."

[0708] This invention enables rapid response to failures and reduces operational burden. Furthermore, it improves data center management efficiency by monitoring and visualizing the system status in real time.

[0709] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0710] Step 1:

[0711] The user uses a terminal to input inquiry information about the system. This information is sent to the server via the network. The input data is the user's inquiry in text format, and the output is the raw data received by the server.

[0712] Step 2:

[0713] The server uses a generative AI model to analyze the received query information. It analyzes the input query information to identify specific problems. Data processing involves text analysis using natural language processing, and the resulting output is a list of identified problems.

[0714] Step 3:

[0715] The server determines the optimal automated solution based on the identified problem. It utilizes a generative AI model to automatically generate solutions tailored to the type of problem. The input is a list of identified problems, and the output is the determined solution. This process references a database of past solutions and their results.

[0716] Step 4:

[0717] The server then implements the determined countermeasures. Specifically, it adjusts system resources and restarts processes via the API. The input is the automated countermeasure, and the output is the result of the implemented countermeasure.

[0718] Step 5:

[0719] After implementation, the server uses physical sensors to monitor the system's operational status in real time. The input is operational status data obtained from the sensors, and after analysis, it verifies whether the problem has been resolved. This result is recorded as a log.

[0720] Step 6:

[0721] The server generates a status report based on the monitored information and notifies the user. The input is monitoring data and the problem resolution status, and the output is a detailed report sent to the user. Specifically, the status is updated in real time on the smartphone display device.

[0722] Step 7:

[0723] If necessary, the server escalates unresolved issues or incidents that the AI ​​cannot handle to human operators. The input is information about unresolved issues, and the output is a list of escalated issues. In this step, the operators are notified and instructed to take further manual action if necessary.

[0724] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0725] This invention is a system automation technology that combines an AI agent and an emotion engine, and highly automates inquiry handling and fault resolution in system operations. This system consists of a server, terminals, and users.

[0726] When a user using a terminal enters an inquiry or problem with the system, the server receives that information. The server uses an AI agent to analyze the inquiry information and identifies the problem by referring to log data and other relevant information. Furthermore, an emotion engine recognizes emotions from the user's input. For example, if a user reports "the payment screen isn't working" with feelings of frustration, the emotion engine analyzes that emotional state and provides feedback to the AI ​​agent.

[0727] Considering the identified problem and emotional state, the AI ​​agent determines the optimal automated response. The server then executes that response, taking actions such as restarting the relevant service.

[0728] After execution, the server monitors the system status to check for improvements. It sends feedback to the user and reports on the status of problem resolution. If the problem cannot be resolved through the automated process, it escalates detailed information, including the results of the sentiment engine, to the operations team. This allows for quick and appropriate human intervention if the user is dissatisfied or highly stressed.

[0729] This system can not only improve the efficiency of system operations but also contribute to increased user satisfaction.

[0730] The following describes the processing flow.

[0731] Step 1:

[0732] Users enter inquiries or reports of problems with the system in text format from their devices. Users then submit details using forms or chat widgets.

[0733] Step 2:

[0734] The server receives inquiry information from the user and passes it to the AI ​​agent. Simultaneously, the emotion engine analyzes the user's emotions based on their input.

[0735] Step 3:

[0736] The AI ​​agent analyzes the received inquiry, checks relevant log data and performance information to identify the problem. For example, it can detect an issue where a particular function is not responding.

[0737] Step 4:

[0738] Based on the results analyzed by the emotion engine, the AI ​​agent is notified of the user's emotional state. If emotions such as "stress" or "dissatisfaction" are detected, processing will be prioritized.

[0739] Step 5:

[0740] The server executes automated countermeasures based on instructions from the AI ​​agent. For example, it may restart system processes or adjust settings.

[0741] Step 6:

[0742] The server will verify the execution results and monitor system performance, evaluating whether improvements are being made. Further action will be taken as needed.

[0743] Step 7:

[0744] The server sends feedback to the user about the problem resolution status and the measures taken. The report is written in a way that quickly soothes the user's emotions.

[0745] Step 8:

[0746] If the issue is difficult to resolve, the server escalates the problem to the operations team along with the results of the emotion engine's analysis. This allows the operations team to take appropriate action based on the emotional situation.

[0747] (Example 2)

[0748] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0749] In modern information systems, promptly and accurately addressing user inquiries and problem reports is a crucial challenge. In particular, a lack of consideration for emotional aspects can lead to decreased user satisfaction and misunderstandings. Therefore, automated responses that consider user emotions, along with analysis of inquiry content, are required.

[0750] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0751] In this invention, the server includes means for receiving inquiry information, means for analyzing the received information in natural language to identify a problem, means for recognizing the user's emotional state, means for determining an automated response based on the identified problem and emotional state, means for executing the determined response, means for monitoring the state after execution to confirm improvement, means for reporting the improvement to the user, and means for escalating the information to the operations manager as necessary. This enables a swift and accurate response that takes the user's emotions into consideration.

[0752] "Inquiry information" refers to information about problems or questions that users enter into the system.

[0753] "Natural language" refers to the language that humans use on a daily basis, and is a form of unstructured, free-form text data.

[0754] "Emotional state" refers to the state of the user's psychological reactions and feelings as evaluated during input.

[0755] "Automated response measures" refer to solutions or actions that a system mechanically determines in response to a specific problem.

[0756] "Post-execution state" refers to the overall operating status and performance of the system after the system has implemented automated countermeasures.

[0757] "Monitoring" is the process of continuously checking the operation and status of a system and comparing it to expected standards.

[0758] "Escalation" is the process of handing over information to a human operations manager and requesting them to address a problem if it cannot be resolved automatically.

[0759] This system begins with the user entering a query to the system via a terminal. The terminal can be a digital device such as a personal computer or smartphone. When the user makes a query in natural language, that information is sent to the server.

[0760] The server receives this inquiry information and processes it using an AI agent and an emotion engine. The AI ​​agent uses natural language processing techniques to analyze the content of the inquiry. This analysis identifies the problem included in the user's inquiry. For example, if the user enters "the screen froze," the server will refer to log data to identify that problem.

[0761] Furthermore, the server uses an emotion engine to identify the emotional state the user is experiencing. For example, if a user's message is filled with frustration or anxiety, the server analyzes that emotional state and provides feedback to the AI ​​agent.

[0762] The AI ​​agent determines the optimal automated response based on the identified problem and the user's emotional state. This may involve actions such as restarting relevant software services. The server then monitors the system status to check for improvements. It provides feedback to the user regarding the improvement status and notifies them that the problem has been resolved.

[0763] For example, if a user reports a problem such as "the webpage won't display," the server uses an AI agent to check the web server logs and identify the connection problem. When the emotion engine detects the user's frustration, it prioritizes taking corrective action based on that information.

[0764] When analyzing problems through a generative AI model, the prompt "Please suggest an automated solution for when a user emotionally inputs 'the payment screen isn't working'" is used. In this way, the system can achieve problem-solving that takes user emotions into account, leading to increased efficiency in responses and improved user satisfaction.

[0765] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0766] Step 1:

[0767] The user enters inquiry information through their device. The device accepts input in natural language text format and sends the content to the server. The input includes specific information about the problem, such as "I cannot log in."

[0768] Step 2:

[0769] The server receives inquiry information from the terminal. It takes the received text data as input and prepares to pass it on to the AI ​​agent.

[0770] Step 3:

[0771] The server activates an AI agent to analyze the received query information. The AI ​​agent uses natural language processing techniques to analyze the text data and perform data calculations to identify the problem from the query. Specifically, it extracts problematic lines and error messages from relevant logs. The output generates information about the identified problem.

[0772] Step 4:

[0773] The server uses an emotion engine to analyze the user's emotional state. The input is the user's text message, and the emotion engine analyzes the tone and emotional expressions in the text to recognize the user's psychological state (e.g., irritation or anxiety). The output is the analyzed emotional state data.

[0774] Step 5:

[0775] The server receives problem identification information from the AI ​​agent and emotional state data from the emotion engine as input, and automatically determines a course of action. The AI ​​agent utilizes rule-based models and machine learning to select the optimal solution. This process creates an action list and generates its output.

[0776] Step 6:

[0777] The server will execute the determined course of action. Specifically, it will restart the relevant services within the system or run a script to resolve the problem. During this step, the server's activity history will be logged.

[0778] Step 7:

[0779] The server monitors the system status after execution. It checks performance metrics and log data to see if the problem has been resolved. The input is the data obtained from the previous execution steps, and the output is the evaluation result of the resolution status.

[0780] Step 8:

[0781] The server provides feedback to the user based on the monitoring results. It notifies the user that the system is functioning correctly and reports that the problem has been resolved. Specifically, it sends messages via email or a notification system.

[0782] Step 9:

[0783] If the problem is not resolved, the server will escalate the issue to operations personnel with detailed information, including the results of the sentiment engine. This provides operations staff with the necessary context to manually address the problem.

[0784] (Application Example 2)

[0785] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0786] In the operation of information systems, it is crucial to respond to user inquiries and resolve problems quickly and effectively. However, conventional systems have limited automated response capabilities, and the optimization of response measures based on user emotions is insufficient. Furthermore, because inquiries are in natural language, misinterpretations and delays in response can occur. This leads to challenges such as decreased user satisfaction and reduced operational efficiency.

[0787] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0788] In this invention, the server includes means for receiving inquiry information, means for analyzing the received inquiry to identify a problem, means for analyzing user sentiment information, means for optimizing automated response measures based on the analyzed sentiment information, means for determining automated response measures for the problem, means for executing the determined response measures, means for monitoring the system state after execution, and means for escalating to a human operator as necessary. This enables automated inquiry handling and highly accurate problem solving that takes user sentiment into consideration.

[0789] "Means for receiving inquiry information" refers to devices or programs that have the function of receiving data related to inquiries and problems sent from users to the system.

[0790] "Means for analyzing received inquiries and identifying problems" refers to devices or programs that process data based on received inquiry information and have the function of identifying specific problems.

[0791] "Means for determining automated countermeasures" refers to devices or programs that have the function of selecting the optimal countermeasure for an identified problem and formulating a plan for its implementation.

[0792] "Means for implementing the decided countermeasures" refers to devices or programs that have the function of actually putting the decided countermeasures into action and carrying out the set processes.

[0793] "Means for monitoring the system state after execution" refers to devices or programs that have the function of observing the system status after the implementation of countermeasures and confirming the degree of improvement of the problem.

[0794] "Means of escalating to operations personnel" refers to devices or programs that, when a problem cannot be resolved through automated means, have the function of forwarding detailed information to a human administrator and requesting further action.

[0795] "Means for analyzing user emotional information" refers to devices or programs that have the function of processing and analyzing the emotions contained in information received from users in order to understand their content.

[0796] "Means for optimizing automated response measures based on analyzed emotional information" refers to devices or programs that have the function of selecting or adjusting the optimal response measure while taking into account the user's emotional data.

[0797] The system implementing this invention consists of a server, a terminal, and a user. The server receives query information and performs analysis using an AI agent and an emotion engine. Specifically, the server interprets the query information sent by the user using natural language processing techniques to identify the problem. Python and TensorFlow are used as the main software environments.

[0798] After analysis, the server uses an emotion engine to evaluate the emotional information contained in the user's input. At this time, data analysis that takes the user's emotions into account is performed, and the optimal automated response to the problem is determined. For example, if a user inputs an expression of frustration such as "payment not completed," the AI ​​agent will use that information to identify the cause and consider whether to reconnect to the service.

[0799] The selected countermeasures are automatically implemented, and the system's improvement status is monitored. If automatic resolution is difficult, detailed information is escalated to the operations team. The server has a function to provide feedback on problem resolution in a way that also takes user feelings into consideration.

[0800] As a concrete example, consider a case where a user reports that "a payment failed, but the charge is still showing up in my account." The server immediately identifies the problem, automates the appropriate action, and sends reassuring feedback to the user saying, "The problem has been resolved. Please try again."

[0801] An example of a prompt using a generative AI model is: "Analyze the user's frustration regarding the payment failure and suggest a solution." This allows the server to perform advanced sentiment analysis and improve the accuracy of automated responses.

[0802] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0803] Step 1:

[0804] The user enters query information using a terminal. This input includes a description of the problem and expressions of sentiment in natural language. The terminal sends this information to the server, which receives the input data. Here, the input is the text data entered by the user, and the output is the query data transferred to the server.

[0805] Step 2:

[0806] The server analyzes the received query data using natural language processing techniques (generative AI models using Python and TensorFlow). The input is query data, which the server analyzes as text to identify the problem. The output of this process is the identified problem and the keywords and information that comprise it. Specific operations include word tokenization and sentiment extraction.

[0807] Step 3:

[0808] The server uses an emotion engine to analyze the user's emotional information. The input at this stage is the user's original text data and analyzed keywords, while the output is information about the type and intensity of the emotion. Based on this, the server evaluates the user's emotional state and feeds this information back to an AI model that generates emotional information. Specific actions include applying emotion analysis algorithms to classify the emotions.

[0809] Step 4:

[0810] The server determines an automated response based on the identified problem and analyzed sentiment information. The input consists of the problem identification information and sentiment information obtained in the preceding steps. The server processes this information and selects an appropriate response. The output is the selected response. Specific operations include referencing an internal database and analyzing similar past cases to determine the appropriate response.

[0811] Step 5:

[0812] The server executes the chosen countermeasure. The input here is the automatically selected countermeasure, and the output is the result of the countermeasure's execution and the change in system state. The server attempts to resolve the problem by calling relevant APIs or launching internal processes. Specific actions may include system restarts or database modifications.

[0813] Step 6:

[0814] The server monitors the system status after execution and checks for improvements. The input is system log data after execution, and the output is the evaluation result of the system's health or abnormality. If the server detects an anomaly during monitoring, it further analyzes it and takes additional actions as needed. Specific actions include inspecting real-time logs and evaluating performance metrics.

[0815] Step 7:

[0816] The system sends feedback to the user and reports on the status of problem resolution. Inputs include an evaluation of the improvement status and the user's emotional information, while output is a feedback message that takes emotional information into consideration. The server sends the feedback and provides a specific and heartfelt message to reassure the user.

[0817] Step 8:

[0818] If the issue cannot be resolved, the server escalates detailed information, including the results of the sentiment engine, to the operations team. Inputs are unresolved issue information and sentiment information, while output is a detailed escalation report received by the operations team. The server generates a detailed report and forwards it to the appropriate person to facilitate a quick human response. Specific actions include organizing detailed log files and automatically generating reports.

[0819] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0820] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0821] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0822] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0823] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0824] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0825] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0826] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0827] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0828] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0829] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0830] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0831] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0832] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0833] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0834] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0835] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0836] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0837] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0838] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0839] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0840] The following is further disclosed regarding the embodiments described above.

[0841] (Claim 1)

[0842] Means for receiving inquiry information,

[0843] A means of analyzing received inquiries to identify problems,

[0844] A means of determining automated countermeasures for a problem,

[0845] The means to implement the decided countermeasures,

[0846] A means of monitoring the system state after execution,

[0847] A means to escalate the issue to a human operations manager as needed,

[0848] A system that includes this.

[0849] (Claim 2)

[0850] The system according to claim 1, comprising means for analyzing inquiry information in natural language.

[0851] (Claim 3)

[0852] The system according to claim 1, further comprising means for reporting to the user the improvements to the system resulting from the implemented countermeasures.

[0853] "Example 1"

[0854] (Claim 1)

[0855] Means of obtaining inquiry information,

[0856] A means of identifying problems by analyzing the acquired queries,

[0857] A means of determining an automated solution based on the identified problem,

[0858] The means to implement the decided solution,

[0859] A means of checking the operating status of the system after execution,

[0860] If necessary, a means to notify the operations staff of the problem,

[0861] Information processing device including

[0862] (Claim 2)

[0863] The information processing apparatus according to claim 1, comprising means for analyzing query information using natural language.

[0864] (Claim 3)

[0865] The information processing apparatus according to claim 1, further comprising means for notifying the user of system improvements resulting from implemented solutions.

[0866] "Application Example 1"

[0867] (Claim 1)

[0868] Means for receiving inquiry information,

[0869] A means of analyzing received inquiries to identify problems,

[0870] A means of determining automated countermeasures for a problem,

[0871] The means to implement the decided countermeasures,

[0872] A means of monitoring the system state after execution,

[0873] A means to escalate the issue to a human operations manager as needed,

[0874] A means of visualizing the operating status of the system in real time through physical sensors,

[0875] A means of providing visualized information to a display device,

[0876] A system that includes this.

[0877] (Claim 2)

[0878] The system according to claim 1, comprising means for analyzing inquiry information in natural language.

[0879] (Claim 3)

[0880] The system according to claim 1, further comprising means for notifying the user of system improvements resulting from implemented countermeasures via a reporting device.

[0881] "Example 2 of combining an emotion engine"

[0882] (Claim 1)

[0883] Means for receiving inquiry information,

[0884] A method for identifying problems by analyzing received inquiries in natural language,

[0885] A means of recognizing the user's emotional state,

[0886] A means of determining automated response measures based on identified problems and emotional states,

[0887] The means to implement the decided countermeasures,

[0888] A means of monitoring the state after execution and confirming improvements,

[0889] A means of reporting improvements to users,

[0890] A means of escalating information to the operations manager as needed,

[0891] A system that includes this.

[0892] (Claim 2)

[0893] The system according to claim 1, comprising means for analyzing inquiry information in natural language and for recognizing emotional states.

[0894] (Claim 3)

[0895] The system according to claim 1, further comprising means for reporting the improvement status after the implementation of automated countermeasures to the user and for escalating the information to the operations manager as necessary.

[0896] "Application example 2 when combining with an emotional engine"

[0897] (Claim 1)

[0898] Means for receiving inquiry information,

[0899] A means of analyzing received inquiries to identify problems,

[0900] A means of determining automated countermeasures for a problem,

[0901] The means to implement the decided countermeasures,

[0902] A means of monitoring the system state after execution,

[0903] A means to escalate the issue to a human operations manager as needed,

[0904] A means of analyzing user sentiment information,

[0905] A means to optimize automated response measures based on analyzed emotional information,

[0906] A system that includes this.

[0907] (Claim 2)

[0908] The system according to claim 1, comprising means for analyzing inquiry information in natural language and means for analyzing user sentiment information.

[0909] (Claim 3)

[0910] The system according to claim 1, comprising means for reporting to the user the improvements to the system resulting from the implemented countermeasures, and means for optimizing feedback based on emotional information. [Explanation of Symbols]

[0911] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. Means for receiving inquiry information, A means of analyzing received inquiries to identify problems, A means of determining automated countermeasures for a problem, The means to implement the decided countermeasures, A means of monitoring the system state after execution, A means to escalate the issue to a human operations manager as needed, A means of visualizing the operating status of the system in real time through physical sensors, A means of providing visualized information to a display device, A system that includes this.

2. The system according to claim 1, comprising means for analyzing inquiry information in natural language.

3. The system according to claim 1, further comprising means for notifying the user of system improvements resulting from implemented countermeasures via a reporting device.

Images