system

Abstract

We provide the system. [Solution] A means of identifying similar cases by referring to a database of past failure cases, A method for analyzing the characteristics of a problem input using artificial intelligence and estimating the cause of the problem, A means of proposing specific solutions based on the estimated cause, A means of determining whether on-site support is necessary, A means of registering follow-up data and updating the learning model, A means of reporting problems and providing solutions using a mobile terminal for store operations, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In dealing with malfunctions of in-store equipment, it is difficult for inexperienced front-line staff to respond quickly and accurately, and there is a problem that unnecessary on-site work due to incorrect judgments occurs. Also, there is a need for a system that can quickly distinguish the types and causes of malfunctions and present optimal solutions.

Means for Solving the Problems

[0005] The present invention solves the aforementioned problems by providing a system that includes means for identifying similar cases by referring to a database of past failure cases, means for analyzing the characteristics of an input problem using artificial intelligence and estimating the cause of the problem, means for presenting specific solutions based on the estimated cause, means for determining whether on-site response is necessary, and means for registering follow-up data and updating a learning model.

[0006] The "past failure case database" is a database that records past malfunctions in store equipment and serves as a source of information to be referenced when similar problems occur.

[0007] "Artificial intelligence" is a technology in which computer programs simulate human intelligence, possessing the ability to analyze input data and estimate the cause of a problem.

[0008] "Natural language processing" is a technology that enables computers to understand, interpret, and generate natural language used by humans in everyday life, and is used to analyze user input information.

[0009] "On-site support" refers to a method of resolving a problem by having a specialist technician go to the site and perform the work directly.

[0010] "Follow-up data" refers to data collected after a problem has been resolved, and includes the results of the response and user feedback.

[0011] A "learning model" is a structure or algorithm of artificial intelligence that learns patterns based on past data and makes predictions and inferences about new data. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, a communication I / F (Interface) with a reference numeral is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] The system of the present invention is designed to efficiently and accurately address malfunctions in store equipment. This system processes malfunctions through multiple stages, as described below.

[0034] First, when a user detects a malfunction in a store's equipment, they report the problem to the server using a dedicated interface. The report may include details of the problem, the date and time it occurred, the type of equipment, and possibly photos or screenshots of error messages.

[0035] When problem information is submitted, the server uses an AI analysis module to analyze the received data and investigate similar problems by referring to a database of past failure cases. Here, the artificial intelligence uses natural language processing to analyze the characteristics of the input problem and estimate its cause.

[0036] The server provides the user with specific solutions based on the cause of the problem estimated by the AI. For example, if it estimates that there is a paper jam in the printer, it will provide the user with instructions such as, "Open the printer cover and remove the jammed paper."

[0037] Furthermore, the server determines whether on-site support is required to resolve the issue. If on-site support is not deemed necessary, it prioritizes providing the user with remotely resolvable solutions. If on-site support is required, a notification will be sent to dispatch a technician at the optimal time.

[0038] Finally, once the problem is resolved, the server records follow-up data and updates the learning model based on it. In this way, the system continuously improves the accuracy of future problem-solving.

[0039] For example, if a user reports that "the cash register won't work even when powered on," the server will analyze the issue and instruct the user to "please check that the power cable is securely connected." This helps avoid unnecessary on-site visits and allows for efficient problem solving.

[0040] The following describes the processing flow.

[0041] Step 1:

[0042] If a user detects a problem with a store device, they first access the reporting interface using a terminal. Here, they enter information such as the details of the problem, the date, the type of device, photos, and error messages, and then send this information to the server.

[0043] Step 2:

[0044] The server accesses a database of past failure cases based on the information received from the user and searches for similar problems. During this process, the data is sent to an AI analysis module, where the characteristics of the problem are analyzed.

[0045] Step 3:

[0046] The server uses artificial intelligence and natural language processing to analyze user input in detail and identify the root cause of the problem. The estimated causes are listed as several candidates and ranked based on their confidence level.

[0047] Step 4:

[0048] The server presents the user with specific solutions based on the cause estimated by the AI. The solutions are generated as easy-to-understand instructions, which the user follows step by step.

[0049] Step 5:

[0050] The server uses AI to assess whether on-site support is needed to resolve the issue. Based on the assessment, if on-site support is not required, the user is prioritized to receive a remote solution.

[0051] Step 6:

[0052] The user tries the solution provided by the server and checks if the problem is resolved. The user reports the result to the server and provides feedback on the effectiveness of the solution.

[0053] Step 7:

[0054] The server records follow-up data based on user feedback. This data is used to update the learning model and improve the accuracy of future responses.

[0055] (Example 1)

[0056] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0057] Traditional troubleshooting systems often require significant time and effort to efficiently and effectively identify and resolve equipment problems. In particular, when numerous equipment malfunctions occur, delays can occur in determining whether on-site support is necessary and in arranging the dispatch of technicians. Furthermore, inaccurate estimation of the root cause of the problem can lead to unnecessary dispatches. This results in wasted time and costs, and a decline in the quality of customer service.

[0058] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0059] In this invention, the server includes means for referring to a collection of past cases and identifying similar cases, means for analyzing the characteristics of the input problem using an artificial intelligence processing unit and estimating the cause of the problem, and means for receiving problem reports and storing them in a database. This enables rapid and accurate fault diagnosis and countermeasures.

[0060] A "case study collection" is a database that compiles records of failures and problems that have occurred in the past, and is used to identify similar cases.

[0061] The "artificial intelligence processing unit" is a mechanism that uses machine learning and natural language processing technologies to analyze input data, understand the characteristics of a problem, and estimate its cause.

[0062] "Problem characteristics" refer to information that characterizes a problem, such as the details and circumstances of the malfunction, and are essential elements for estimating the cause and proposing solutions.

[0063] "On-site response" refers to a technician directly visiting the location of the equipment experiencing a problem to resolve the issue.

[0064] "Tracking data" refers to data that records information about the problem-solving process and results, and is used to update the system's learning structure.

[0065] A "learning structure" is a collection of models and algorithms that learn from collected data to improve the accuracy of estimating the cause of a problem and suggesting solutions.

[0066] "Remote support instructions" refer to providing users with specific procedures and instructions to resolve problems without sending technicians to the site.

[0067] "Arranging the dispatch of engineers" refers to the process of sending engineers with the necessary expertise to the site at the optimal time when needed to solve a problem.

[0068] This system is designed to efficiently handle malfunctions in store equipment. The following describes a specific embodiment of this invention.

[0069] If a user experiences a malfunction with a storefront device, they can report the problem to the server through a dedicated interface. This interface includes a smartphone application and a web portal. When reporting a problem, users can attach detailed descriptions of the issue, photos, and screenshots of error messages.

[0070] When the server receives a reported problem, it stores the information in a database. Next, the server uses a generative AI model to analyze the characteristics of the problem. This AI model employs natural language processing techniques to extract the features of the problem and estimate its cause. In this process, it is possible to quickly and accurately estimate the cause by referring to a collection of past cases and searching for similar cases.

[0071] Once the problem analysis is complete, the server will provide the user with specific solutions. For example, in response to a report that "the printer is not printing," the server will provide instructions such as "please ensure that paper is loaded in the printer tray."

[0072] Furthermore, the server determines whether on-site support is required. If on-site support is not needed, the server prioritizes suggesting remote solutions to the user. If necessary, it arranges for a technician to be dispatched at the optimal time. Through this entire process, the server can efficiently resolve problems.

[0073] For example, if a user reports that "the cash register won't start," the server uses its AI model to suggest a solution such as "please check that the power cable is securely connected." Once the problem is resolved, the server collects follow-up data and updates its learning structure.

[0074] An example of a prompt message could be: "According to a user report, the cash register is not working even when powered on. Based on this situation, use the AI ​​analysis module to estimate the cause of the problem and provide a specific solution to the user." This will enable the system to perform more accurate analysis in future problem handling.

[0075] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0076] Step 1:

[0077] When a user discovers a problem with a storefront device, they report the issue to the server using a dedicated interface. Input includes details of the problem, the date and time it occurred, the type of device, and screenshots of photos or error messages. The server receives the problem report data once the user enters this information into the interface.

[0078] Step 2:

[0079] When the server receives a problem report from a user, it saves the entered data to a database. This data is used to clearly record the problem situation at the time of the report. The server verifies the integrity of the input data and converts its format as needed before storing it.

[0080] Step 3:

[0081] The server sends the problem data to the AI ​​analysis module and begins the analysis. Here, a generative AI model is used to analyze the characteristics of the input problem. Specifically, natural language processing is used to extract features from the reported text and image data, and similar problems are searched for by comparing them with a set of past cases. The output is presented as the estimated cause of the problem.

[0082] Step 4:

[0083] The server presents specific solutions to the user based on the estimated causes set by the AI. It references a corresponding solution database based on the estimated causes as input and generates a solution procedure. For example, for the problem "the printer is not printing," it outputs the instruction "check that there is paper loaded in the printer tray" to the user.

[0084] Step 5:

[0085] The server verifies whether the problem has been resolved after implementing the provided solution. The user then provides feedback to the server indicating whether the problem has been resolved. If the problem is not resolved, the server will either suggest additional solutions or arrange for a technician to be dispatched. The output will show the status as resolved.

[0086] Step 6:

[0087] Once the problem is resolved, the server records follow-up data and updates the system's learning structure. This processes the collected data to provide more accurate estimations and solutions for future problems. User feedback and the effectiveness of the solutions are evaluated, and the output facilitates system improvements.

[0088] (Application Example 1)

[0089] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0090] Equipment malfunctions in stores directly impact customer service, requiring prompt and accurate responses. However, store staff often lack technical expertise, which can lead to delays in identifying the root cause of problems and implementing solutions. This can result in unnecessary on-site support and service delays, disrupting store operations. A system is needed to efficiently resolve these issues.

[0091] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0092] In this invention, the server includes means for referring to a database of past failure cases and identifying similar cases, means for analyzing the characteristics of the input problem using artificial intelligence and estimating the cause of the problem, and means for reporting the problem and suggesting solutions via a mobile terminal for store operations. This makes it possible for store staff to quickly report malfunctions and immediately check the solutions analyzed by the AI.

[0093] A "database of past failure cases" is an information aggregation system that records past failure cases to help solve similar problems.

[0094] "Artificial intelligence" is a technology in which computers imitate human intellectual activity and perform data analysis and decision-making.

[0095] "Means for estimating the cause of a problem" refers to a technical process for analyzing input data and predicting the cause of a specific problem.

[0096] A "means of providing solutions" refers to a function that provides users with specific instructions for resolving a problem based on the identified cause of the problem.

[0097] "Means for determining the necessity of on-site support" refers to a system that automatically determines whether or not on-site support by a technician is required.

[0098] "Means for registering follow-up data and updating the learning model" refers to a function that collects information even after a problem has been solved and continuously improves and updates the system's learning model based on that information.

[0099] A "store-use mobile terminal" is a portable information terminal that store staff carry and use while on duty.

[0100] This system combines various hardware and software components to efficiently resolve equipment malfunctions in stores. It utilizes mobile terminals for store operations, servers, and a database.

[0101] The server receives malfunction information transmitted from mobile terminals used for store operations via a communication module using Flask. This malfunction information may include symptoms of the failure and images taken from the terminal. Based on this information, the server uses a generative AI model implemented with TENSORFLOW® to analyze the characteristics of the problem using natural language processing technology. The analysis results are then compared with a database of past failure cases to identify problems with high similarity, and the server estimates the cause and generates a solution based on that information.

[0102] The solution is returned as text data to the mobile device, which the user (store staff) can then review. For example, when a user reports a problem with the POS system, the server may generate specific instructions such as, "Please ensure the power cable is securely connected."

[0103] In this process, the server collects follow-up data and continuously updates its learning model, thereby improving the accuracy of future problem-solving.

[0104] An example of a prompt message would be: "The POS system is not working. The power is on, but the screen remains black. Please suggest the cause of the problem and a solution, comparing it to past cases." This allows the artificial intelligence to efficiently analyze the problem and quickly provide a solution.

[0105] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0106] Step 1:

[0107] The terminal receives malfunction information entered by the user. This input includes a detailed description of the problem, the time it occurred, the type of equipment, and, if necessary, image data. The terminal then prepares to send this input information to the server.

[0108] Step 2:

[0109] The server receives the malfunction information sent from the terminal. As a preprocessing step for analysis, the received information undergoes data transformation to separate text data from image data. The text data of the malfunction information is then converted into a format suitable for natural language processing.

[0110] Step 3:

[0111] The server uses a generative AI model to analyze the text data of the problem using natural language processing. Specifically, it uses Hugging Face's Transformers to extract the characteristics of the problem and generates analysis results based on those characteristics. Based on the input text, it searches the database for similar past cases.

[0112] Step 4:

[0113] The server identifies similar failure cases from the database and estimates the cause of the problem based on the related information. It compares the failure information with past cases and performs data calculations to select the most similar solution.

[0114] Step 5:

[0115] The server develops a specific solution based on the suspected cause and outputs it as text. The solution includes operational steps that are considered effective for the problem. It then prepares to send this solution to the terminal.

[0116] Step 6:

[0117] The device receives the solution sent from the server. The user can view the solution on the device screen and attempt to resolve the problem by following the provided steps.

[0118] Step 7:

[0119] The user attempts to solve the problem according to the suggested solution and inputs the result into the terminal. The terminal then resends the result to the server, providing follow-up data on the problem resolution.

[0120] Step 8:

[0121] The server updates its learning model based on follow-up data received from the user. It incorporates this new information and performs further learning to improve the accuracy of future responses to similar problems.

[0122] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0123] This invention features a system for providing responses that take user emotions into account when dealing with malfunctions in retail equipment. This system not only provides quick and appropriate solutions based on past malfunction data, but also uses an emotion engine to analyze the user's emotional state and utilize the results in feedback.

[0124] Initially, when a user reports a problem, emotional data is collected from voice and text input along with the information entered through the device. The emotion engine uses natural language processing technology to analyze the user's emotional state from their writing style, tone of voice, and word choices during the report. For example, if anger or dissatisfaction is evident in the user's statements, that emotion is captured as data.

[0125] Next, the server analyzes the overall situation, including this sentiment data, and selects the optimal solution for the user. When a solution is presented, adjustments are made to take the user's feelings into consideration, and explanations are made carefully as needed. As a result, the user receives a service that is more acceptable and satisfying.

[0126] Furthermore, emotional data obtained during user interactions is logged and used to improve the overall system responsiveness. This allows the server to continuously improve its learning model so that it can provide more accurate, emotion-aware responses when addressing future issues.

[0127] For example, if a user is frustrated because the register isn't working, the server recognizes their emotion and offers a solution with empathetic language. By providing flexible services that respond to user emotions in this way, customer satisfaction is improved.

[0128] The following describes the processing flow.

[0129] Step 1:

[0130] If a user experiences a problem with a store device, they first report the issue using a terminal. The report includes details of the problem, the date and time, the type of device, and can be done via voice input or text message.

[0131] Step 2:

[0132] When the device sends input information to the server, the emotion engine analyzes the user's emotional state from the voice and text. Natural language processing technology is used to identify how emotions are expressed.

[0133] Step 3:

[0134] Based on the received problem report, the server consults a database of past failure cases to search for similar issues. It also considers the results of the emotion engine's analysis to generate an appropriate solution.

[0135] Step 4:

[0136] The server adjusts the tone of its responses and the level of detail in its explanations to the user based on the analyzed sentiment data. It then displays solutions on the device in a way that is sensitive to the user's emotions.

[0137] Step 5:

[0138] The user reviews the solutions provided by the server and attempts to resolve the problem by following the instructions. The server also responds to any subsequent changes in emotions and provides follow-up confirmation.

[0139] Step 6:

[0140] After a user reports a problem resolved, the server logs the result and sentiment data. This data can then be used to improve the learning model for future service enhancements.

[0141] (Example 2)

[0142] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0143] The present invention aims to provide prompt and appropriate solutions to malfunctions in retail equipment, while also considering the user's feelings. Existing systems focus solely on technical problem-solving, neglecting the user's emotional state, which sometimes leads to dissatisfaction. Therefore, there was a need to improve user satisfaction and provide more acceptable responses.

[0144] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0145] In this invention, the server includes means for referring to past failure case information and identifying similar cases, means for analyzing the characteristics of the input situation using an intelligent model and estimating the cause, and means for analyzing the user's emotional data and generating an emotionally sensitive response. This enables the presentation of appropriate solutions to the user's technical problems and a response that is sensitive to their emotions.

[0146] "Past malfunction case information" refers to data that integrates cases and records of equipment malfunctions that have occurred in the past.

[0147] "Means for identifying similar cases" refers to a function that finds cases with similar characteristics to the current problem based on past information.

[0148] An "intelligent model" is an artificial intelligence system that uses algorithms and machine learning techniques to analyze input information and make inferences.

[0149] "Means for analyzing characteristics and estimating causes" refers to an analytical function that uses detailed information about the input malfunction to identify its cause.

[0150] "Means of providing concrete solutions" refers to a system that shows users specific countermeasures and procedures based on the cause of the problem.

[0151] "User sentiment data" refers to information that indicates the user's emotional state, extracted from voice tone and text content.

[0152] "Means for generating emotionally sensitive responses" refers to a function that generates improved responses that are more empathetic to the user's emotions, based on the user's emotional data.

[0153] "A means of logging and updating the learning model" refers to a mechanism that saves collected data as history and uses that information for training to continuously improve the model's performance.

[0154] This invention provides a system for dealing with malfunctions in in-store equipment, and is particularly specialized in generating responses that take user emotions into consideration. This system is implemented in the following manner, with a server, terminal, and user working together.

[0155] First, if a user discovers a problem, they report it through the device. The device uses a speech recognition system and a text analysis system to convert the voice and text data collected from the user into a digital format. In particular, an emotion engine extracts emotional information from the user's voice tone and the language expressions used.

[0156] The server receives the collected data and analyzes it using an intelligent model. This intelligent model incorporates natural language processing technology and has the ability to estimate the cause of a problem from user input information and sentiment data. The server also refers to past problem cases and performs calculations to identify similar cases. Based on this process, the optimal solution is presented to the user.

[0157] The solutions presented are generated in a way that takes the user's emotions into consideration. Specifically, they are adjusted to include empathetic expressions and reassuring language. This response generation process uses a generation AI model, which is then presented to the user as a prompt. An example of a prompt is: "The user reported a problem: 'The cash register is not working.' The user's emotion includes 'anger.' Please generate an appropriate, empathetic response."

[0158] Furthermore, emotional data obtained from user interactions is logged and used to continuously update the server's learning model. This update will enable the server to provide more appropriate emotional responses in the future.

[0159] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0160] Step 1:

[0161] A user reports a problem. The user uses a device to report the device malfunction via voice or text. The device uses a generative AI model to convert the input voice into text and extract the user's sentiment data from word selection and context. The output is the problem report and sentiment data in text format.

[0162] Step 2:

[0163] The device transmits data. The device sends collected bug reports and sentiment data to the server. This data includes the user's statements, sentiment state, and related metadata. The output data is transmitted to the server via the communication network.

[0164] Step 3:

[0165] The server analyzes the data. The server analyzes the received data and uses a natural language processing model to extract characteristics of the malfunction. Next, an intelligent model compares this information with past malfunction case data to identify similar cases and estimate the most likely cause of the malfunction. The output is the estimated cause of the malfunction and a list of possible solutions.

[0166] Step 4:

[0167] The server generates an emotionally sensitive response. Based on estimated causes and user emotion data, the server uses a generative AI model to generate an appropriate and empathetic response. The response includes a solution along with expressions that acknowledge the user's feelings. The output is the response statement presented to the user.

[0168] Step 5:

[0169] The server sends a response. The server sends the generated response to the terminal and displays it to the user. The terminal shows the response to the user and prompts them to follow the instructed solution. The output of this step is the response message displayed on the user terminal.

[0170] Step 6:

[0171] The server records the feedback. The device collects user reactions and additional feedback and sends it to the server. The server uses this information to update its learning model and improve the accuracy of future responses. The output is the feedback data accumulated in the system.

[0172] (Application Example 2)

[0173] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0174] Traditional systems for reporting problems within stores often resulted in purely mechanical responses, making it difficult to consider customer emotions. This could lead to decreased customer satisfaction and damage the store's reputation. Therefore, providing prompt and emotionally sensitive responses when problems occur is crucial for improving customer satisfaction.

[0175] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0176] In this invention, the server includes means for referring to a database of past failure cases and identifying similar cases, means for analyzing user emotion data, and means for adjusting and presenting solutions based on emotion. This makes it possible to analyze the emotion of a customer when they report a problem in a store and provide a flexible response accordingly.

[0177] A "database of past failure cases" is a database that systematically collects and manages various failure and malfunction cases that have occurred in the past.

[0178] "Methods for identifying similar cases" refer to techniques and algorithms for selecting the case that most closely resembles the current problem from a vast database of failure cases.

[0179] "A means of analyzing the characteristics of a problem entered using artificial intelligence and estimating the cause of the problem" refers to a process that uses artificial intelligence technology to analyze the details of a problem entered by a user and identify its cause.

[0180] "Means of presenting concrete solutions" refers to technologies that propose feasible and effective solutions to users based on the identified causes of problems.

[0181] "An emotion analysis method for analyzing user emotion data" refers to a method or technology for analyzing an emotional state using natural language processing techniques based on voice or text information from a user.

[0182] "Means of adjusting and presenting solutions based on emotions" refers to technologies that take into account analyzed user emotional data and present solutions in an appropriate tone and content accordingly.

[0183] "Means for determining the necessity of on-site response" refers to the criteria and techniques for determining whether direct on-site response is necessary, depending on the urgency and progression of the problem.

[0184] "Methods for registering follow-up information and updating the learning model" refers to technologies that record customer feedback and results after a problem has been addressed as data, and use that data to improve the artificial intelligence learning model.

[0185] In order to implement this invention, it is necessary to specifically design a customer service support system for stores. In this system, various hardware and advanced software are used to coordinate a terminal that receives malfunction reports with a server that analyzes them.

[0186] The server first receives problem information entered by the user via the terminal. This information is provided in either voice or text format. Using the Google® Cloud Natural Language API, natural language processing is performed on the voice or text to extract customer sentiment data. This allows for analysis of the user's report and their emotional state at the time.

[0187] The server further references a database of past failure cases to identify instances highly similar to the reported malfunction. This data analysis employs complex algorithms to quickly and accurately pinpoint the root cause of the problem. Additionally, IBM Watson® Tone Analyzer is used to analyze the extracted sentiment data in detail. This enables the presentation of flexible solutions that take user emotions into consideration.

[0188] By using Azure® AI, the server generates the optimal solution based on analyzed situation and sentiment data, and presents it to the terminal. The generative AI model designs responses in an empathetic and appropriate tone, taking into account the user's emotions.

[0189] For example, if a customer reports a malfunction at a cash register in a store, the server can identify an angry tone from the voice input and quickly present a solution on the terminal such as, "We're sorry, please wait a moment while we check and resolve the issue." An example of a prompt to the generating AI model would be, "When a customer reports a cash register malfunction and expresses anger, please present a solution that conveys empathy and promptness."

[0190] By implementing this system, stores can maintain high-quality customer service while also enabling them to resolve problems quickly and improve customer satisfaction.

[0191] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0192] Step 1:

[0193] The user reports the problem to their device.

[0194] The input consists of voice or text information from the user, and this report is sent to the server as digital data by the terminal.

[0195] Step 2:

[0196] The server uses the Google Cloud Natural Language API to analyze the user's voice or text data.

[0197] The input is the user's voice or text information obtained in Step 1, and the output is the analyzed user's emotional state. The server grammatically analyzes the customer's utterances through natural language processing and simultaneously extracts emotional data.

[0198] Step 3:

[0199] The server refers to a database of past failure cases to identify cases similar to the current malfunction.

[0200] The input is the analyzed user utterance obtained in step 2, and the output is information on similar failure cases. The server searches the database and extracts the past case with the highest degree of match.

[0201] Step 4:

[0202] The server uses IBM Watson Tone Analyzer to perform a detailed analysis of the user's emotional data.

[0203] The input is the emotional data obtained in step 2, and the output is a detailed analysis of the user's emotional state. The server performs a multifaceted analysis of the emotional data to thoroughly evaluate the urgency and type of the customer's emotions.

[0204] Step 5:

[0205] The server uses Azure AI to generate the optimal solution based on the analysis results.

[0206] The input consists of the failure scenario from Step 3 and the sentiment analysis results from Step 4, while the output is the solution to be presented to the user. The server utilizes a generative AI model to design an appropriate solution in a tone that takes the user's emotions into consideration.

[0207] Step 6:

[0208] The terminal presents the solution received from the server to the user.

[0209] The input is the solution generated in step 5, and the output is the display or audio guidance of the solution to the user. The terminal provides the solution in a way that is easy for the user to understand, either through the screen or audio output.

[0210] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0211] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0212] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0213] [Second Embodiment]

[0214] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0215] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0216] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0217] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0218] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0219] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0220] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0221] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0222] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0223] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0224] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0225] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0226] The system of the present invention is designed to efficiently and accurately address malfunctions in store equipment. This system processes malfunctions through multiple stages, as described below.

[0227] First, when a user detects a malfunction in a store's equipment, they report the problem to the server using a dedicated interface. The report may include details of the problem, the date and time it occurred, the type of equipment, and possibly photos or screenshots of error messages.

[0228] When problem information is submitted, the server uses an AI analysis module to analyze the received data and investigate similar problems by referring to a database of past failure cases. Here, the artificial intelligence uses natural language processing to analyze the characteristics of the input problem and estimate its cause.

[0229] The server provides the user with specific solutions based on the cause of the problem estimated by the AI. For example, if it estimates that there is a paper jam in the printer, it will provide the user with instructions such as, "Open the printer cover and remove the jammed paper."

[0230] Furthermore, the server determines whether on-site support is required to resolve the issue. If on-site support is not deemed necessary, it prioritizes providing the user with remotely resolvable solutions. If on-site support is required, a notification will be sent to dispatch a technician at the optimal time.

[0231] Finally, once the problem is resolved, the server records follow-up data and updates the learning model based on it. In this way, the system continuously improves the accuracy of future problem-solving.

[0232] For example, if a user reports that "the cash register won't work even when powered on," the server will analyze the issue and instruct the user to "please check that the power cable is securely connected." This helps avoid unnecessary on-site visits and allows for efficient problem solving.

[0233] The following describes the processing flow.

[0234] Step 1:

[0235] If a user detects a problem with a store device, they first access the reporting interface using a terminal. Here, they enter information such as the details of the problem, the date, the type of device, photos, and error messages, and then send this information to the server.

[0236] Step 2:

[0237] The server accesses a database of past failure cases based on the information received from the user and searches for similar problems. During this process, the data is sent to an AI analysis module, where the characteristics of the problem are analyzed.

[0238] Step 3:

[0239] The server uses artificial intelligence and natural language processing to analyze user input in detail and identify the root cause of the problem. The estimated causes are listed as several candidates and ranked based on their confidence level.

[0240] Step 4:

[0241] The server presents the user with specific solutions based on the cause estimated by the AI. The solutions are generated as easy-to-understand instructions, which the user follows step by step.

[0242] Step 5:

[0243] The server uses AI to assess whether on-site support is needed to resolve the issue. Based on the assessment, if on-site support is not required, the user is prioritized to receive a remote solution.

[0244] Step 6:

[0245] The user tries the solution provided by the server and checks if the problem is resolved. The user reports the result to the server and provides feedback on the effectiveness of the solution.

[0246] Step 7:

[0247] The server records follow-up data based on user feedback. This data is used to update the learning model and improve the accuracy of future responses.

[0248] (Example 1)

[0249] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0250] Traditional troubleshooting systems often require significant time and effort to efficiently and effectively identify and resolve equipment problems. In particular, when numerous equipment malfunctions occur, delays can occur in determining whether on-site support is necessary and in arranging the dispatch of technicians. Furthermore, inaccurate estimation of the root cause of the problem can lead to unnecessary dispatches. This results in wasted time and costs, and a decline in the quality of customer service.

[0251] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0252] In this invention, the server includes means for referring to a collection of past cases and identifying similar cases, means for analyzing the characteristics of the input problem using an artificial intelligence processing unit and estimating the cause of the problem, and means for receiving problem reports and storing them in a database. This enables rapid and accurate fault diagnosis and countermeasures.

[0253] A "case study collection" is a database that compiles records of failures and problems that have occurred in the past, and is used to identify similar cases.

[0254] The "artificial intelligence processing unit" is a mechanism that uses machine learning and natural language processing technologies to analyze input data, understand the characteristics of a problem, and estimate its cause.

[0255] "Problem characteristics" refer to information that characterizes a problem, such as the details and circumstances of the malfunction, and are essential elements for estimating the cause and proposing solutions.

[0256] "On-site response" refers to a technician directly visiting the location of the equipment experiencing a problem to resolve the issue.

[0257] "Tracking data" refers to data that records information about the problem-solving process and results, and is used to update the system's learning structure.

[0258] A "learning structure" is a collection of models and algorithms that learn from collected data to improve the accuracy of estimating the cause of a problem and suggesting solutions.

[0259] "Remote support instructions" refer to providing users with specific procedures and instructions to resolve problems without sending technicians to the site.

[0260] "Arranging the dispatch of engineers" refers to the process of sending engineers with the necessary expertise to the site at the optimal time when needed to solve a problem.

[0261] This system is designed to efficiently handle malfunctions in store equipment. The following describes a specific embodiment of this invention.

[0262] If a user experiences a malfunction with a storefront device, they can report the problem to the server through a dedicated interface. This interface includes a smartphone application and a web portal. When reporting a problem, users can attach detailed descriptions of the issue, photos, and screenshots of error messages.

[0263] When the server receives a reported problem, it stores the information in a database. Next, the server uses a generative AI model to analyze the characteristics of the problem. This AI model employs natural language processing techniques to extract the features of the problem and estimate its cause. In this process, it is possible to quickly and accurately estimate the cause by referring to a collection of past cases and searching for similar cases.

[0264] Once the problem analysis is complete, the server will provide the user with specific solutions. For example, in response to a report that "the printer is not printing," the server will provide instructions such as "please ensure that paper is loaded in the printer tray."

[0265] Furthermore, the server determines whether on-site support is required. If on-site support is not needed, the server prioritizes suggesting remote solutions to the user. If necessary, it arranges for a technician to be dispatched at the optimal time. Through this entire process, the server can efficiently resolve problems.

[0266] For example, if a user reports that "the cash register won't start," the server uses its AI model to suggest a solution such as "please check that the power cable is securely connected." Once the problem is resolved, the server collects follow-up data and updates its learning structure.

[0267] An example of a prompt message could be: "According to a user report, the cash register is not working even when powered on. Based on this situation, use the AI ​​analysis module to estimate the cause of the problem and provide a specific solution to the user." This will enable the system to perform more accurate analysis in future problem handling.

[0268] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0269] Step 1:

[0270] When a user discovers a problem with a storefront device, they report the issue to the server using a dedicated interface. Input includes details of the problem, the date and time it occurred, the type of device, and screenshots of photos or error messages. The server receives the problem report data once the user enters this information into the interface.

[0271] Step 2:

[0272] When the server receives a problem report from a user, it saves the entered data to a database. This data is used to clearly record the problem situation at the time of the report. The server verifies the integrity of the input data and converts its format as needed before storing it.

[0273] Step 3:

[0274] The server sends the problem data to the AI ​​analysis module and begins the analysis. Here, a generative AI model is used to analyze the characteristics of the input problem. Specifically, natural language processing is used to extract features from the reported text and image data, and similar problems are searched for by comparing them with a set of past cases. The output is presented as the estimated cause of the problem.

[0275] Step 4:

[0276] The server presents specific solutions to the user based on the estimated causes set by the AI. It references a corresponding solution database based on the estimated causes as input and generates a solution procedure. For example, for the problem "the printer is not printing," it outputs the instruction "check that there is paper loaded in the printer tray" to the user.

[0277] Step 5:

[0278] The server verifies whether the problem has been resolved after implementing the provided solution. The user then provides feedback to the server indicating whether the problem has been resolved. If the problem is not resolved, the server will either suggest additional solutions or arrange for a technician to be dispatched. The output will show the status as resolved.

[0279] Step 6:

[0280] When the problem is solved, the server records the follow-up data and updates the learning structure of the system. As a result, based on the collected data, data processing is performed to provide more accurate estimations and solutions when the next problem occurs. The feedback from the user and the effectiveness of the solution are evaluated, and the improvement of the system is promoted as output.

[0281] (Application Example 1)

[0282] Next, Application Example 1 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart glasses 214 are referred to as the "terminal".

[0283] Since malfunctions of equipment in stores directly affect customer service, it is necessary to respond quickly and accurately. However, store staff often do not have technical expertise, and it may take time to identify the cause of the problem and execute the solution. This may result in wasted on-site responses and service delays, which may impede store operations. There is a need for a system to efficiently solve such problems.

[0284] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0285] In this invention, the server includes means for referring to a past failure case database to identify similar cases, means for analyzing the characteristics of the input problem using artificial intelligence to estimate the cause of the problem, and means for reporting problems and presenting solutions using a mobile terminal for store operations. As a result, it becomes possible for store staff to quickly report malfunctions and immediately confirm the solutions analyzed by AI.

[0286] The "past failure case database" is an information integration system that records past failure cases and is useful for solving similar problems.

[0287] "Artificial intelligence" is a technology in which a computer imitates human intellectual activities and performs data analysis and judgment.

[0288] "Means for estimating the cause of a problem" refers to a technical process for analyzing input data and predicting the cause of a specific problem.

[0289] A "means of providing solutions" refers to a function that provides users with specific instructions for resolving a problem based on the identified cause of the problem.

[0290] "Means for determining the necessity of on-site support" refers to a system that automatically determines whether or not on-site support by a technician is required.

[0291] "Means for registering follow-up data and updating the learning model" refers to a function that collects information even after a problem has been solved and continuously improves and updates the system's learning model based on that information.

[0292] A "store-use mobile terminal" is a portable information terminal that store staff carry and use while on duty.

[0293] This system combines various hardware and software components to efficiently resolve equipment malfunctions in stores. It utilizes mobile terminals for store operations, servers, and a database.

[0294] The server receives malfunction information transmitted from mobile terminals used for store operations via a communication module using Flask. This malfunction information may include symptoms of the failure and images taken from the terminal. Based on this information, the server uses a generative AI model implemented with TensorFlow to analyze the characteristics of the problem using natural language processing techniques. The analysis results are then compared with a database of past failure cases to identify problems with high similarity, and the server estimates the cause and generates a solution based on that information.

[0295] The solution is returned as text data to the mobile device, which the user (store staff) can then review. For example, when a user reports a problem with the POS system, the server may generate specific instructions such as, "Please ensure the power cable is securely connected."

[0296] In this process, the server collects follow-up data and continuously updates its learning model, thereby improving the accuracy of future problem-solving.

[0297] An example of a prompt message would be: "The POS system is not working. The power is on, but the screen remains black. Please suggest the cause of the problem and a solution, comparing it to past cases." This allows the artificial intelligence to efficiently analyze the problem and quickly provide a solution.

[0298] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0299] Step 1:

[0300] The terminal receives malfunction information entered by the user. This input includes a detailed description of the problem, the time it occurred, the type of equipment, and, if necessary, image data. The terminal then prepares to send this input information to the server.

[0301] Step 2:

[0302] The server receives the malfunction information sent from the terminal. As a preprocessing step for analysis, the received information undergoes data transformation to separate text data from image data. The text data of the malfunction information is then converted into a format suitable for natural language processing.

[0303] Step 3:

[0304] The server uses a generative AI model to analyze defective text data through natural language processing. Specifically, it utilizes Hugging Face's Transformers to extract the features of the problem and generate an analysis result based on those features. It searches the database for similar past cases based on the input text.

[0305] Step 4:

[0306] The server identifies similar failure cases from the database and estimates the cause of the problem based on the related information. It compares the defect information with past cases and performs data calculations to select the most similar solution.

[0307] Step 5:

[0308] The server formulates a specific solution based on the estimated cause and outputs it as text. The solution includes the operation procedures that are considered effective for the problem. It prepares to send this solution to the terminal.

[0309] Step 6:

[0310] The terminal receives the solution sent from the server. The user can view the solution through the terminal screen and attempt to solve the problem by following the presented procedures.

[0311] Step 7:

[0312] The user attempts to solve the problem according to the presented solution and inputs the result into the terminal. The terminal provides follow-up data for problem-solving by resending the result to the server.

[0313] Step 8:

[0314] The server updates the learning model based on the follow-up data received from the user. It performs learning for future utilization in solving similar problems, taking into account the new information. This improves the accuracy of future responses.

[0315] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0316] This invention features a system for providing responses that take user emotions into account when dealing with malfunctions in retail equipment. This system not only provides quick and appropriate solutions based on past malfunction data, but also uses an emotion engine to analyze the user's emotional state and utilize the results in feedback.

[0317] Initially, when a user reports a problem, emotional data is collected from voice and text input along with the information entered through the device. The emotion engine uses natural language processing technology to analyze the user's emotional state from their writing style, tone of voice, and word choices during the report. For example, if anger or dissatisfaction is evident in the user's statements, that emotion is captured as data.

[0318] Next, the server analyzes the overall situation, including this sentiment data, and selects the optimal solution for the user. When a solution is presented, adjustments are made to take the user's feelings into consideration, and explanations are made carefully as needed. As a result, the user receives a service that is more acceptable and satisfying.

[0319] Furthermore, emotional data obtained during user interactions is logged and used to improve the overall system responsiveness. This allows the server to continuously improve its learning model so that it can provide more accurate, emotion-aware responses when addressing future issues.

[0320] For example, if a user is frustrated because the register isn't working, the server recognizes their emotion and offers a solution with empathetic language. By providing flexible services that respond to user emotions in this way, customer satisfaction is improved.

[0321] The following describes the processing flow.

[0322] Step 1:

[0323] If a user experiences a problem with a store device, they first report the issue using a terminal. The report includes details of the problem, the date and time, the type of device, and can be done via voice input or text message.

[0324] Step 2:

[0325] When the device sends input information to the server, the emotion engine analyzes the user's emotional state from the voice and text. Natural language processing technology is used to identify how emotions are expressed.

[0326] Step 3:

[0327] Based on the received problem report, the server consults a database of past failure cases to search for similar issues. It also considers the results of the emotion engine's analysis to generate an appropriate solution.

[0328] Step 4:

[0329] The server adjusts the tone of its responses and the level of detail in its explanations to the user based on the analyzed sentiment data. It then displays solutions on the device in a way that is sensitive to the user's emotions.

[0330] Step 5:

[0331] The user reviews the solutions provided by the server and attempts to resolve the problem by following the instructions. The server also responds to any subsequent changes in emotions and provides follow-up confirmation.

[0332] Step 6:

[0333] After a user reports a problem resolved, the server logs the result and sentiment data. This data can then be used to improve the learning model for future service enhancements.

[0334] (Example 2)

[0335] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0336] The present invention aims to provide prompt and appropriate solutions to malfunctions in retail equipment, while also considering the user's feelings. Existing systems focus solely on technical problem-solving, neglecting the user's emotional state, which sometimes leads to dissatisfaction. Therefore, there was a need to improve user satisfaction and provide more acceptable responses.

[0337] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0338] In this invention, the server includes means for referring to past failure case information and identifying similar cases, means for analyzing the characteristics of the input situation using an intelligent model and estimating the cause, and means for analyzing the user's emotional data and generating an emotionally sensitive response. This enables the presentation of appropriate solutions to the user's technical problems and a response that is sensitive to their emotions.

[0339] "Past malfunction case information" refers to data that integrates cases and records of equipment malfunctions that have occurred in the past.

[0340] "Means for identifying similar cases" refers to a function that finds cases with similar characteristics to the current problem based on past information.

[0341] An "intelligent model" is an artificial intelligence system that uses algorithms and machine learning techniques to analyze input information and make inferences.

[0342] "Means for analyzing characteristics and estimating causes" refers to an analytical function that uses detailed information about the input malfunction to identify its cause.

[0343] "Means of providing concrete solutions" refers to a system that shows users specific countermeasures and procedures based on the cause of the problem.

[0344] "User sentiment data" refers to information that indicates the user's emotional state, extracted from voice tone and text content.

[0345] "Means for generating emotionally sensitive responses" refers to a function that generates improved responses that are more empathetic to the user's emotions, based on the user's emotional data.

[0346] "A means of logging and updating the learning model" refers to a mechanism that saves collected data as history and uses that information for training to continuously improve the model's performance.

[0347] This invention provides a system for dealing with malfunctions in in-store equipment, and is particularly specialized in generating responses that take user emotions into consideration. This system is implemented in the following manner, with a server, terminal, and user working together.

[0348] First, if a user discovers a problem, they report it through the device. The device uses a speech recognition system and a text analysis system to convert the voice and text data collected from the user into a digital format. In particular, an emotion engine extracts emotional information from the user's voice tone and the language expressions used.

[0349] The server receives the collected data and analyzes it using an intelligent model. This intelligent model incorporates natural language processing technology and has the ability to estimate the cause of a problem from user input information and sentiment data. The server also refers to past problem cases and performs calculations to identify similar cases. Based on this process, the optimal solution is presented to the user.

[0350] The solutions presented are generated in a way that takes the user's emotions into consideration. Specifically, they are adjusted to include empathetic expressions and reassuring language. This response generation process uses a generation AI model, which is then presented to the user as a prompt. An example of a prompt is: "The user reported a problem: 'The cash register is not working.' The user's emotion includes 'anger.' Please generate an appropriate, empathetic response."

[0351] Furthermore, emotional data obtained from user interactions is logged and used to continuously update the server's learning model. This update will enable the server to provide more appropriate emotional responses in the future.

[0352] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0353] Step 1:

[0354] A user reports a problem. The user uses a device to report the device malfunction via voice or text. The device uses a generative AI model to convert the input voice into text and extract the user's sentiment data from word selection and context. The output is the problem report and sentiment data in text format.

[0355] Step 2:

[0356] The device transmits data. The device sends collected bug reports and sentiment data to the server. This data includes the user's statements, sentiment state, and related metadata. The output data is transmitted to the server via the communication network.

[0357] Step 3:

[0358] The server analyzes the data. The server analyzes the received data and uses a natural language processing model to extract characteristics of the malfunction. Next, an intelligent model compares this information with past malfunction case data to identify similar cases and estimate the most likely cause of the malfunction. The output is the estimated cause of the malfunction and a list of possible solutions.

[0359] Step 4:

[0360] The server generates an emotionally sensitive response. Based on estimated causes and user emotion data, the server uses a generative AI model to generate an appropriate and empathetic response. The response includes a solution along with expressions that acknowledge the user's feelings. The output is the response statement presented to the user.

[0361] Step 5:

[0362] The server sends a response. The server sends the generated response to the terminal and displays it to the user. The terminal shows the response to the user and prompts them to follow the instructed solution. The output of this step is the response message displayed on the user terminal.

[0363] Step 6:

[0364] The server records the feedback. The device collects user reactions and additional feedback and sends it to the server. The server uses this information to update its learning model and improve the accuracy of future responses. The output is the feedback data accumulated in the system.

[0365] (Application Example 2)

[0366] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0367] Traditional systems for reporting problems within stores often resulted in purely mechanical responses, making it difficult to consider customer emotions. This could lead to decreased customer satisfaction and damage the store's reputation. Therefore, providing prompt and emotionally sensitive responses when problems occur is crucial for improving customer satisfaction.

[0368] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0369] In this invention, the server includes means for referring to a database of past failure cases and identifying similar cases, means for analyzing user emotion data, and means for adjusting and presenting solutions based on emotion. This makes it possible to analyze the emotion of a customer when they report a problem in a store and provide a flexible response accordingly.

[0370] A "database of past failure cases" is a database that systematically collects and manages various failure and malfunction cases that have occurred in the past.

[0371] "Methods for identifying similar cases" refer to techniques and algorithms for selecting the case that most closely resembles the current problem from a vast database of failure cases.

[0372] "A means of analyzing the characteristics of a problem entered using artificial intelligence and estimating the cause of the problem" refers to a process that uses artificial intelligence technology to analyze the details of a problem entered by a user and identify its cause.

[0373] "Means of presenting concrete solutions" refers to technologies that propose feasible and effective solutions to users based on the identified causes of problems.

[0374] "An emotion analysis method for analyzing user emotion data" refers to a method or technology for analyzing an emotional state using natural language processing techniques based on voice or text information from a user.

[0375] "Means of adjusting and presenting solutions based on emotions" refers to technologies that take into account analyzed user emotional data and present solutions in an appropriate tone and content accordingly.

[0376] "Means for determining the necessity of on-site response" refers to the criteria and techniques for determining whether direct on-site response is necessary, depending on the urgency and progression of the problem.

[0377] "Methods for registering follow-up information and updating the learning model" refers to technologies that record customer feedback and results after a problem has been addressed as data, and use that data to improve the artificial intelligence learning model.

[0378] In order to implement this invention, it is necessary to specifically design a customer service support system for stores. In this system, various hardware and advanced software are used to coordinate a terminal that receives malfunction reports with a server that analyzes them.

[0379] The server first receives problem information entered by the user via a terminal. This information is provided in either voice or text format. Using the Google Cloud Natural Language API, natural language processing is performed on the voice or text to extract customer sentiment data. This allows for analysis of the user's report and their emotional state at the time.

[0380] The server further consults a database of past failure cases to identify instances highly similar to the reported malfunction. This data analysis employs complex algorithms to quickly and accurately pinpoint the root cause of the problem. Additionally, IBM Watson Tone Analyzer is used to analyze the extracted sentiment data in detail. This enables the presentation of flexible solutions that take user emotions into consideration.

[0381] By using Azure AI, the server generates the optimal solution based on analyzed situation and sentiment data, and presents it to the device. The generative AI model designs responses in an empathetic and appropriate tone, taking into account the user's emotions.

[0382] For example, if a customer reports a malfunction at a cash register in a store, the server can identify an angry tone from the voice input and quickly present a solution on the terminal such as, "We're sorry, please wait a moment while we check and resolve the issue." An example of a prompt to the generating AI model would be, "When a customer reports a cash register malfunction and expresses anger, please present a solution that conveys empathy and promptness."

[0383] By implementing this system, stores can maintain high-quality customer service while also enabling them to resolve problems quickly and improve customer satisfaction.

[0384] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0385] Step 1:

[0386] The user reports the problem to their device.

[0387] The input consists of voice or text information from the user, and this report is sent to the server as digital data by the terminal.

[0388] Step 2:

[0389] The server uses the Google Cloud Natural Language API to analyze the user's voice or text data.

[0390] The input is the user's voice or text information obtained in Step 1, and the output is the analyzed user's emotional state. The server grammatically analyzes the customer's utterances through natural language processing and simultaneously extracts emotional data.

[0391] Step 3:

[0392] The server refers to a database of past failure cases to identify cases similar to the current malfunction.

[0393] The input is the analyzed user utterance obtained in step 2, and the output is information on similar failure cases. The server searches the database and extracts the past case with the highest degree of match.

[0394] Step 4:

[0395] The server uses IBM Watson Tone Analyzer to perform a detailed analysis of the user's emotional data.

[0396] The input is the emotional data obtained in step 2, and the output is a detailed analysis of the user's emotional state. The server performs a multifaceted analysis of the emotional data to thoroughly evaluate the urgency and type of the customer's emotions.

[0397] Step 5:

[0398] The server uses Azure AI to generate the optimal solution based on the analysis results.

[0399] The input consists of the failure scenario from Step 3 and the sentiment analysis results from Step 4, while the output is the solution to be presented to the user. The server utilizes a generative AI model to design an appropriate solution in a tone that takes the user's emotions into consideration.

[0400] Step 6:

[0401] The terminal presents the solution received from the server to the user.

[0402] The input is the solution generated in step 5, and the output is the display or audio guidance of the solution to the user. The terminal provides the solution in a way that is easy for the user to understand, either through the screen or audio output.

[0403] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0404] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0405] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0406] [Third Embodiment]

[0407] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0408] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0409] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0410] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0411] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0412] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0413] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0414] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0415] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0416] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0417] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0418] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0419] The system of the present invention is designed to efficiently and accurately address malfunctions in store equipment. This system processes malfunctions through multiple stages, as described below.

[0420] First, when a user detects a malfunction in a store's equipment, they report the problem to the server using a dedicated interface. The report may include details of the problem, the date and time it occurred, the type of equipment, and possibly photos or screenshots of error messages.

[0421] When problem information is submitted, the server uses an AI analysis module to analyze the received data and investigate similar problems by referring to a database of past failure cases. Here, the artificial intelligence uses natural language processing to analyze the characteristics of the input problem and estimate its cause.

[0422] The server provides the user with specific solutions based on the cause of the problem estimated by the AI. For example, if it estimates that there is a paper jam in the printer, it will provide the user with instructions such as, "Open the printer cover and remove the jammed paper."

[0423] Furthermore, the server determines whether on-site support is required to resolve the issue. If on-site support is not deemed necessary, it prioritizes providing the user with remotely resolvable solutions. If on-site support is required, a notification will be sent to dispatch a technician at the optimal time.

[0424] Finally, once the problem is resolved, the server records follow-up data and updates the learning model based on it. In this way, the system continuously improves the accuracy of future problem-solving.

[0425] For example, if a user reports that "the cash register won't work even when powered on," the server will analyze the issue and instruct the user to "please check that the power cable is securely connected." This helps avoid unnecessary on-site visits and allows for efficient problem solving.

[0426] The following describes the processing flow.

[0427] Step 1:

[0428] If a user detects a problem with a store device, they first access the reporting interface using a terminal. Here, they enter information such as the details of the problem, the date, the type of device, photos, and error messages, and then send this information to the server.

[0429] Step 2:

[0430] The server accesses a database of past failure cases based on the information received from the user and searches for similar problems. During this process, the data is sent to an AI analysis module, where the characteristics of the problem are analyzed.

[0431] Step 3:

[0432] The server uses artificial intelligence and natural language processing to analyze user input in detail and identify the root cause of the problem. The estimated causes are listed as several candidates and ranked based on their confidence level.

[0433] Step 4:

[0434] The server presents the user with specific solutions based on the cause estimated by the AI. The solutions are generated as easy-to-understand instructions, which the user follows step by step.

[0435] Step 5:

[0436] The server uses AI to assess whether on-site support is needed to resolve the issue. Based on the assessment, if on-site support is not required, the user is prioritized to receive a remote solution.

[0437] Step 6:

[0438] The user tries the solution provided by the server and checks if the problem is resolved. The user reports the result to the server and provides feedback on the effectiveness of the solution.

[0439] Step 7:

[0440] The server records follow-up data based on user feedback. This data is used to update the learning model and improve the accuracy of future responses.

[0441] (Example 1)

[0442] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0443] Traditional troubleshooting systems often require significant time and effort to efficiently and effectively identify and resolve equipment problems. In particular, when numerous equipment malfunctions occur, delays can occur in determining whether on-site support is necessary and in arranging the dispatch of technicians. Furthermore, inaccurate estimation of the root cause of the problem can lead to unnecessary dispatches. This results in wasted time and costs, and a decline in the quality of customer service.

[0444] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0445] In this invention, the server includes means for referring to a collection of past cases and identifying similar cases, means for analyzing the characteristics of the input problem using an artificial intelligence processing unit and estimating the cause of the problem, and means for receiving problem reports and storing them in a database. This enables rapid and accurate fault diagnosis and countermeasures.

[0446] A "case study collection" is a database that compiles records of failures and problems that have occurred in the past, and is used to identify similar cases.

[0447] The "artificial intelligence processing unit" is a mechanism that uses machine learning and natural language processing technologies to analyze input data, understand the characteristics of a problem, and estimate its cause.

[0448] "Problem characteristics" refer to information that characterizes a problem, such as the details and circumstances of the malfunction, and are essential elements for estimating the cause and proposing solutions.

[0449] "On-site response" refers to a technician directly visiting the location of the equipment experiencing a problem to resolve the issue.

[0450] "Tracking data" refers to data that records information about the problem-solving process and results, and is used to update the system's learning structure.

[0451] A "learning structure" is a collection of models and algorithms that learn from collected data to improve the accuracy of estimating the cause of a problem and suggesting solutions.

[0452] "Remote support instructions" refer to providing users with specific procedures and instructions to resolve problems without sending technicians to the site.

[0453] "Arranging the dispatch of engineers" refers to the process of sending engineers with the necessary expertise to the site at the optimal time when needed to solve a problem.

[0454] This system is designed to efficiently handle malfunctions in store equipment. The following describes a specific embodiment of this invention.

[0455] If a user experiences a malfunction with a storefront device, they can report the problem to the server through a dedicated interface. This interface includes a smartphone application and a web portal. When reporting a problem, users can attach detailed descriptions of the issue, photos, and screenshots of error messages.

[0456] When the server receives a reported problem, it stores the information in a database. Next, the server uses a generative AI model to analyze the characteristics of the problem. This AI model employs natural language processing techniques to extract the features of the problem and estimate its cause. In this process, it is possible to quickly and accurately estimate the cause by referring to a collection of past cases and searching for similar cases.

[0457] Once the problem analysis is complete, the server will provide the user with specific solutions. For example, in response to a report that "the printer is not printing," the server will provide instructions such as "please ensure that paper is loaded in the printer tray."

[0458] Furthermore, the server determines whether on-site support is required. If on-site support is not needed, the server prioritizes suggesting remote solutions to the user. If necessary, it arranges for a technician to be dispatched at the optimal time. Through this entire process, the server can efficiently resolve problems.

[0459] For example, if a user reports that "the cash register won't start," the server uses its AI model to suggest a solution such as "please check that the power cable is securely connected." Once the problem is resolved, the server collects follow-up data and updates its learning structure.

[0460] An example of a prompt message could be: "According to a user report, the cash register is not working even when powered on. Based on this situation, use the AI ​​analysis module to estimate the cause of the problem and provide a specific solution to the user." This will enable the system to perform more accurate analysis in future problem handling.

[0461] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0462] Step 1:

[0463] When a user discovers a problem with a storefront device, they report the issue to the server using a dedicated interface. Input includes details of the problem, the date and time it occurred, the type of device, and screenshots of photos or error messages. The server receives the problem report data once the user enters this information into the interface.

[0464] Step 2:

[0465] When the server receives a problem report from a user, it saves the entered data to a database. This data is used to clearly record the problem situation at the time of the report. The server verifies the integrity of the input data and converts its format as needed before storing it.

[0466] Step 3:

[0467] The server sends the problem data to the AI ​​analysis module and begins the analysis. Here, a generative AI model is used to analyze the characteristics of the input problem. Specifically, natural language processing is used to extract features from the reported text and image data, and similar problems are searched for by comparing them with a set of past cases. The output is presented as the estimated cause of the problem.

[0468] Step 4:

[0469] The server presents specific solutions to the user based on the estimated causes set by the AI. It references a corresponding solution database based on the estimated causes as input and generates a solution procedure. For example, for the problem "the printer is not printing," it outputs the instruction "check that there is paper loaded in the printer tray" to the user.

[0470] Step 5:

[0471] The server verifies whether the problem has been resolved after implementing the provided solution. The user then provides feedback to the server indicating whether the problem has been resolved. If the problem is not resolved, the server will either suggest additional solutions or arrange for a technician to be dispatched. The output will show the status as resolved.

[0472] Step 6:

[0473] Once the problem is resolved, the server records follow-up data and updates the system's learning structure. This processes the collected data to provide more accurate estimations and solutions for future problems. User feedback and the effectiveness of the solutions are evaluated, and the output facilitates system improvements.

[0474] (Application Example 1)

[0475] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0476] Equipment malfunctions in stores directly impact customer service, requiring prompt and accurate responses. However, store staff often lack technical expertise, which can lead to delays in identifying the root cause of problems and implementing solutions. This can result in unnecessary on-site support and service delays, disrupting store operations. A system is needed to efficiently resolve these issues.

[0477] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0478] In this invention, the server includes means for referring to a database of past failure cases and identifying similar cases, means for analyzing the characteristics of the input problem using artificial intelligence and estimating the cause of the problem, and means for reporting the problem and suggesting solutions via a mobile terminal for store operations. This makes it possible for store staff to quickly report malfunctions and immediately check the solutions analyzed by the AI.

[0479] A "database of past failure cases" is an information aggregation system that records past failure cases to help solve similar problems.

[0480] "Artificial intelligence" is a technology in which computers imitate human intellectual activity and perform data analysis and decision-making.

[0481] "Means for estimating the cause of a problem" refers to a technical process for analyzing input data and predicting the cause of a specific problem.

[0482] A "means of providing solutions" refers to a function that provides users with specific instructions for resolving a problem based on the identified cause of the problem.

[0483] "Means for determining the necessity of on-site support" refers to a system that automatically determines whether or not on-site support by a technician is required.

[0484] "Means for registering follow-up data and updating the learning model" refers to a function that collects information even after a problem has been solved and continuously improves and updates the system's learning model based on that information.

[0485] A "store-use mobile terminal" is a portable information terminal that store staff carry and use while on duty.

[0486] This system combines various hardware and software components to efficiently resolve equipment malfunctions in stores. It utilizes mobile terminals for store operations, servers, and a database.

[0487] The server receives malfunction information transmitted from mobile terminals used for store operations via a communication module using Flask. This malfunction information may include symptoms of the failure and images taken from the terminal. Based on this information, the server uses a generative AI model implemented with TensorFlow to analyze the characteristics of the problem using natural language processing techniques. The analysis results are then compared with a database of past failure cases to identify problems with high similarity, and the server estimates the cause and generates a solution based on that information.

[0488] The solution is returned as text data to the mobile device, which the user (store staff) can then review. For example, when a user reports a problem with the POS system, the server may generate specific instructions such as, "Please ensure the power cable is securely connected."

[0489] In this process, the server collects follow-up data and continuously updates its learning model, thereby improving the accuracy of future problem-solving.

[0490] An example of a prompt message would be: "The POS system is not working. The power is on, but the screen remains black. Please suggest the cause of the problem and a solution, comparing it to past cases." This allows the artificial intelligence to efficiently analyze the problem and quickly provide a solution.

[0491] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0492] Step 1:

[0493] The terminal receives malfunction information entered by the user. This input includes a detailed description of the problem, the time it occurred, the type of equipment, and, if necessary, image data. The terminal then prepares to send this input information to the server.

[0494] Step 2:

[0495] The server receives the malfunction information sent from the terminal. As a preprocessing step for analysis, the received information undergoes data transformation to separate text data from image data. The text data of the malfunction information is then converted into a format suitable for natural language processing.

[0496] Step 3:

[0497] The server uses a generative AI model to analyze the text data of the problem using natural language processing. Specifically, it uses Hugging Face's Transformers to extract the characteristics of the problem and generates analysis results based on those characteristics. Based on the input text, it searches the database for similar past cases.

[0498] Step 4:

[0499] The server identifies similar failure cases from the database and estimates the cause of the problem based on the related information. It compares the failure information with past cases and performs data calculations to select the most similar solution.

[0500] Step 5:

[0501] The server develops a specific solution based on the suspected cause and outputs it as text. The solution includes operational steps that are considered effective for the problem. It then prepares to send this solution to the terminal.

[0502] Step 6:

[0503] The device receives the solution sent from the server. The user can view the solution on the device screen and attempt to resolve the problem by following the provided steps.

[0504] Step 7:

[0505] The user attempts to solve the problem according to the suggested solution and inputs the result into the terminal. The terminal then resends the result to the server, providing follow-up data on the problem resolution.

[0506] Step 8:

[0507] The server updates its learning model based on follow-up data received from the user. It incorporates this new information and performs further learning to improve the accuracy of future responses to similar problems.

[0508] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0509] This invention features a system for providing responses that take user emotions into account when dealing with malfunctions in retail equipment. This system not only provides quick and appropriate solutions based on past malfunction data, but also uses an emotion engine to analyze the user's emotional state and utilize the results in feedback.

[0510] Initially, when a user reports a problem, emotional data is collected from voice and text input along with the information entered through the device. The emotion engine uses natural language processing technology to analyze the user's emotional state from their writing style, tone of voice, and word choices during the report. For example, if anger or dissatisfaction is evident in the user's statements, that emotion is captured as data.

[0511] Next, the server analyzes the overall situation, including this sentiment data, and selects the optimal solution for the user. When a solution is presented, adjustments are made to take the user's feelings into consideration, and explanations are made carefully as needed. As a result, the user receives a service that is more acceptable and satisfying.

[0512] Furthermore, emotional data obtained during user interactions is logged and used to improve the overall system responsiveness. This allows the server to continuously improve its learning model so that it can provide more accurate, emotion-aware responses when addressing future issues.

[0513] For example, if a user is frustrated because the register isn't working, the server recognizes their emotion and offers a solution with empathetic language. By providing flexible services that respond to user emotions in this way, customer satisfaction is improved.

[0514] The following describes the processing flow.

[0515] Step 1:

[0516] If a user experiences a problem with a store device, they first report the issue using a terminal. The report includes details of the problem, the date and time, the type of device, and can be done via voice input or text message.

[0517] Step 2:

[0518] When the device sends input information to the server, the emotion engine analyzes the user's emotional state from the voice and text. Natural language processing technology is used to identify how emotions are expressed.

[0519] Step 3:

[0520] Based on the received problem report, the server consults a database of past failure cases to search for similar issues. It also considers the results of the emotion engine's analysis to generate an appropriate solution.

[0521] Step 4:

[0522] The server adjusts the tone of its responses and the level of detail in its explanations to the user based on the analyzed sentiment data. It then displays solutions on the device in a way that is sensitive to the user's emotions.

[0523] Step 5:

[0524] The user reviews the solutions provided by the server and attempts to resolve the problem by following the instructions. The server also responds to any subsequent changes in emotions and provides follow-up confirmation.

[0525] Step 6:

[0526] After a user reports a problem resolved, the server logs the result and sentiment data. This data can then be used to improve the learning model for future service enhancements.

[0527] (Example 2)

[0528] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0529] The present invention aims to provide prompt and appropriate solutions to malfunctions in retail equipment, while also considering the user's feelings. Existing systems focus solely on technical problem-solving, neglecting the user's emotional state, which sometimes leads to dissatisfaction. Therefore, there was a need to improve user satisfaction and provide more acceptable responses.

[0530] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0531] In this invention, the server includes means for referring to past failure case information and identifying similar cases, means for analyzing the characteristics of the input situation using an intelligent model and estimating the cause, and means for analyzing the user's emotional data and generating an emotionally sensitive response. This enables the presentation of appropriate solutions to the user's technical problems and a response that is sensitive to their emotions.

[0532] "Past malfunction case information" refers to data that integrates cases and records of equipment malfunctions that have occurred in the past.

[0533] "Means for identifying similar cases" refers to a function that finds cases with similar characteristics to the current problem based on past information.

[0534] An "intelligent model" is an artificial intelligence system that uses algorithms and machine learning techniques to analyze input information and make inferences.

[0535] "Means for analyzing characteristics and estimating causes" refers to an analytical function that uses detailed information about the input malfunction to identify its cause.

[0536] "Means of providing concrete solutions" refers to a system that shows users specific countermeasures and procedures based on the cause of the problem.

[0537] "User sentiment data" refers to information that indicates the user's emotional state, extracted from voice tone and text content.

[0538] "Means for generating emotionally sensitive responses" refers to a function that generates improved responses that are more empathetic to the user's emotions, based on the user's emotional data.

[0539] "A means of logging and updating the learning model" refers to a mechanism that saves collected data as history and uses that information for training to continuously improve the model's performance.

[0540] This invention provides a system for dealing with malfunctions in in-store equipment, and is particularly specialized in generating responses that take user emotions into consideration. This system is implemented in the following manner, with a server, terminal, and user working together.

[0541] First, if a user discovers a problem, they report it through the device. The device uses a speech recognition system and a text analysis system to convert the voice and text data collected from the user into a digital format. In particular, an emotion engine extracts emotional information from the user's voice tone and the language expressions used.

[0542] The server receives the collected data and analyzes it using an intelligent model. This intelligent model incorporates natural language processing technology and has the ability to estimate the cause of a problem from user input information and sentiment data. The server also refers to past problem cases and performs calculations to identify similar cases. Based on this process, the optimal solution is presented to the user.

[0543] The solutions presented are generated in a way that takes the user's emotions into consideration. Specifically, they are adjusted to include empathetic expressions and reassuring language. This response generation process uses a generation AI model, which is then presented to the user as a prompt. An example of a prompt is: "The user reported a problem: 'The cash register is not working.' The user's emotion includes 'anger.' Please generate an appropriate, empathetic response."

[0544] Furthermore, emotional data obtained from user interactions is logged and used to continuously update the server's learning model. This update will enable the server to provide more appropriate emotional responses in the future.

[0545] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0546] Step 1:

[0547] A user reports a problem. The user uses a device to report the device malfunction via voice or text. The device uses a generative AI model to convert the input voice into text and extract the user's sentiment data from word selection and context. The output is the problem report and sentiment data in text format.

[0548] Step 2:

[0549] The device transmits data. The device sends collected bug reports and sentiment data to the server. This data includes the user's statements, sentiment state, and related metadata. The output data is transmitted to the server via the communication network.

[0550] Step 3:

[0551] The server analyzes the data. The server analyzes the received data and uses a natural language processing model to extract characteristics of the malfunction. Next, an intelligent model compares this information with past malfunction case data to identify similar cases and estimate the most likely cause of the malfunction. The output is the estimated cause of the malfunction and a list of possible solutions.

[0552] Step 4:

[0553] The server generates an emotionally sensitive response. Based on estimated causes and user emotion data, the server uses a generative AI model to generate an appropriate and empathetic response. The response includes a solution along with expressions that acknowledge the user's feelings. The output is the response statement presented to the user.

[0554] Step 5:

[0555] The server sends a response. The server sends the generated response to the terminal and displays it to the user. The terminal shows the response to the user and prompts them to follow the instructed solution. The output of this step is the response message displayed on the user terminal.

[0556] Step 6:

[0557] The server records the feedback. The device collects user reactions and additional feedback and sends it to the server. The server uses this information to update its learning model and improve the accuracy of future responses. The output is the feedback data accumulated in the system.

[0558] (Application Example 2)

[0559] Next, we will explain Application Example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0560] Traditional systems for reporting problems within stores often resulted in purely mechanical responses, making it difficult to consider customer emotions. This could lead to decreased customer satisfaction and damage the store's reputation. Therefore, providing prompt and emotionally sensitive responses when problems occur is crucial for improving customer satisfaction.

[0561] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0562] In this invention, the server includes means for referring to a database of past failure cases and identifying similar cases, means for analyzing user emotion data, and means for adjusting and presenting solutions based on emotion. This makes it possible to analyze the emotion of a customer when they report a problem in a store and provide a flexible response accordingly.

[0563] A "database of past failure cases" is a database that systematically collects and manages various failure and malfunction cases that have occurred in the past.

[0564] "Methods for identifying similar cases" refer to techniques and algorithms for selecting the case that most closely resembles the current problem from a vast database of failure cases.

[0565] "A means of analyzing the characteristics of a problem entered using artificial intelligence and estimating the cause of the problem" refers to a process that uses artificial intelligence technology to analyze the details of a problem entered by a user and identify its cause.

[0566] "Means of presenting concrete solutions" refers to technologies that propose feasible and effective solutions to users based on the identified causes of problems.

[0567] "An emotion analysis method for analyzing user emotion data" refers to a method or technology for analyzing an emotional state using natural language processing techniques based on voice or text information from a user.

[0568] "Means of adjusting and presenting solutions based on emotions" refers to technologies that take into account analyzed user emotional data and present solutions in an appropriate tone and content accordingly.

[0569] "Means for determining the necessity of on-site response" refers to the criteria and techniques for determining whether direct on-site response is necessary, depending on the urgency and progression of the problem.

[0570] "Methods for registering follow-up information and updating the learning model" refers to technologies that record customer feedback and results after a problem has been addressed as data, and use that data to improve the artificial intelligence learning model.

[0571] In order to implement this invention, it is necessary to specifically design a customer service support system for stores. In this system, various hardware and advanced software are used to coordinate a terminal that receives malfunction reports with a server that analyzes them.

[0572] The server first receives problem information entered by the user via a terminal. This information is provided in either voice or text format. Using the Google Cloud Natural Language API, natural language processing is performed on the voice or text to extract customer sentiment data. This allows for analysis of the user's report and their emotional state at the time.

[0573] The server further consults a database of past failure cases to identify instances highly similar to the reported malfunction. This data analysis employs complex algorithms to quickly and accurately pinpoint the root cause of the problem. Additionally, IBM Watson Tone Analyzer is used to analyze the extracted sentiment data in detail. This enables the presentation of flexible solutions that take user emotions into consideration.

[0574] By using Azure AI, the server generates the optimal solution based on analyzed situation and sentiment data, and presents it to the device. The generative AI model designs responses in an empathetic and appropriate tone, taking into account the user's emotions.

[0575] For example, if a customer reports a malfunction at a cash register in a store, the server can identify an angry tone from the voice input and quickly present a solution on the terminal such as, "We're sorry, please wait a moment while we check and resolve the issue." An example of a prompt to the generating AI model would be, "When a customer reports a cash register malfunction and expresses anger, please present a solution that conveys empathy and promptness."

[0576] By implementing this system, stores can maintain high-quality customer service while also enabling them to resolve problems quickly and improve customer satisfaction.

[0577] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0578] Step 1:

[0579] The user reports the problem to their device.

[0580] The input consists of voice or text information from the user, and this report is sent to the server as digital data by the terminal.

[0581] Step 2:

[0582] The server uses the Google Cloud Natural Language API to analyze the user's voice or text data.

[0583] The input is the user's voice or text information obtained in Step 1, and the output is the analyzed user's emotional state. The server grammatically analyzes the customer's utterances through natural language processing and simultaneously extracts emotional data.

[0584] Step 3:

[0585] The server refers to a database of past failure cases to identify cases similar to the current malfunction.

[0586] The input is the analyzed user utterance obtained in step 2, and the output is information on similar failure cases. The server searches the database and extracts the past case with the highest degree of match.

[0587] Step 4:

[0588] The server uses IBM Watson Tone Analyzer to perform a detailed analysis of the user's emotional data.

[0589] The input is the emotional data obtained in step 2, and the output is a detailed analysis of the user's emotional state. The server performs a multifaceted analysis of the emotional data to thoroughly evaluate the urgency and type of the customer's emotions.

[0590] Step 5:

[0591] The server uses Azure AI to generate the optimal solution based on the analysis results.

[0592] The input consists of the failure scenario from Step 3 and the sentiment analysis results from Step 4, while the output is the solution to be presented to the user. The server utilizes a generative AI model to design an appropriate solution in a tone that takes the user's emotions into consideration.

[0593] Step 6:

[0594] The terminal presents the solution received from the server to the user.

[0595] The input is the solution generated in step 5, and the output is the display or audio guidance of the solution to the user. The terminal provides the solution in a way that is easy for the user to understand, either through the screen or audio output.

[0596] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0597] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0598] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0599] [Fourth Embodiment]

[0600] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0601] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0602] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0603] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0604] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0605] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0606] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0607] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0608] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0609] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0610] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0611] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0612] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0613] The system of the present invention is designed to efficiently and accurately address malfunctions in store equipment. This system processes malfunctions through multiple stages, as described below.

[0614] First, when a user detects a malfunction in a store's equipment, they report the problem to the server using a dedicated interface. The report may include details of the problem, the date and time it occurred, the type of equipment, and possibly photos or screenshots of error messages.

[0615] When problem information is submitted, the server uses an AI analysis module to analyze the received data and investigate similar problems by referring to a database of past failure cases. Here, the artificial intelligence uses natural language processing to analyze the characteristics of the input problem and estimate its cause.

[0616] The server provides the user with specific solutions based on the cause of the problem estimated by the AI. For example, if it estimates that there is a paper jam in the printer, it will provide the user with instructions such as, "Open the printer cover and remove the jammed paper."

[0617] Furthermore, the server determines whether on-site support is required to resolve the issue. If on-site support is not deemed necessary, it prioritizes providing the user with remotely resolvable solutions. If on-site support is required, a notification will be sent to dispatch a technician at the optimal time.

[0618] Finally, once the problem is resolved, the server records follow-up data and updates the learning model based on it. In this way, the system continuously improves the accuracy of future problem-solving.

[0619] For example, if a user reports that "the cash register won't work even when powered on," the server will analyze the issue and instruct the user to "please check that the power cable is securely connected." This helps avoid unnecessary on-site visits and allows for efficient problem solving.

[0620] The following describes the processing flow.

[0621] Step 1:

[0622] If a user detects a problem with a store device, they first access the reporting interface using a terminal. Here, they enter information such as the details of the problem, the date, the type of device, photos, and error messages, and then send this information to the server.

[0623] Step 2:

[0624] The server accesses a database of past failure cases based on the information received from the user and searches for similar problems. During this process, the data is sent to an AI analysis module, where the characteristics of the problem are analyzed.

[0625] Step 3:

[0626] The server uses artificial intelligence and natural language processing to analyze user input in detail and identify the root cause of the problem. The estimated causes are listed as several candidates and ranked based on their confidence level.

[0627] Step 4:

[0628] The server presents the user with specific solutions based on the cause estimated by the AI. The solutions are generated as easy-to-understand instructions, which the user follows step by step.

[0629] Step 5:

[0630] The server uses AI to assess whether on-site support is needed to resolve the issue. Based on the assessment, if on-site support is not required, the user is prioritized to receive a remote solution.

[0631] Step 6:

[0632] The user tries the solution provided by the server and checks if the problem is resolved. The user reports the result to the server and provides feedback on the effectiveness of the solution.

[0633] Step 7:

[0634] The server records follow-up data based on user feedback. This data is used to update the learning model and improve the accuracy of future responses.

[0635] (Example 1)

[0636] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0637] Traditional troubleshooting systems often require significant time and effort to efficiently and effectively identify and resolve equipment problems. In particular, when numerous equipment malfunctions occur, delays can occur in determining whether on-site support is necessary and in arranging the dispatch of technicians. Furthermore, inaccurate estimation of the root cause of the problem can lead to unnecessary dispatches. This results in wasted time and costs, and a decline in the quality of customer service.

[0638] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0639] In this invention, the server includes means for referring to a collection of past cases and identifying similar cases, means for analyzing the characteristics of the input problem using an artificial intelligence processing unit and estimating the cause of the problem, and means for receiving problem reports and storing them in a database. This enables rapid and accurate fault diagnosis and countermeasures.

[0640] A "case study collection" is a database that compiles records of failures and problems that have occurred in the past, and is used to identify similar cases.

[0641] The "artificial intelligence processing unit" is a mechanism that uses machine learning and natural language processing technologies to analyze input data, understand the characteristics of a problem, and estimate its cause.

[0642] "Problem characteristics" refer to information that characterizes a problem, such as the details and circumstances of the malfunction, and are essential elements for estimating the cause and proposing solutions.

[0643] "On-site response" refers to a technician directly visiting the location of the equipment experiencing a problem to resolve the issue.

[0644] "Tracking data" refers to data that records information about the problem-solving process and results, and is used to update the system's learning structure.

[0645] A "learning structure" is a collection of models and algorithms that learn from collected data to improve the accuracy of estimating the cause of a problem and suggesting solutions.

[0646] "Remote support instructions" refer to providing users with specific procedures and instructions to resolve problems without sending technicians to the site.

[0647] "Arranging the dispatch of engineers" refers to the process of sending engineers with the necessary expertise to the site at the optimal time when needed to solve a problem.

[0648] This system is designed to efficiently handle malfunctions in store equipment. The following describes a specific embodiment of this invention.

[0649] If a user experiences a malfunction with a storefront device, they can report the problem to the server through a dedicated interface. This interface includes a smartphone application and a web portal. When reporting a problem, users can attach detailed descriptions of the issue, photos, and screenshots of error messages.

[0650] When the server receives a reported problem, it stores the information in a database. Next, the server uses a generative AI model to analyze the characteristics of the problem. This AI model employs natural language processing techniques to extract the features of the problem and estimate its cause. In this process, it is possible to quickly and accurately estimate the cause by referring to a collection of past cases and searching for similar cases.

[0651] Once the problem analysis is complete, the server will provide the user with specific solutions. For example, in response to a report that "the printer is not printing," the server will provide instructions such as "please ensure that paper is loaded in the printer tray."

[0652] Furthermore, the server determines whether on-site support is required. If on-site support is not needed, the server prioritizes suggesting remote solutions to the user. If necessary, it arranges for a technician to be dispatched at the optimal time. Through this entire process, the server can efficiently resolve problems.

[0653] For example, if a user reports that "the cash register won't start," the server uses its AI model to suggest a solution such as "please check that the power cable is securely connected." Once the problem is resolved, the server collects follow-up data and updates its learning structure.

[0654] An example of a prompt message could be: "According to a user report, the cash register is not working even when powered on. Based on this situation, use the AI ​​analysis module to estimate the cause of the problem and provide a specific solution to the user." This will enable the system to perform more accurate analysis in future problem handling.

[0655] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0656] Step 1:

[0657] When a user discovers a problem with a storefront device, they report the issue to the server using a dedicated interface. Input includes details of the problem, the date and time it occurred, the type of device, and screenshots of photos or error messages. The server receives the problem report data once the user enters this information into the interface.

[0658] Step 2:

[0659] When the server receives a problem report from a user, it saves the entered data to a database. This data is used to clearly record the problem situation at the time of the report. The server verifies the integrity of the input data and converts its format as needed before storing it.

[0660] Step 3:

[0661] The server sends the problem data to the AI ​​analysis module and begins the analysis. Here, a generative AI model is used to analyze the characteristics of the input problem. Specifically, natural language processing is used to extract features from the reported text and image data, and similar problems are searched for by comparing them with a set of past cases. The output is presented as the estimated cause of the problem.

[0662] Step 4:

[0663] The server presents specific solutions to the user based on the estimated causes set by the AI. It references a corresponding solution database based on the estimated causes as input and generates a solution procedure. For example, for the problem "the printer is not printing," it outputs the instruction "check that there is paper loaded in the printer tray" to the user.

[0664] Step 5:

[0665] The server verifies whether the problem has been resolved after implementing the provided solution. The user then provides feedback to the server indicating whether the problem has been resolved. If the problem is not resolved, the server will either suggest additional solutions or arrange for a technician to be dispatched. The output will show the status as resolved.

[0666] Step 6:

[0667] Once the problem is resolved, the server records follow-up data and updates the system's learning structure. This processes the collected data to provide more accurate estimations and solutions for future problems. User feedback and the effectiveness of the solutions are evaluated, and the output facilitates system improvements.

[0668] (Application Example 1)

[0669] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0670] Equipment malfunctions in stores directly impact customer service, requiring prompt and accurate responses. However, store staff often lack technical expertise, which can lead to delays in identifying the root cause of problems and implementing solutions. This can result in unnecessary on-site support and service delays, disrupting store operations. A system is needed to efficiently resolve these issues.

[0671] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0672] In this invention, the server includes means for referring to a database of past failure cases and identifying similar cases, means for analyzing the characteristics of the input problem using artificial intelligence and estimating the cause of the problem, and means for reporting the problem and suggesting solutions via a mobile terminal for store operations. This makes it possible for store staff to quickly report malfunctions and immediately check the solutions analyzed by the AI.

[0673] A "database of past failure cases" is an information aggregation system that records past failure cases to help solve similar problems.

[0674] "Artificial intelligence" is a technology in which computers imitate human intellectual activity and perform data analysis and decision-making.

[0675] "Means for estimating the cause of a problem" refers to a technical process for analyzing input data and predicting the cause of a specific problem.

[0676] A "means of providing solutions" refers to a function that provides users with specific instructions for resolving a problem based on the identified cause of the problem.

[0677] "Means for determining the necessity of on-site support" refers to a system that automatically determines whether or not on-site support by a technician is required.

[0678] "Means for registering follow-up data and updating the learning model" refers to a function that collects information even after a problem has been solved and continuously improves and updates the system's learning model based on that information.

[0679] A "store-use mobile terminal" is a portable information terminal that store staff carry and use while on duty.

[0680] This system combines various hardware and software components to efficiently resolve equipment malfunctions in stores. It utilizes mobile terminals for store operations, servers, and a database.

[0681] The server receives malfunction information transmitted from mobile terminals used for store operations via a communication module using Flask. This malfunction information may include symptoms of the failure and images taken from the terminal. Based on this information, the server uses a generative AI model implemented with TensorFlow to analyze the characteristics of the problem using natural language processing techniques. The analysis results are then compared with a database of past failure cases to identify problems with high similarity, and the server estimates the cause and generates a solution based on that information.

[0682] The solution is returned as text data to the mobile device, which the user (store staff) can then review. For example, when a user reports a problem with the POS system, the server may generate specific instructions such as, "Please ensure the power cable is securely connected."

[0683] In this process, the server collects follow-up data and continuously updates its learning model, thereby improving the accuracy of future problem-solving.

[0684] An example of a prompt message would be: "The POS system is not working. The power is on, but the screen remains black. Please suggest the cause of the problem and a solution, comparing it to past cases." This allows the artificial intelligence to efficiently analyze the problem and quickly provide a solution.

[0685] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0686] Step 1:

[0687] The terminal receives malfunction information entered by the user. This input includes a detailed description of the problem, the time it occurred, the type of equipment, and, if necessary, image data. The terminal then prepares to send this input information to the server.

[0688] Step 2:

[0689] The server receives the malfunction information sent from the terminal. As a preprocessing step for analysis, the received information undergoes data transformation to separate text data from image data. The text data of the malfunction information is then converted into a format suitable for natural language processing.

[0690] Step 3:

[0691] The server uses a generative AI model to analyze the text data of the problem using natural language processing. Specifically, it uses Hugging Face's Transformers to extract the characteristics of the problem and generates analysis results based on those characteristics. Based on the input text, it searches the database for similar past cases.

[0692] Step 4:

[0693] The server identifies similar failure cases from the database and estimates the cause of the problem based on the related information. It compares the failure information with past cases and performs data calculations to select the most similar solution.

[0694] Step 5:

[0695] The server develops a specific solution based on the suspected cause and outputs it as text. The solution includes operational steps that are considered effective for the problem. It then prepares to send this solution to the terminal.

[0696] Step 6:

[0697] The device receives the solution sent from the server. The user can view the solution on the device screen and attempt to resolve the problem by following the provided steps.

[0698] Step 7:

[0699] The user attempts to solve the problem according to the suggested solution and inputs the result into the terminal. The terminal then resends the result to the server, providing follow-up data on the problem resolution.

[0700] Step 8:

[0701] The server updates its learning model based on follow-up data received from the user. It incorporates this new information and performs further learning to improve the accuracy of future responses to similar problems.

[0702] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0703] This invention features a system for providing responses that take user emotions into account when dealing with malfunctions in retail equipment. This system not only provides quick and appropriate solutions based on past malfunction data, but also uses an emotion engine to analyze the user's emotional state and utilize the results in feedback.

[0704] Initially, when a user reports a problem, emotional data is collected from voice and text input along with the information entered through the device. The emotion engine uses natural language processing technology to analyze the user's emotional state from their writing style, tone of voice, and word choices during the report. For example, if anger or dissatisfaction is evident in the user's statements, that emotion is captured as data.

[0705] Next, the server analyzes the overall situation, including this sentiment data, and selects the optimal solution for the user. When a solution is presented, adjustments are made to take the user's feelings into consideration, and explanations are made carefully as needed. As a result, the user receives a service that is more acceptable and satisfying.

[0706] Furthermore, emotional data obtained during user interactions is logged and used to improve the overall system responsiveness. This allows the server to continuously improve its learning model so that it can provide more accurate, emotion-aware responses when addressing future issues.

[0707] For example, if a user is frustrated because the register isn't working, the server recognizes their emotion and offers a solution with empathetic language. By providing flexible services that respond to user emotions in this way, customer satisfaction is improved.

[0708] The following describes the processing flow.

[0709] Step 1:

[0710] If a user experiences a problem with a store device, they first report the issue using a terminal. The report includes details of the problem, the date and time, the type of device, and can be done via voice input or text message.

[0711] Step 2:

[0712] When the device sends input information to the server, the emotion engine analyzes the user's emotional state from the voice and text. Natural language processing technology is used to identify how emotions are expressed.

[0713] Step 3:

[0714] Based on the received problem report, the server consults a database of past failure cases to search for similar issues. It also considers the results of the emotion engine's analysis to generate an appropriate solution.

[0715] Step 4:

[0716] The server adjusts the tone of its responses and the level of detail in its explanations to the user based on the analyzed sentiment data. It then displays solutions on the device in a way that is sensitive to the user's emotions.

[0717] Step 5:

[0718] The user reviews the solutions provided by the server and attempts to resolve the problem by following the instructions. The server also responds to any subsequent changes in emotions and provides follow-up confirmation.

[0719] Step 6:

[0720] After a user reports a problem resolved, the server logs the result and sentiment data. This data can then be used to improve the learning model for future service enhancements.

[0721] (Example 2)

[0722] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0723] The present invention aims to provide prompt and appropriate solutions to malfunctions in retail equipment, while also considering the user's feelings. Existing systems focus solely on technical problem-solving, neglecting the user's emotional state, which sometimes leads to dissatisfaction. Therefore, there was a need to improve user satisfaction and provide more acceptable responses.

[0724] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0725] In this invention, the server includes means for referring to past failure case information and identifying similar cases, means for analyzing the characteristics of the input situation using an intelligent model and estimating the cause, and means for analyzing the user's emotional data and generating an emotionally sensitive response. This enables the presentation of appropriate solutions to the user's technical problems and a response that is sensitive to their emotions.

[0726] "Past malfunction case information" refers to data that integrates cases and records of equipment malfunctions that have occurred in the past.

[0727] "Means for identifying similar cases" refers to a function that finds cases with similar characteristics to the current problem based on past information.

[0728] An "intelligent model" is an artificial intelligence system that uses algorithms and machine learning techniques to analyze input information and make inferences.

[0729] "Means for analyzing characteristics and estimating causes" refers to an analytical function that uses detailed information about the input malfunction to identify its cause.

[0730] "Means of providing concrete solutions" refers to a system that shows users specific countermeasures and procedures based on the cause of the problem.

[0731] "User sentiment data" refers to information that indicates the user's emotional state, extracted from voice tone and text content.

[0732] "Means for generating emotionally sensitive responses" refers to a function that generates improved responses that are more empathetic to the user's emotions, based on the user's emotional data.

[0733] "A means of logging and updating the learning model" refers to a mechanism that saves collected data as history and uses that information for training to continuously improve the model's performance.

[0734] This invention provides a system for dealing with malfunctions in in-store equipment, and is particularly specialized in generating responses that take user emotions into consideration. This system is implemented in the following manner, with a server, terminal, and user working together.

[0735] First, if a user discovers a problem, they report it through the device. The device uses a speech recognition system and a text analysis system to convert the voice and text data collected from the user into a digital format. In particular, an emotion engine extracts emotional information from the user's voice tone and the language expressions used.

[0736] The server receives the collected data and analyzes it using an intelligent model. This intelligent model incorporates natural language processing technology and has the ability to estimate the cause of a problem from user input information and sentiment data. The server also refers to past problem cases and performs calculations to identify similar cases. Based on this process, the optimal solution is presented to the user.

[0737] The solutions presented are generated in a way that takes the user's emotions into consideration. Specifically, they are adjusted to include empathetic expressions and reassuring language. This response generation process uses a generation AI model, which is then presented to the user as a prompt. An example of a prompt is: "The user reported a problem: 'The cash register is not working.' The user's emotion includes 'anger.' Please generate an appropriate, empathetic response."

[0738] Furthermore, emotional data obtained from user interactions is logged and used to continuously update the server's learning model. This update will enable the server to provide more appropriate emotional responses in the future.

[0739] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0740] Step 1:

[0741] A user reports a problem. The user uses a device to report the device malfunction via voice or text. The device uses a generative AI model to convert the input voice into text and extract the user's sentiment data from word selection and context. The output is the problem report and sentiment data in text format.

[0742] Step 2:

[0743] The device transmits data. The device sends collected bug reports and sentiment data to the server. This data includes the user's statements, sentiment state, and related metadata. The output data is transmitted to the server via the communication network.

[0744] Step 3:

[0745] The server analyzes the data. The server analyzes the received data and uses a natural language processing model to extract characteristics of the malfunction. Next, an intelligent model compares this information with past malfunction case data to identify similar cases and estimate the most likely cause of the malfunction. The output is the estimated cause of the malfunction and a list of possible solutions.

[0746] Step 4:

[0747] The server generates an emotionally sensitive response. Based on estimated causes and user emotion data, the server uses a generative AI model to generate an appropriate and empathetic response. The response includes a solution along with expressions that acknowledge the user's feelings. The output is the response statement presented to the user.

[0748] Step 5:

[0749] The server sends a response. The server sends the generated response to the terminal and displays it to the user. The terminal shows the response to the user and prompts them to follow the instructed solution. The output of this step is the response message displayed on the user terminal.

[0750] Step 6:

[0751] The server records the feedback. The device collects user reactions and additional feedback and sends it to the server. The server uses this information to update its learning model and improve the accuracy of future responses. The output is the feedback data accumulated in the system.

[0752] (Application Example 2)

[0753] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0754] Traditional systems for reporting problems within stores often resulted in purely mechanical responses, making it difficult to consider customer emotions. This could lead to decreased customer satisfaction and damage the store's reputation. Therefore, providing prompt and emotionally sensitive responses when problems occur is crucial for improving customer satisfaction.

[0755] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means.

[0756] In this invention, the server includes means for referring to a database of past failure cases and identifying similar cases, means for analyzing user emotion data, and means for adjusting and presenting solutions based on emotion. This makes it possible to analyze the emotion of a customer when they report a problem in a store and provide a flexible response accordingly.

[0757] A "database of past failure cases" is a database that systematically collects and manages various failure and malfunction cases that have occurred in the past.

[0758] "Methods for identifying similar cases" refer to techniques and algorithms for selecting the case that most closely resembles the current problem from a vast database of failure cases.

[0759] "A means of analyzing the characteristics of a problem entered using artificial intelligence and estimating the cause of the problem" refers to a process that uses artificial intelligence technology to analyze the details of a problem entered by a user and identify its cause.

[0760] "Means of presenting concrete solutions" refers to technologies that propose feasible and effective solutions to users based on the identified causes of problems.

[0761] "An emotion analysis method for analyzing user emotion data" refers to a method or technology for analyzing an emotional state using natural language processing techniques based on voice or text information from a user.

[0762] "Means of adjusting and presenting solutions based on emotions" refers to technologies that take into account analyzed user emotional data and present solutions in an appropriate tone and content accordingly.

[0763] "Means for determining the necessity of on-site response" refers to the criteria and techniques for determining whether direct on-site response is necessary, depending on the urgency and progression of the problem.

[0764] "Methods for registering follow-up information and updating the learning model" refers to technologies that record customer feedback and results after a problem has been addressed as data, and use that data to improve the artificial intelligence learning model.

[0765] In order to implement this invention, it is necessary to specifically design a customer service support system for stores. In this system, various hardware and advanced software are used to coordinate a terminal that receives malfunction reports with a server that analyzes them.

[0766] The server first receives problem information entered by the user via a terminal. This information is provided in either voice or text format. Using the Google Cloud Natural Language API, natural language processing is performed on the voice or text to extract customer sentiment data. This allows for analysis of the user's report and their emotional state at the time.

[0767] The server further consults a database of past failure cases to identify instances highly similar to the reported malfunction. This data analysis employs complex algorithms to quickly and accurately pinpoint the root cause of the problem. Additionally, IBM Watson Tone Analyzer is used to analyze the extracted sentiment data in detail. This enables the presentation of flexible solutions that take user emotions into consideration.

[0768] By using Azure AI, the server generates the optimal solution based on analyzed situation and sentiment data, and presents it to the device. The generative AI model designs responses in an empathetic and appropriate tone, taking into account the user's emotions.

[0769] For example, if a customer reports a malfunction at a cash register in a store, the server can identify an angry tone from the voice input and quickly present a solution on the terminal such as, "We're sorry, please wait a moment while we check and resolve the issue." An example of a prompt to the generating AI model would be, "When a customer reports a cash register malfunction and expresses anger, please present a solution that conveys empathy and promptness."

[0770] By implementing this system, stores can maintain high-quality customer service while also enabling them to resolve problems quickly and improve customer satisfaction.

[0771] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0772] Step 1:

[0773] The user reports the problem to their device.

[0774] The input consists of voice or text information from the user, and this report is sent to the server as digital data by the terminal.

[0775] Step 2:

[0776] The server uses the Google Cloud Natural Language API to analyze the user's voice or text data.

[0777] The input is the user's voice or text information obtained in Step 1, and the output is the analyzed user's emotional state. The server grammatically analyzes the customer's utterances through natural language processing and simultaneously extracts emotional data.

[0778] Step 3:

[0779] The server refers to a database of past failure cases to identify cases similar to the current malfunction.

[0780] The input is the analyzed user utterance obtained in step 2, and the output is information on similar failure cases. The server searches the database and extracts the past case with the highest degree of match.

[0781] Step 4:

[0782] The server uses IBM Watson Tone Analyzer to perform a detailed analysis of the user's emotional data.

[0783] The input is the emotional data obtained in step 2, and the output is a detailed analysis of the user's emotional state. The server performs a multifaceted analysis of the emotional data to thoroughly evaluate the urgency and type of the customer's emotions.

[0784] Step 5:

[0785] The server uses Azure AI to generate the optimal solution based on the analysis results.

[0786] The input consists of the failure scenario from Step 3 and the sentiment analysis results from Step 4, while the output is the solution to be presented to the user. The server utilizes a generative AI model to design an appropriate solution in a tone that takes the user's emotions into consideration.

[0787] Step 6:

[0788] The terminal presents the solution received from the server to the user.

[0789] The input is the solution generated in step 5, and the output is the display or audio guidance of the solution to the user. The terminal provides the solution in a way that is easy for the user to understand, either through the screen or audio output.

[0790] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0791] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0792] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[0793] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0794] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0795] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0796] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0797] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0798] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0799] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0800] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[0801] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[0802] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0803] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0804] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0805] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0806] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0807] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0808] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0809] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0810] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0811] The following is further disclosed regarding the embodiments described above.

[0812] (Claim 1)

[0813] A means of identifying similar cases by referring to a database of past failure cases,

[0814] A method for analyzing the characteristics of a problem input using artificial intelligence and estimating the cause of the problem,

[0815] A means of proposing specific solutions based on the estimated cause,

[0816] A means of determining whether on-site support is necessary,

[0817] A means of registering follow-up data and updating the learning model,

[0818] A system that includes this.

[0819] (Claim 2)

[0820] The system according to claim 1, comprising means for artificial intelligence to analyze user input information using natural language processing.

[0821] (Claim 3)

[0822] The system according to claim 1, comprising means for guiding the user by sequentially presenting the operation procedure.

[0823] "Example 1"

[0824] (Claim 1)

[0825] A means of identifying similar cases by referring to a collection of past cases,

[0826] A means of analyzing the characteristics of an input problem using an artificial intelligence processing unit and estimating the cause of the problem,

[0827] A means of proposing specific solutions based on the estimated cause,

[0828] A means of determining whether on-site action is necessary,

[0829] A means of registering tracking data and updating the learning structure,

[0830] A means of receiving and saving problem reports to a database,

[0831] A means of providing instructions to the user to respond remotely,

[0832] Means of arranging the dispatch of engineers,

[0833] A system that includes this.

[0834] (Claim 2)

[0835] The system according to claim 1, wherein the artificial intelligence processing unit includes means for analyzing user input information using natural language processing.

[0836] (Claim 3)

[0837] The system according to claim 1, comprising means for guiding the user by sequentially presenting operating procedures.

[0838] "Application Example 1"

[0839] (Claim 1)

[0840] A means of identifying similar cases by referring to a database of past failure cases,

[0841] A method for analyzing the characteristics of a problem input using artificial intelligence and estimating the cause of the problem,

[0842] A means of proposing specific solutions based on the estimated cause,

[0843] A means of determining whether on-site support is necessary,

[0844] A means of registering follow-up data and updating the learning model,

[0845] A means of reporting problems and providing solutions using a mobile terminal for store operations,

[0846] A system that includes this.

[0847] (Claim 2)

[0848] The system according to claim 1, comprising means for artificial intelligence to analyze user input information using natural language processing.

[0849] (Claim 3)

[0850] The system according to claim 1, comprising means for guiding the user by sequentially presenting the operation procedure.

[0851] "Example 2 of combining an emotion engine"

[0852] (Claim 1)

[0853] A means of identifying similar cases by referring to past failure case information,

[0854] A means of analyzing the characteristics of an input situation using an intelligent model and estimating its cause,

[0855] A means of proposing specific solutions based on the estimated cause,

[0856] A means for analyzing user emotion data and generating emotionally sensitive responses,

[0857] A means of logging collected emotional data and updating the learning model,

[0858] A system that includes this.

[0859] (Claim 2)

[0860] The system according to claim 1, wherein the intelligent model includes means for analyzing user input information using natural language technology.

[0861] (Claim 3)

[0862] The system according to claim 1, further comprising means for guiding the user by sequentially presenting the operating procedure.

[0863] "Application example 2 when combining with an emotional engine"

[0864] (Claim 1)

[0865] A means of identifying similar cases by referring to a database of past failure cases,

[0866] A method for analyzing the characteristics of a problem input using artificial intelligence and estimating the cause of the problem,

[0867] A means of proposing specific solutions based on the estimated cause,

[0868] A sentiment analysis method for analyzing user sentiment data,

[0869] A means of adjusting and presenting solutions based on emotions,

[0870] A means of determining whether on-site support is necessary,

[0871] A means of registering follow-up information and updating the learning model,

[0872] A system that includes this.

[0873] (Claim 2)

[0874] The system according to claim 1, comprising means for artificial intelligence to analyze user input information using natural language processing and analyze the emotional state.

[0875] (Claim 3)

[0876] The system according to claim 1, comprising means for guiding the user by sequentially presenting operating procedures and providing emotionally sensitive feedback. [Explanation of Symbols]

[0877] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of identifying similar cases by referring to a database of past failure cases, A method for analyzing the characteristics of a problem input using artificial intelligence and estimating the cause of the problem, A means of proposing specific solutions based on the estimated cause, A means of determining whether on-site support is necessary, A means of registering follow-up data and updating the learning model, A means of reporting problems and providing solutions using a mobile terminal for store operations, A system that includes this.

2. The system according to claim 1, comprising means for artificial intelligence to analyze user input information using natural language processing.

3. The system according to claim 1, comprising means for guiding the user by sequentially presenting the operation procedure.

Images