System

Abstract

A system is provided.SOLUTION: The system includes a means for receiving failure information input from a user, a means for analyzing the received failure information by natural language processing, a means for retrieving a past similar case from a database based on an analysis result, a means for calculating a probability of occurrence of a failure cause using statistical data based on a retrieval result, a means for estimating a failure cause based on the calculated probability, a means for generating an optimal solution based on the estimated failure cause, and a means for presenting the generated solution to a user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] For manufacturers and companies that sell their products, the time and cost required to identify the cause of a breakdown or malfunction is a major issue. Without specialized product knowledge, identifying the cause and narrowing down solutions can take time, leading to risks such as reduced customer satisfaction and lost sales opportunities. In these circumstances, there is a need for a system that can quickly identify the cause of a malfunction without specialized knowledge and suggest appropriate repair methods and replacement parts. [Means for solving the problem]

[0005] This invention provides a system that includes means for receiving defect information input by a user and analyzing it using natural language processing, means for searching a database for similar past cases based on the analysis results, means for calculating the probability of occurrence of a failure cause using statistical data from the search results, means for estimating the failure cause based on the calculated probability, means for generating an optimal solution based on the estimated failure cause, and means for presenting the generated solution to the user. This system allows the user to quickly identify the cause of the failure and obtain an appropriate repair method without requiring specialized knowledge. Furthermore, by including means for receiving the results of the user's execution of the presented solution and recording them in a database, and means for generating a list of replacement parts required for the solution and presenting it to the user, the accuracy and efficiency of repairs are further improved.

[0006] A "user" is an entity that uses the system to input defect information and receive analysis results and proposed solutions.

[0007] A "terminal" is a device that is operated by a user to input defect information into the system, and is a device that communicates data with the server.

[0008] "Problem information" refers to detailed information such as product problems, error messages, and the circumstances under which the problem occurred, which are entered into the system by the user via a terminal.

[0009] "Natural language processing" is a technology that allows a system to analyze input defect information and extract important keywords and patterns.

[0010] "Server" refers to the computer system that serves as the center of the system and receives, processes, analyzes, calculates statistics, searches the database, and manages the generation of proposals.

[0011] A "database" is a collection of information within a system that stores past failure cases, statistical data, solutions, feedback information, etc., and is used for searching.

[0012] "Statistical data" refers to data used by the system to estimate the cause of a failure, such as the probability of the failure cause occurring calculated based on similar past cases.

[0013] The "cause of failure" is the fundamental reason or factor behind a defect that occurs in a product, and is what the system estimates.

[0014] A "solution" is a specific repair procedure or countermeasure that the system generates based on the estimated cause of the failure.

[0015] "List of replacement parts" refers to a list of parts required for the solution, which the system presents to the user. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10]1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that allows a user to input product defect information, identifies the cause of the failure based on that information, and proposes an optimal solution. Specific embodiments of this system will be described below.

[0038] Overall system configuration

[0039] The system mainly consists of a user's terminal, a server, and a database. The user uses the terminal to input information about the problem, which is then sent to the server. The server processes the information, compares it with the database, deduces the cause of the problem, and generates a solution.

[0040] User Roles

[0041] The user mainly performs the following operations:

[0042] 1. Entering defect information: The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form.

[0043] 2. Confirmation of the solution: The user confirms the solution sent from the server on the device.

[0044] 3. Implementing the solution and providing feedback: The user carries out the repair or part replacement based on the proposed solution, and then enters the results into the terminal again and sends them to the server.

[0045] Server Roles

[0046] The server is the central part of the system and performs the following tasks:

[0047] 1. Data reception and analysis: The server receives the defect information sent from the user's device and analyzes it using a natural language processing engine. Important keywords and patterns are extracted to identify the problem.

[0048] 2. Database collation: Based on the analysis results, the results are collated with a database of past failures, similar cases are searched for, and statistical data is generated.

[0049] 3. Estimation of the cause of failure: Based on the statistical data obtained from the search results, the probability of occurrence of the cause of failure is calculated and the cause with the highest probability is estimated.

[0050] 4. Solution Generation: Based on the estimated cause of the failure, generative AI is used to generate the optimal solution, as well as a list of required replacement parts.

[0051] 5. Send Proposal: Send the generated solution and parts list to the user's device.

[0052] 6. Receiving and recording feedback: Receive feedback from users and record it in a database. This data will be used to identify the cause of future failures.

[0053] The role of the database

[0054] The database stores the following information:

[0055] 1. Past failure data: Records of past failures and their responses.

[0056] 2. Statistical data: Basic data for calculating the probability of occurrence of failure causes.

[0057] 3. Solutions and parts list: Information on the best solutions and required parts for each estimated cause of failure.

[0058] 4. Feedback information: Results after repair and feedback information from users.

[0059] Specific examples

[0060] For example, if your air conditioner is not cooling properly:

[0061] 1. User operation: The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[0062] 2. Server analysis: The server extracts the keywords "not cooling," "1 hour," and "air conditioning not working."

[0063] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[0064] 4. Application of statistical data: Based on statistical data obtained from past cases, the probability of refrigerant shortage, compressor failure, and filter clogging occurring is calculated.

[0065] 5. Estimation of the cause of failure: It is estimated that the most likely cause is a lack of refrigerant.

[0066] 6. Solution generation: The AI ​​generates a solution, adding refrigerant and cleaning the filter, and presents it to the user.

[0067] 7. User execution and feedback: The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[0068] 8. Server Recording: The server records this feedback in a database for future analysis.

[0069] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a failure, even without specialized knowledge.

[0070] The processing flow will be explained below.

[0071] Step 1:

[0072] The user accesses the device's input form and enters product defect information, including the product name, model number, detailed problem description, the circumstances under which the problem occurred, and any error messages.

[0073] Step 2:

[0074] When the user has finished entering the defect information in the input form, the terminal sends the information to the server, where it is converted into an appropriate data format before being sent.

[0075] Step 3:

[0076] The server receives the defect information sent from the device and starts a natural language processing (NLP) engine to analyze the received data.

[0077] Step 4:

[0078] The server uses a natural language processing engine to analyze the malfunction information and extract important keywords and phrases, such as "not cooling," "after one hour," and "air conditioning not working."

[0079] Step 5:

[0080] The server then searches the database based on the analyzed keywords and phrases to find similar past cases. For example, it extracts cases that include past reports of the product not cooling down.

[0081] Step 6:

[0082] The server uses statistical data based on the data obtained from the search results to calculate the probability of occurrence of each cause of failure, such as refrigerant shortage, compressor failure, and filter clogging.

[0083] Step 7:

[0084] The server estimates the most likely cause of the failure based on the calculated probability of occurrence. For example, it estimates that a refrigerant shortage is the most likely cause.

[0085] Step 8:

[0086] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, such as adding refrigerant or cleaning the filter.

[0087] Step 9:

[0088] The server compiles the generated solution and a list of required replacement parts and sends it to the user's device, where the user can view it.

[0089] Step 10:

[0090] The user then performs repairs or part replacements based on the proposed solutions, for example adding refrigerant and cleaning the filters.

[0091] Step 11:

[0092] The user inputs the results of the repairs they performed into the terminal and sends them to the server, for example, sending feedback such as "After adding the refrigerant, the air conditioner worked normally."

[0093] Step 12:

[0094] The server receives feedback from users, analyzes the data, and records it in a database. This information is used to identify the cause of future failures and improve solutions.

[0095] Example 1

[0096] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0097] Conventional systems for identifying the cause of a failure and presenting solutions have had difficulty accurately analyzing failure information from users and quickly and effectively presenting appropriate solutions. Furthermore, because user feedback could not be reflected in the next failure analysis, the learning and evolution of the entire system was delayed. This resulted in delays in troubleshooting product failures and reduced user convenience.

[0098] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0099] In this invention, the server includes means for receiving failure information, means for analyzing the information using natural language processing, means for searching for similar cases in a database, means for estimating the cause of the failure using statistical data, means for generating an optimal solution using a generative model, means for presenting the solution to the user, and means for receiving feedback and recording the feedback in the database. This makes it possible to quickly and accurately identify the cause of the failure based on detailed failure information from the user and provide an optimal solution. Furthermore, by reflecting user feedback in the system, learning and evolution of the entire system are promoted.

[0100] "Defect information" is information in which the user specifically and in detail describes abnormalities or problems with the product.

[0101] "Natural language processing" is a technology that uses computers to analyze natural language and understand its meaning and structure.

[0102] A "database" is a collection of data that systematically stores past failure cases and related data and manages them in a searchable format.

[0103] "Similar cases" are records of similar malfunctions or problems that have occurred in the past.

[0104] "Statistical data" refers to data for calculating the probability of occurrence of a failure cause based on numerical data obtained from a database.

[0105] The "cause of failure" is the fundamental cause of a product defect or abnormality.

[0106] A "generative model" is an algorithm that uses AI technology to generate an appropriate solution from given data.

[0107] A "solution" is a specific procedure or action proposed to solve a problem based on the estimated cause of the failure.

[0108] "Replacement Part" means a part or component required to resolve a defect.

[0109] "Feedback" is the evaluation or information provided by a user based on the results or experience of the solution they have implemented.

[0110] A "server" is a computer system that receives, analyzes, and stores data via a network, and provides the processed results to a client terminal.

[0111] A "user terminal" is a device such as a computer or smartphone that a user uses to input information or receive data from a server.

[0112] Overall system configuration

[0113] The system of the present invention mainly consists of a user terminal, a server, and a database. The user inputs fault information using the terminal, and the information is sent to the server. The server processes the information, compares it with the database, deduces the cause of the fault, and generates a solution.

[0114] Role of user terminal

[0115] The user terminal is the device that inputs information about the problem and receives and checks solutions from the server. Specifically, the user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the terminal. This information is then sent to the server. User terminals can be PCs or smartphones.

[0116] Server Roles

[0117] The server is the central part of the system and performs a variety of processes. First, it receives the failure information sent from the user's device. Next, it analyzes the information using a natural language processing engine (such as SpaCy or BERT) and extracts important keywords, which allows it to identify the problem. The server then searches a database based on the extracted keywords and refers to similar cases. The server estimates the cause of the failure based on statistical data obtained from past failure data. It then uses a generative AI model (such as a GPT model) to generate an optimal solution. The server also generates a list of required replacement parts and sends this to the user's device. Finally, it receives feedback from the user and records it in a database.

[0118] The role of the database

[0119] The database stores past failure cases and their corresponding records, statistical data, solutions, replacement parts lists, feedback information, etc. This provides the server with the basic data to quickly identify the cause of the failure and generate an appropriate solution.

[0120] Specific examples

[0121] For example, the following shows the flow when the user inputs "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[0122] 1. User operation: The user enters the defect information into the input form on the terminal.

[0123] 2. Server analysis: The server extracts the keywords "not cooling," "one hour later," and "air conditioning not working."

[0124] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[0125] 4. Estimation of failure causes: Calculate the probability of occurrence of causes such as refrigerant shortage, compressor failure, and filter blockage from statistical data. Estimate the cause with the highest probability (e.g., refrigerant shortage).

[0126] 5. Solution generation: The AI ​​generates solutions, "add refrigerant" and "clean the filter," and presents them to the user.

[0127] 6. User execution and feedback: The user executes the proposed solution and enters the results into the terminal.

[0128] 7. Server Recording: The server receives the feedback and records it in a database.

[0129] Prompt Sentence Examples

[0130] "My air conditioner isn't cooling. If it stops working after about an hour of use, what could be the cause of the problem? Please tell me what to do."

[0131] This system allows users to quickly and accurately identify the cause of a failure and find the optimal solution simply by entering detailed information about the product's malfunction.In addition, accumulating feedback from users will improve the analytical capabilities and scope of application of the entire system.

[0132] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0133] Step 1:

[0134] The user inputs the defect information.

[0135] Input: Product name, model number, problem description, circumstances under which it occurred, error messages, etc.

[0136] Specific operation: The user enters information such as "The air conditioner is not cooling. After about an hour of use, the cooling stops working" into the input form on the device.

[0137] Output: The entered defect information is saved on the terminal and later sent to the server.

[0138] Step 2:

[0139] The device sends the defect information to the server.

[0140] Input: The defect information entered by the user.

[0141] Specific operation: The device automatically sends the saved defect information to the server's API.

[0142] Output: The failure information is passed to the server.

[0143] Step 3:

[0144] The server receives and analyzes the defect information.

[0145] Input: The error information sent from the device.

[0146] Specific operation: The server uses a natural language processing engine (e.g., SpaCy, BERT) to analyze the malfunction information and extract important keywords (e.g., "not cooling," "after one hour," "air conditioning not working").

[0147] Output: Extracted keyword list.

[0148] Step 4:

[0149] The server searches the database based on the keyword list.

[0150] Input: Extracted keyword list.

[0151] Specific operation: The server searches the database and retrieves past similar cases.

[0152] Output: A list of similar cases.

[0153] Step 5:

[0154] The server uses statistical data from similar cases to estimate the cause of the failure.

[0155] Input: A list of similar cases.

[0156] Specific operation: Based on the statistical data, the server calculates the occurrence probability of each cause of failure (e.g., refrigerant shortage, compressor failure, clogged filter).

[0157] Output: Probability of occurrence of each failure cause.

[0158] Step 6:

[0159] The server uses a generative model based on the most probable causes of failure to generate an optimal solution.

[0160] Input: Probability of occurrence of failure cause.

[0161] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate the optimal solution for the most probable cause of the failure, for example, "add refrigerant" and "clean the filter."

[0162] Output: A list of solutions.

[0163] Step 7:

[0164] The server transmits the generated solution and a list of required replacement parts to the user terminal.

[0165] Input: List of solutions, list of replacement parts needed.

[0166] Specific operation: The server compiles the solution and replacement parts list and sends them to the user terminal.

[0167] Output: Solution and parts list displayed on the user's terminal.

[0168] Step 8:

[0169] The user executes the solution and enters the results into the terminal.

[0170] Input: Solution and replacement parts list sent from the server.

[0171] Specific actions: The user takes problem-solving actions according to the presented steps and inputs the results (e.g., success or failure of the solution) into the terminal.

[0172] Output: The feedback information is stored on the device and later sent to the server.

[0173] Step 9:

[0174] The server receives the feedback from the user and records it in a database.

[0175] Input: Feedback information entered by the user into the device.

[0176] Specific operation: The server receives the feedback sent by the device and stores it in a database.

[0177] Output: Feedback information is recorded in a database.

[0178] Through these steps, users can input product defect information, the server performs analysis and estimation, and provides the user with the optimal solution. By incorporating feedback, the accuracy and efficiency of the entire system are improved.

[0179] (Application example 1)

[0180] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0181] Conventional fault diagnosis systems estimate the cause of a fault based on fault information entered by the user and provide solutions. However, as factories become more automated, a faster and more advanced response is required. However, these systems rely on manual user operation, making it difficult to respond in real time or collect feedback. In particular, when a factory robot breaks down, the manual input of fault information is cumbersome, making it difficult to respond quickly. Therefore, a new system that can diagnose faults and provide a rapid response for factory robots is needed.

[0182] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0183] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the received defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for calculating the probability of occurrence of a failure cause using statistical data from the search results, means for estimating the failure cause based on the calculated probability, means for generating an optimal solution based on the estimated failure cause, means for presenting the generated solution to the user, means for coordinating with a factory robot to execute the presented solution, and means for receiving feedback from the user via the factory robot and recording it in the database, thereby enabling factory robot failure diagnosis and rapid response.

[0184] "User" refers to the factory technician or operator who inputs the defect information and implements the proposed solution.

[0185] "Fault information" is information that provides detailed explanations of failures and abnormalities that occur in factory robots and other equipment.

[0186] "Means for receiving" refers to a device or program that has the function of sending and receiving defect information entered by a user to a server.

[0187] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning.

[0188] "Means for analyzing" refers to a program or device that has the function of analyzing received defect information using natural language processing technology.

[0189] "Database" refers to a computer system that stores and manages past failure cases and solutions.

[0190] "Searching means" refers to a program that has the function of searching for similar past cases from a database based on the analysis results.

[0191] "Statistical data" refers to data for calculating the probability of occurrence of a specific cause of failure based on past cases.

[0192] "Means for calculating" refers to a program that has the function of calculating the probability of occurrence of a failure cause using statistical data.

[0193] "Cause of failure" refers to the factors that cause a specific malfunction or abnormality to occur.

[0194] "Means for estimating" refers to a program that has the function of estimating the most likely cause of a failure based on the calculated probability of occurrence.

[0195] "Solution method" refers to the procedure for instructing the optimal repair or countermeasure for the estimated cause of the failure.

[0196] "Means for generating" refers to a program that has the function of automatically generating a solution based on the estimated cause of the failure.

[0197] "Presentation means" refers to a display device or program for showing the generated solution to the user.

[0198] "Means for collaboration" refers to devices or programs that have the function of communicating with factory robots and instructing their operations in order to implement the proposed solution.

[0199] "Feedback" refers to the results and opinions of users who implement the solutions provided.

[0200] A "factory robot" refers to a mechanical device that performs work automatically within a factory.

[0201] "Means for recording" refers to a program that has the function of storing feedback information in a database.

[0202] MODE FOR CARRYING OUT THE INVENTION

[0203] This invention relates to a fault diagnosis system for factory robots that estimates the cause of a fault based on malfunction information provided by the user and proposes the optimal solution. This system is composed of a user terminal, a server, and a database, and realizes factory automation and efficient fault response.

[0204] Hardware and software used

[0205] An embodiment of the present invention uses the following hardware and software:

[0206] Hardware: smartphones, tablets, factory robots, servers

[0207] Software: Programming languages ​​(Python, JavaScript®), databases (MySQL®, PostgreSQL), AI models (GPT-3®, BERT), NLP engines (spaCy, NLTK), cloud services (AWS®, Google® Cloud)

[0208] Data processing and calculation flow

[0209] 1. User operations

[0210] Users use smartphones or tablets to input malfunction information for factory robots. The malfunction information includes the product name, model number, details of the problem, the circumstances under which the problem occurred, and error messages. This information is then sent from the user's device to the server.

[0211] 2. Data Reception and Analysis

[0212] The server receives the defect information sent from the user terminal. The received defect information is analyzed using a natural language processing engine (e.g., spaCy) to extract important keywords and patterns.

[0213] 3. Database Verification

[0214] Based on the analyzed data, the server searches the database for similar past cases, referencing past failure cases and response records, and generates statistical data based on the search results.

[0215] 4. Estimation of the cause of the failure

[0216] The server uses the generated statistical data to calculate the probability of occurrence of each cause of failure, and estimates the cause with the highest probability.

[0217] 5. Solution Generation

[0218] Based on the estimated cause of the failure, a generative AI model (e.g., GPT-3) is used to generate an optimal solution, along with a list of required replacement parts.

[0219] 6. Presentation to the User

[0220] The server sends the generated solution and replacement part list to the user's terminal and presents them to the user, who then executes the proposed solution.

[0221] 7. Collecting and recording feedback

[0222] The results of the user's execution are sent back to the server from the terminal, and the server receives and records this feedback information. This data is stored in a database and will be used for future failure analysis.

[0223] Specific examples

[0224] For example, if a robot arm in a factory stops working, the user inputs the following information about the problem into a tablet: "The robot arm won't move. There's an abnormal noise. Error code 5678 is displayed." Based on this information, the server analyzes it using an NLP engine and performs a database search. Statistical data is generated from the search results, and after calculating the probability of occurrence, it is estimated that the most likely cause is "motor failure." A generative AI model (e.g., GPT-3) is used to suggest "replacing the motor" as a solution. The user replaces the motor and re-enters the results into the system. This series of steps enables factory robot failures to be handled quickly and accurately.

[0225] Prompt Sentence Examples

[0226] "We are experiencing the following problems with our factory robot: The robot arm does not move, there is an abnormal noise, and error code 5678 is displayed." Please suggest the best solution based on this situation."

[0227] As a result, the present invention realizes rapid and accurate failure analysis and presentation of solutions based on the defect information input by the user.

[0228] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0229] Program processing steps and specific operations

[0230] Step 1:

[0231] The terminal receives the defect information input by the user. The user inputs the product name, model number, details of the defect, the circumstances when the defect occurred, and an error message into the input form on the terminal. This input data is sent from the terminal to the server.

[0232] Input: Product name, model number, problem details, circumstances under which it occurred, error message

[0233] Output: Defect information data sent to the server

[0234] Step 2:

[0235] The server receives the defect information data sent from the device. The received data is analyzed using a natural language processing engine (e.g., spaCy). During the analysis, important keywords and patterns are extracted and used in the next step, a database search.

[0236] Input: Received defect information data

[0237] Output: Extracted keywords and patterns

[0238] Step 3:

[0239] The server then searches the database based on the analysis results. The search locates similar cases from past failure cases and response records. In this process, a list of related cases and statistical data is generated.

[0240] Input: Extracted keywords or patterns

[0241] Output: List of similar cases and statistical data

[0242] Step 4:

[0243] The server uses the generated statistical data to calculate the probability of occurrence of the cause of the failure, and estimates the most likely cause of the failure based on the calculated probability of occurrence.

[0244] Input: List of similar cases and statistical data

[0245] Output: Estimated cause of failure and its probability of occurrence

[0246] Step 5:

[0247] The server uses a generative AI model (e.g., GPT-3) to generate an optimal solution based on the estimated cause of the failure, while simultaneously generating a list of required replacement parts.

[0248] Input: Estimated cause of failure and its probability of occurrence

[0249] Output: Solution and replacement part list

[0250] Step 6:

[0251] The server sends the generated solutions and a list of replacement parts to the user's terminal and presents them to the user, who then checks and implements the solutions.

[0252] Input: Solution and replacement part list

[0253] Output: A list of solutions and replacement parts sent to the user's device

[0254] Step 7:

[0255] The feedback obtained based on the solution method the user performs is sent from the terminal to the server. The server receives and records this feedback and stores it in a database. This feedback data is used for future fault analysis and resolution.

[0256] Input: Feedback of the results of the user's actions

[0257] Output: Feedback information recorded in a database

[0258] This allows for rapid and accurate diagnosis and resolution of factory robot failures through a series of steps.

[0259] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0260] The present invention is a system that combines a system in which a user inputs product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution, with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[0261] Overall system configuration

[0262] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[0263] User Roles

[0264] The user mainly performs the following operations:

[0265] 1. Enter the defect information:

[0266] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the device.

[0267] The emotion engine also simultaneously monitors the user's input information and behavior and analyzes emotions.

[0268] 2. Check the solution:

[0269] The user checks the solution sent from the server on the terminal.

[0270] Content optimized by the emotion engine is presented.

[0271] 3. Response implementation and feedback:

[0272] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[0273] Server Roles

[0274] The server is the center of the system and performs the following tasks:

[0275] 1. Data reception and analysis:

[0276] The server receives the defect information sent from the user terminal and launches a natural language processing (NLP) engine to analyze the received data.

[0277] Extract important keywords and phrases.

[0278] 2. Database Matching:

[0279] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[0280] 3. Applying statistical data and estimating the cause of failure:

[0281] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[0282] 4. Generate a solution:

[0283] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution.

[0284] It also generates a list of required replacement parts.

[0285] 5. Solution suggestions:

[0286] The generated solution and the required parts list are sent to the user's terminal.

[0287] The content presented is optimized based on the analysis results of the emotion engine.

[0288] 6. Receiving and recording feedback:

[0289] Receive feedback from users, analyze the data and record it in a database.

[0290] The role of the emotional engine

[0291] The Emotion Engine is responsible for:

[0292] 1. User Emotion Recognition:

[0293] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[0294] 2. Optimizing the solution:

[0295] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[0296] 3. Recording emotional data:

[0297] The recognized user emotion data is recorded in a database and used for future analysis.

[0298] The role of the database

[0299] The database stores the following information:

[0300] 1. Past failure data:

[0301] Past failure cases and their corresponding records.

[0302] 2. Statistics:

[0303] Basic data for calculating the probability of occurrence of a failure cause.

[0304] 3. Solution and parts list:

[0305] Information on the optimal solution and required parts for each estimated cause of failure.

[0306] 4. Feedback Information:

[0307] Repair results and user feedback information.

[0308] 5. Emotional Data:

[0309] Record the results of user sentiment analysis and use the data for future reference.

[0310] Specific examples

[0311] For example, if your air conditioner is not cooling properly:

[0312] 1. User Action:

[0313] The user types into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes that the user is frustrated based on the input data and typing speed.

[0314] 2. Server Analysis:

[0315] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working." The emotion engine analyzes the speed of input and the parts that are emphasized.

[0316] 3. Database Search:

[0317] The server searches the database based on these keywords and refers to similar past cases.

[0318] 4. Statistical Data Application:

[0319] Based on the statistical data obtained from the search, the probability of refrigerant shortage, compressor failure, and filter blockage occurring is calculated.

[0320] 5. Estimation of the cause of the failure:

[0321] The most likely cause is a lack of refrigerant.

[0322] 6. Generate a solution:

[0323] The solution is to add refrigerant and clean the filter. The emotion engine reflects the user's frustration and presents the solution in a format that is as simple and quick to respond as possible.

[0324] 7. User execution and feedback:

[0325] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[0326] 8. Server Records:

[0327] The server records this feedback in a database, along with emotional data, for future analysis.

[0328] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a malfunction without requiring specialized knowledge, and also enables responses that take into consideration the user's feelings.

[0329] The processing flow will be explained below.

[0330] Step 1:

[0331] The user accesses the input form on the device and enters product defect information, including the product name, model number, detailed problem description, circumstances at the time of the occurrence, and error message. As the user types, the emotion engine analyzes the user's typing speed and writing style to evaluate the user's emotions.

[0332] Step 2:

[0333] When the user completes the input and presses the send button, the device sends the defect information and emotion data to the server, including metadata reflecting the user's emotion.

[0334] Step 3:

[0335] The server launches a natural language processing engine to analyze the malfunction information received from the device. The server extracts important keywords and phrases to identify the problem. For example, it extracts keywords such as "not cooling," "one hour later," and "air conditioning not working."

[0336] Step 4:

[0337] The server searches a database for similar cases based on the analyzed keywords and phrases. The server checks whether a similar problem has already been reported.

[0338] Step 5:

[0339] Based on the search results, the server uses statistical data to calculate the probability of each cause of failure, such as a refrigerant shortage, compressor failure, or clogged filter.

[0340] Step 6:

[0341] The server evaluates the probability of occurrence of each cause and estimates the most likely cause of the failure, for example, a refrigerant shortage.

[0342] Step 7:

[0343] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, generating specific steps to add refrigerant and clean the filter.

[0344] Step 8:

[0345] The server compiles the generated list of solutions and replacement parts and optimizes it based on the user's emotional data, for example prioritizing simple and fast solutions if the user is frustrated.

[0346] Step 9:

[0347] The server sends the optimized solution and parts list to the user's device, where the user can review and understand the specific steps to take.

[0348] Step 10:

[0349] The user performs repairs or part replacements based on the proposed solutions, for example, the user adds refrigerant and cleans the filter.

[0350] Step 11:

[0351] The user inputs the results of the repair into the terminal and sends them to the server. For example, the user sends feedback such as, "After adding the refrigerant, the air conditioner worked normally."

[0352] Step 12:

[0353] The server receives feedback from users, analyzes the data, and records it in a database. At the same time, it also records the user's emotional data for future analysis.

[0354] In this way, this system is not only able to quickly and appropriately identify the cause of a malfunction and propose a solution, but also to respond in a way that takes the user's feelings into consideration.

[0355] Example 2

[0356] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0357] Conventional fault diagnosis systems have difficulty efficiently analyzing malfunction information from users and identifying the cause of the malfunction. They also lack the ability to recognize users' emotions and stress levels and provide optimal solutions, potentially resulting in a poor user experience. Furthermore, the solutions they generate often do not provide a list of replacement parts needed, making it difficult for users to prepare for repairs. These issues make it difficult to provide prompt and appropriate responses.

[0358] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: means for receiving malfunction information input by a user; means for analyzing the received malfunction information using natural language processing; means for searching a database for similar past cases based on the analysis results; means for calculating the occurrence probability of a malfunction cause using statistical data from the search results; means for estimating the malfunction cause based on the calculated probability; means for generating an optimal solution based on the estimated malfunction cause using a generative AI model; means for presenting the generated solution to the user; means for analyzing the user's input information and behavior and recognizing emotions; and means for optimizing the presented solution based on the recognized emotions. This makes it possible to quickly and appropriately identify the malfunction cause and provide an efficient solution. Furthermore, it is possible to respond in consideration of the user's emotions, thereby improving the user experience.

[0359] A "user" is an entity that uses the system to input product defect information and implement suggested solutions.

[0360] A "terminal" is an electronic device that a user uses to input defect information, check solutions, and provide feedback on the results of the solutions.

[0361] The "server" is a central processing unit that receives defect information from users, analyzes it, and provides the information.

[0362] "Problem information" is information that indicates details of a problem that has occurred in a product, and is entered by the user.

[0363] "Natural language processing" is a technology that allows computers to understand human language.

[0364] "Analysis" is the process of interpreting defect information and extracting important keywords and phrases.

[0365] A "database" is an information system that stores past failure cases, statistical data, solutions, replacement part information, and so on.

[0366] "Statistical data" is numerical information such as the probability of an event occurring, calculated based on past data.

[0367] "Cause of failure" refers to the factor that causes the product to not function properly.

[0368] "Occurrence probability" is a numerical value that indicates the degree of possibility that a particular cause of failure will occur.

[0369] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate optimal solutions from specific inputs.

[0370] A "solution" is a specific procedure or measure to resolve a specific cause of a failure.

[0371] An "emotion engine" is a system that analyzes the user's input information and operational behavior and recognizes the user's emotions.

[0372] "Emotions" refer to the psychological state that a user feels while using a system, and include feelings of frustration and satisfaction.

[0373] "Recognition" is the process by which the system identifies the user's emotions.

[0374] "Optimization" means adjusting the solution presentation based on the user's feelings to make it more acceptable.

[0375] The "list of replacement parts" is a list of parts required to eliminate the cause of the failure.

[0376] "Feedback" is information that a user provides to the system as a result of implementing a solution.

[0377] This invention combines a system that allows users to input product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution with an emotion engine that recognizes the user's emotions. This system is implemented using the following hardware and software.

[0378] Overall system configuration

[0379] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[0380] User Roles

[0381] The user uses the terminal to:

[0382] 1. Enter the defect information:

[0383] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form. The emotion engine simultaneously monitors the user's input speed, choices, etc., and analyzes their emotions.

[0384] 2. Check the solution:

[0385] The user checks the solution sent from the server on their device, and the solution optimized by the emotion engine is presented.

[0386] 3. Response implementation and feedback:

[0387] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[0388] Server Roles

[0389] The server is the center of the system and performs the following tasks:

[0390] 1. Data reception and analysis:

[0391] The server receives the defect information sent from the user's device and analyzes it using a natural language processing (NLP) engine to extract important keywords and phrases.

[0392] 2. Database Matching:

[0393] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[0394] 3. Applying statistical data and estimating the cause of failure:

[0395] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each failure cause, and the failure cause with the highest probability is estimated.

[0396] 4. Generate a solution:

[0397] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution, along with a list of required replacement parts.

[0398] 5. Solution suggestions:

[0399] The generated solution and required parts list are sent to the user's device, and the content presented is optimized based on the analysis results of the emotion engine.

[0400] 6. Receiving and recording feedback:

[0401] Receive feedback from users, analyze the data and record it in a database.

[0402] The role of the emotional engine

[0403] The Emotion Engine is responsible for:

[0404] 1. User Emotion Recognition:

[0405] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[0406] 2. Optimizing the solution:

[0407] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[0408] 3. Recording emotional data:

[0409] The recognized user emotion data is recorded in a database and used for future analysis.

[0410] The role of the database

[0411] The database stores the following information:

[0412] 1. Past failure data:

[0413] Store records of past failure cases and their responses.

[0414] 2. Statistics:

[0415] Contains basic data for calculating the probability of occurrence of a failure cause.

[0416] 3. Solution and parts list:

[0417] Save information on optimal solutions and required parts.

[0418] 4. Feedback Information:

[0419] Stores the results of repairs and feedback information from users.

[0420] 5. Emotional Data:

[0421] Records the results of user sentiment analysis and includes data for future reference.

[0422] Specific examples

[0423] For example, if your air conditioner is not cooling properly:

[0424] 1. User Action:

[0425] The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes the user's frustration.

[0426] 2. Server Analysis:

[0427] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working."

[0428] 3. Database Search:

[0429] The server searches the database based on these keywords and refers to similar past cases.

[0430] 4. Statistical Data Application:

[0431] Based on statistical data, the probability of refrigerant shortage, compressor failure, and filter clogging is calculated.

[0432] 5. Estimation of the cause of the failure:

[0433] The most likely cause is a lack of refrigerant.

[0434] 6. Generate a solution:

[0435] A solution is generated suggesting adding refrigerant and cleaning the filter. The emotion engine reflects the user's frustration and presents the solution in a simple, immediate format.

[0436] 7. User execution and feedback:

[0437] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[0438] 8. Server Records:

[0439] The server records the feedback information and emotion data in a database.

[0440] Prompt Sentence Examples

[0441] "We have received a report that the air conditioner is not cooling. It has been observed that the cooling stops working after an hour of use and the user seems to be frustrated. Please suggest the cause of the problem and how to fix it."

[0442] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0443] Step 1: Enter your defect information

[0444] The user enters information about the problem, such as the product name, model number, details of the problem, the circumstances under which it occurred, and any error messages, into the device's input form. Once the input is complete, the user presses the send button. The emotion engine simultaneously monitors the user's input speed and operating behavior to recognize emotions. For example, if the user enters "The air conditioner isn't cooling. After about an hour of use, the cooling stopped working," the emotion of irritation will be recognized.

[0445] Input: Product name, model number, problem details, occurrence status, error message

[0446] Output: Input defect information, user emotion data

[0447] Step 2: Submit your defect report

[0448] The device packages the entered defect information in JSON format and sends it to the server. The data is sent to the endpoint as a POST request.

[0449] Input: Entered defect information, user emotion data

[0450] Output: Defect information sent to the server, user emotion data

[0451] Step 3: Data reception and analysis

[0452] The server receives the malfunction information sent from the device. The received data is passed to a natural language processing (NLP) engine, which extracts important keywords and phrases. For example, keywords such as "not cooling," "after one hour," and "air conditioning not working" are extracted.

[0453] Input: Error information received from the device

[0454] Output: Extracted keywords and phrases

[0455] Step 4: Search the database

[0456] The server searches the database based on the extracted keywords and phrases. It compares them with past malfunction cases and extracts similar cases. For example, it searches for past malfunction cases related to "not cooling" and retrieves cases such as refrigerant shortages and clogged filters.

[0457] Input: Extracted keywords and phrases

[0458] Output: Similar cases

[0459] Step 5: Identify the cause of the failure

[0460] The server applies statistical data based on similar cases retrieved from the database to calculate the probability of occurrence of each failure cause. Based on the calculated probabilities, it estimates the most likely cause of the failure. For example, it estimates that a refrigerant shortage is the most likely cause.

[0461] Input: Similar cases

[0462] Output: Estimated cause of failure, occurrence probability

[0463] Step 6: Generate a solution

[0464] The server runs a generative AI model to generate solutions based on the estimated cause of the failure, such as adding refrigerant and cleaning the filter, and also generates a list of required replacement parts.

[0465] Input: Estimated cause of failure, occurrence probability

[0466] Output: Generated solution, replacement parts list

[0467] Step 7: Optimize and present the solution

[0468] The server optimizes the solution based on the user's emotions, recognized using an emotion engine. For example, if the user is frustrated, the server presents a solution as a simple and quick procedure. The server then sends the optimized solution and parts list to the user's device.

[0469] Input: Generated solutions, replacement parts list, user sentiment data

[0470] Output: Optimized solution, replacement parts list

[0471] Step 8: Receive and record feedback

[0472] The user repairs or replaces parts based on the proposed solutions. The results are entered into the terminal and sent as feedback to the server. The server receives this feedback information, analyzes it, and records it in a database. Emotional data is also recorded.

[0473] Input: Repair execution results, feedback information

[0474] Output: Feedback information and emotion data recorded in the database

[0475] (Application example 2)

[0476] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0477] Conventional product defect analysis systems do not take into account the user's emotions, and therefore are unable to reduce the stress and frustration experienced when users input defect information or receive solutions. In addition, because generative AI models are not adequately used to propose optimal solutions, the solutions presented are not always effective, making it difficult to quickly resolve problems.

[0478] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0479] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for recognizing emotions, means for generating an optimal solution, and means for presenting the generated solution to the user. This reduces stress by taking the user's emotions into consideration, and enables effective and rapid problem solving using a generative AI model.

[0480] The "means for receiving defect information input by the user" is a function for transmitting information about product or system defects input by the user through a terminal to the server, and for the server to receive this information.

[0481] "Means for analyzing using natural language processing" refers to a technology that receives defect information entered by a user as text data, analyzes the text data, and extracts important keywords and phrases.

[0482] The "means for searching for similar past cases from a database" is a function for searching for similar past recorded cases of defects in a database based on analyzed defect information.

[0483] "Means for recognizing emotions" refers to technology that analyzes the user's input information and operation behavior, and recognizes the user's current emotions (for example, irritation, confusion, tension, etc.).

[0484] The "means for generating an optimal solution" is a function that uses a generative AI model to generate an optimal solution based on the analysis results and the estimated cause of the failure, and then proposes that solution.

[0485] The "means for presenting the generated solution to the user" is a function for displaying the generated solution to the user, and aims to present it in a format that is easy for the user to understand.

[0486] A "generative AI model" is an algorithm that uses artificial intelligence to generate new solutions or suggestions based on input information.

[0487] A "prompt" is a document containing instructions or questions that are input to a generative AI model, and serves as the basis for the AI ​​to propose the optimal solution.

[0488] The present invention is a system that allows a user to input malfunction information about security hardware (e.g., smart locks or surveillance cameras), identifies the cause of the malfunction based on that information, and proposes the optimal solution. It also combines a function that analyzes the user's emotions and provides optimal support based on those emotions. Specific embodiments of this system are described below.

[0489] Overall system configuration

[0490] The system mainly consists of a user's device, a server, a database, and an emotion recognition engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion recognition engine then analyzes the user's input information and behavior, recognizes their emotions, and optimizes the solution suggestions it presents.

[0491] User Roles

[0492] The user mainly performs the following operations:

[0493] 1. Enter the defect information:

[0494] The user enters details of the security hardware problem into the device's input form. For example, if the smart lock is unresponsive, the user can write, "The smart lock is unresponsive. Nothing happens when I press the button." The emotion recognition engine simultaneously monitors the user's input information and behavior and analyzes their emotions.

[0495] 2. Check the solution:

[0496] The user then checks the solution sent from the server on their device, and the content is optimized using an emotion recognition engine. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[0497] 3. Response implementation and feedback:

[0498] The user then takes action based on the proposed solution and inputs the results into the device, which is then sent to the server, which records this feedback in a database for future analysis.

[0499] Server Roles

[0500] The server is the center of the system and performs the following tasks:

[0501] 1. Data reception and analysis:

[0502] The server receives the defect information sent from the user device and starts a natural language processing engine (NLP engine) to analyze the received data. For example, it uses an NLPProcessor to extract important keywords and phrases.

[0503] 2. Database Matching:

[0504] Based on the analysis results, the database of past failures is checked to search for similar cases. This is done using the Database module.

[0505] 3. Applying statistical data and estimating the cause of failure:

[0506] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[0507] 4. Generate a solution:

[0508] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution. The optimal solution is generated using a prompt sentence as input. For example, the following prompt sentence is used:

[0509] - Example prompt: "My smart lock is unresponsive. The situation is as follows: dead battery. Please suggest the best solution."

[0510] If necessary, a list of parts required for the solution is automatically generated.

[0511] 5. Solution suggestions:

[0512] The generated solution and required parts list are sent to the user's device, and the presentation content is optimized based on the analysis results of the emotion recognition engine.

[0513] 6. Receiving and recording feedback:

[0514] It receives feedback from users, analyzes the data, and records it in a database. Emotional data is also recorded at the same time and used for future analysis.

[0515] The role of emotion recognition engines

[0516] The emotion recognition engine is responsible for:

[0517] 1. User Emotion Recognition:

[0518] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[0519] 2. Optimizing the solution:

[0520] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[0521] 3. Recording emotional data:

[0522] The recognized user emotion data is recorded in a database and used for future analysis.

[0523] In this way, the system allows users to quickly and appropriately resolve problems, and also provides a response that takes emotion into consideration.

[0524] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0525] Step 1:

[0526] The user enters the defect information

[0527] The user enters details of the security hardware problem (e.g., a malfunction of a smart lock) in text format into an input form on the device, and the entered information is sent to the server.

[0528] (Input): User's defect information

[0529] (Output): Raw data sent to the server

[0530] Step 2:

[0531] The server receives the failure information

[0532] The server receives the defect information sent from the user's device. Since the received data cannot be analyzed as is, it is analyzed using a natural language processing (NLP) engine.

[0533] (Input): Malfunction information sent from the device

[0534] (Output): Defect information to be passed to natural language processing

[0535] Step 3:

[0536] Analysis of defect information using natural language processing

[0537] The server's NLP engine analyzes the received malfunction information and extracts important keywords and phrases, such as "smart lock," "unresponsive," and "nothing happens when I press the button."

[0538] (Input): Error information sent to the server

[0539] (Output): Parsed keywords and phrases

[0540] Step 4:

[0541] Emotion recognition engine for analyzing user emotions

[0542] The emotion recognition engine analyzes the user's emotions based on their input information and operational behavior (e.g., typing speed). For example, it can detect that the user is irritated based on their typing speed and the text.

[0543] (Input): User input information and operation behavior

[0544] (Output): Parsed user sentiment

[0545] Step 5:

[0546] Search for similar past cases from the database

[0547] The server searches a database for similar past cases based on the analyzed keywords and queries related cases.

[0548] (Input): Parsed keyword

[0549] (Output): Past similar cases

[0550] Step 6:

[0551] Calculating the probability of failure causes using statistical data

[0552] Based on the search results, the server uses statistical data to calculate the probability of the cause of the failure. For example, it estimates that the smart lock's battery is most likely dead.

[0553] (Input): Past similar cases

[0554] (Output): Probability of each failure cause

[0555] Step 7:

[0556] Estimation of the cause of failure

[0557] The server estimates the most likely cause of the failure based on the calculated occurrence probability.

[0558] (Input): Probability of each failure cause

[0559] (Output): Estimated cause of failure

[0560] Step 8:

[0561] Generating solutions using generative AI models

[0562] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure. For example, the server inputs a prompt statement such as "The smart lock is unresponsive. The situation is as follows: the battery is dead. Please suggest the optimal solution." into the generative AI model to obtain a solution.

[0563] (Input): Estimated cause of failure and prompt statement

[0564] (Output): The optimal solution generated

[0565] Step 9:

[0566] Provide the user with a solution

[0567] The server optimizes the generated solution based on the user's emotions and sends it to the user's device. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[0568] (Input): Generated solutions and analyzed user sentiment

[0569] (Output): Optimized solution

[0570] Step 10:

[0571] Receiving feedback and recording it in a database

[0572] The user tries the proposed solutions and inputs the results into the terminal and sends them to the server. The server receives this feedback and records it in a database. The analyzed emotional data is also recorded at the same time.

[0573] (Input): User feedback

[0574] (Output): Feedback and emotion data recorded in a database

[0575] The above processing steps allow the user to quickly and appropriately resolve the problem, and also realize a response that takes into consideration the user's feelings.

[0576] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0577] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0578] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0579] [Second embodiment]

[0580] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0581] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0582] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0583] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0584] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0585] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0586] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0587] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0588] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0589] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0590] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0591] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0592] The present invention is a system that allows a user to input product defect information, identifies the cause of the failure based on that information, and proposes an optimal solution. Specific embodiments of this system will be described below.

[0593] Overall system configuration

[0594] The system mainly consists of a user's terminal, a server, and a database. The user uses the terminal to input information about the problem, which is then sent to the server. The server processes the information, compares it with the database, deduces the cause of the problem, and generates a solution.

[0595] User Roles

[0596] The user mainly performs the following operations:

[0597] 1. Entering defect information: The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form.

[0598] 2. Confirmation of the solution: The user confirms the solution sent from the server on the device.

[0599] 3. Implementing the solution and providing feedback: The user carries out the repair or part replacement based on the proposed solution, and then enters the results into the terminal again and sends them to the server.

[0600] Server Roles

[0601] The server is the central part of the system and performs the following tasks:

[0602] 1. Data reception and analysis: The server receives the defect information sent from the user's device and analyzes it using a natural language processing engine. Important keywords and patterns are extracted to identify the problem.

[0603] 2. Database collation: Based on the analysis results, the results are collated with a database of past failures, similar cases are searched for, and statistical data is generated.

[0604] 3. Estimation of the cause of failure: Based on the statistical data obtained from the search results, the probability of occurrence of the cause of failure is calculated and the cause with the highest probability is estimated.

[0605] 4. Solution Generation: Based on the estimated cause of the failure, generative AI is used to generate the optimal solution, as well as a list of required replacement parts.

[0606] 5. Send Proposal: Send the generated solution and parts list to the user's device.

[0607] 6. Receiving and recording feedback: Receive feedback from users and record it in a database. This data will be used to identify the cause of future failures.

[0608] The role of the database

[0609] The database stores the following information:

[0610] 1. Past failure data: Records of past failures and their responses.

[0611] 2. Statistical data: Basic data for calculating the probability of occurrence of failure causes.

[0612] 3. Solutions and parts list: Information on the best solutions and required parts for each estimated cause of failure.

[0613] 4. Feedback information: Results after repair and feedback information from users.

[0614] Specific examples

[0615] For example, if your air conditioner is not cooling properly:

[0616] 1. User operation: The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[0617] 2. Server analysis: The server extracts the keywords "not cooling," "1 hour," and "air conditioning not working."

[0618] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[0619] 4. Application of statistical data: Based on statistical data obtained from past cases, the probability of refrigerant shortage, compressor failure, and filter clogging occurring is calculated.

[0620] 5. Estimation of the cause of failure: It is estimated that the most likely cause is a lack of refrigerant.

[0621] 6. Solution generation: The AI ​​generates a solution, adding refrigerant and cleaning the filter, and presents it to the user.

[0622] 7. User execution and feedback: The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[0623] 8. Server Recording: The server records this feedback in a database for future analysis.

[0624] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a failure, even without specialized knowledge.

[0625] The processing flow will be explained below.

[0626] Step 1:

[0627] The user accesses the device's input form and enters product defect information, including the product name, model number, detailed problem description, the circumstances under which the problem occurred, and any error messages.

[0628] Step 2:

[0629] When the user has finished entering the defect information in the input form, the terminal sends the information to the server, where it is converted into an appropriate data format before being sent.

[0630] Step 3:

[0631] The server receives the defect information sent from the device and starts a natural language processing (NLP) engine to analyze the received data.

[0632] Step 4:

[0633] The server uses a natural language processing engine to analyze the malfunction information and extract important keywords and phrases, such as "not cooling," "after one hour," and "air conditioning not working."

[0634] Step 5:

[0635] The server then searches the database based on the analyzed keywords and phrases to find similar past cases. For example, it extracts cases that include past reports of the product not cooling down.

[0636] Step 6:

[0637] The server uses statistical data based on the data obtained from the search results to calculate the probability of occurrence of each cause of failure, such as refrigerant shortage, compressor failure, and filter clogging.

[0638] Step 7:

[0639] The server estimates the most likely cause of the failure based on the calculated probability of occurrence. For example, it estimates that a refrigerant shortage is the most likely cause.

[0640] Step 8:

[0641] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, such as adding refrigerant or cleaning the filter.

[0642] Step 9:

[0643] The server compiles the generated solution and a list of required replacement parts and sends it to the user's device, where the user can view it.

[0644] Step 10:

[0645] The user then performs repairs or part replacements based on the proposed solutions, for example adding refrigerant and cleaning the filters.

[0646] Step 11:

[0647] The user inputs the results of the repairs they performed into the terminal and sends them to the server, for example, sending feedback such as "After adding the refrigerant, the air conditioner worked normally."

[0648] Step 12:

[0649] The server receives feedback from users, analyzes the data, and records it in a database. This information is used to identify the cause of future failures and improve solutions.

[0650] Example 1

[0651] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0652] Conventional systems for identifying the cause of a failure and presenting solutions have had difficulty accurately analyzing failure information from users and quickly and effectively presenting appropriate solutions. Furthermore, because user feedback could not be reflected in the next failure analysis, the learning and evolution of the entire system was delayed. This resulted in delays in troubleshooting product failures and reduced user convenience.

[0653] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0654] In this invention, the server includes means for receiving failure information, means for analyzing the information using natural language processing, means for searching for similar cases in a database, means for estimating the cause of the failure using statistical data, means for generating an optimal solution using a generative model, means for presenting the solution to the user, and means for receiving feedback and recording the feedback in the database. This makes it possible to quickly and accurately identify the cause of the failure based on detailed failure information from the user and provide an optimal solution. Furthermore, by reflecting user feedback in the system, learning and evolution of the entire system are promoted.

[0655] "Defect information" is information in which the user specifically and in detail describes abnormalities or problems with the product.

[0656] "Natural language processing" is a technology that uses computers to analyze natural language and understand its meaning and structure.

[0657] A "database" is a collection of data that systematically stores past failure cases and related data and manages them in a searchable format.

[0658] "Similar cases" are records of similar malfunctions or problems that have occurred in the past.

[0659] "Statistical data" refers to data for calculating the probability of occurrence of a failure cause based on numerical data obtained from a database.

[0660] The "cause of failure" is the fundamental cause of a product defect or abnormality.

[0661] A "generative model" is an algorithm that uses AI technology to generate an appropriate solution from given data.

[0662] A "solution" is a specific procedure or action proposed to solve a problem based on the estimated cause of the failure.

[0663] "Replacement Part" means a part or component required to resolve a defect.

[0664] "Feedback" is the evaluation or information provided by a user based on the results or experience of the solution they have implemented.

[0665] A "server" is a computer system that receives, analyzes, and stores data via a network, and provides the processed results to a client terminal.

[0666] A "user terminal" is a device such as a computer or smartphone that a user uses to input information or receive data from a server.

[0667] Overall system configuration

[0668] The system of the present invention mainly consists of a user terminal, a server, and a database. The user inputs fault information using the terminal, and the information is sent to the server. The server processes the information, compares it with the database, deduces the cause of the fault, and generates a solution.

[0669] Role of user terminal

[0670] The user terminal is the device that inputs information about the problem and receives and checks solutions from the server. Specifically, the user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the terminal. This information is then sent to the server. User terminals can be PCs or smartphones.

[0671] Server Roles

[0672] The server is the central part of the system and performs a variety of processes. First, it receives the failure information sent from the user's device. Next, it analyzes the information using a natural language processing engine (such as SpaCy or BERT) and extracts important keywords, which allows it to identify the problem. The server then searches a database based on the extracted keywords and refers to similar cases. The server estimates the cause of the failure based on statistical data obtained from past failure data. It then uses a generative AI model (such as a GPT model) to generate an optimal solution. The server also generates a list of required replacement parts and sends this to the user's device. Finally, it receives feedback from the user and records it in a database.

[0673] The role of the database

[0674] The database stores past failure cases and their corresponding records, statistical data, solutions, replacement parts lists, feedback information, etc. This provides the server with the basic data to quickly identify the cause of the failure and generate an appropriate solution.

[0675] Specific examples

[0676] For example, the following shows the flow when the user inputs "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[0677] 1. User operation: The user enters the defect information into the input form on the terminal.

[0678] 2. Server analysis: The server extracts the keywords "not cooling," "one hour later," and "air conditioning not working."

[0679] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[0680] 4. Estimation of failure causes: Calculate the probability of occurrence of causes such as refrigerant shortage, compressor failure, and filter blockage from statistical data. Estimate the cause with the highest probability (e.g., refrigerant shortage).

[0681] 5. Solution generation: The AI ​​generates solutions, "add refrigerant" and "clean the filter," and presents them to the user.

[0682] 6. User execution and feedback: The user executes the proposed solution and enters the results into the terminal.

[0683] 7. Server Recording: The server receives the feedback and records it in a database.

[0684] Prompt Sentence Examples

[0685] "My air conditioner isn't cooling. If it stops working after about an hour of use, what could be the cause of the problem? Please tell me what to do."

[0686] This system allows users to quickly and accurately identify the cause of a failure and find the optimal solution simply by entering detailed information about the product's malfunction.In addition, accumulating feedback from users will improve the analytical capabilities and scope of application of the entire system.

[0687] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0688] Step 1:

[0689] The user inputs the defect information.

[0690] Input: Product name, model number, problem description, circumstances under which it occurred, error messages, etc.

[0691] Specific operation: The user enters information such as "The air conditioner is not cooling. After about an hour of use, the cooling stops working" into the input form on the device.

[0692] Output: The entered defect information is saved on the terminal and later sent to the server.

[0693] Step 2:

[0694] The device sends the defect information to the server.

[0695] Input: The defect information entered by the user.

[0696] Specific operation: The device automatically sends the saved defect information to the server's API.

[0697] Output: The failure information is passed to the server.

[0698] Step 3:

[0699] The server receives and analyzes the defect information.

[0700] Input: The error information sent from the device.

[0701] Specific operation: The server uses a natural language processing engine (e.g., SpaCy, BERT) to analyze the malfunction information and extract important keywords (e.g., "not cooling," "after one hour," "air conditioning not working").

[0702] Output: Extracted keyword list.

[0703] Step 4:

[0704] The server searches the database based on the keyword list.

[0705] Input: Extracted keyword list.

[0706] Specific operation: The server searches the database and retrieves past similar cases.

[0707] Output: A list of similar cases.

[0708] Step 5:

[0709] The server uses statistical data from similar cases to estimate the cause of the failure.

[0710] Input: A list of similar cases.

[0711] Specific operation: Based on the statistical data, the server calculates the occurrence probability of each cause of failure (e.g., refrigerant shortage, compressor failure, clogged filter).

[0712] Output: Probability of occurrence of each failure cause.

[0713] Step 6:

[0714] The server uses a generative model based on the most probable causes of failure to generate an optimal solution.

[0715] Input: Probability of occurrence of failure cause.

[0716] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate the optimal solution for the most probable cause of the failure, for example, "add refrigerant" and "clean the filter."

[0717] Output: A list of solutions.

[0718] Step 7:

[0719] The server transmits the generated solution and a list of required replacement parts to the user terminal.

[0720] Input: List of solutions, list of replacement parts needed.

[0721] Specific operation: The server compiles the solution and replacement parts list and sends them to the user terminal.

[0722] Output: Solution and parts list displayed on the user's terminal.

[0723] Step 8:

[0724] The user executes the solution and enters the results into the terminal.

[0725] Input: Solution and replacement parts list sent from the server.

[0726] Specific actions: The user takes problem-solving actions according to the presented steps and inputs the results (e.g., success or failure of the solution) into the terminal.

[0727] Output: The feedback information is stored on the device and later sent to the server.

[0728] Step 9:

[0729] The server receives the feedback from the user and records it in a database.

[0730] Input: Feedback information entered by the user into the device.

[0731] Specific operation: The server receives the feedback sent by the device and stores it in a database.

[0732] Output: Feedback information is recorded in a database.

[0733] Through these steps, users can input product defect information, the server performs analysis and estimation, and provides the user with the optimal solution. By incorporating feedback, the accuracy and efficiency of the entire system are improved.

[0734] (Application example 1)

[0735] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0736] Conventional fault diagnosis systems estimate the cause of a fault based on fault information entered by the user and provide solutions. However, as factories become more automated, a faster and more advanced response is required. However, these systems rely on manual user operation, making it difficult to respond in real time or collect feedback. In particular, when a factory robot breaks down, the manual input of fault information is cumbersome, making it difficult to respond quickly. Therefore, a new system that can diagnose faults and provide a rapid response for factory robots is needed.

[0737] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0738] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the received defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for calculating the probability of occurrence of a failure cause using statistical data from the search results, means for estimating the failure cause based on the calculated probability, means for generating an optimal solution based on the estimated failure cause, means for presenting the generated solution to the user, means for coordinating with a factory robot to execute the presented solution, and means for receiving feedback from the user via the factory robot and recording it in the database, thereby enabling factory robot failure diagnosis and rapid response.

[0739] "User" refers to the factory technician or operator who inputs the defect information and implements the proposed solution.

[0740] "Fault information" is information that provides detailed explanations of failures and abnormalities that occur in factory robots and other equipment.

[0741] "Means for receiving" refers to a device or program that has the function of sending and receiving defect information entered by a user to a server.

[0742] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning.

[0743] "Means for analyzing" refers to a program or device that has the function of analyzing received defect information using natural language processing technology.

[0744] "Database" refers to a computer system that stores and manages past failure cases and solutions.

[0745] "Searching means" refers to a program that has the function of searching for similar past cases from a database based on the analysis results.

[0746] "Statistical data" refers to data for calculating the probability of occurrence of a specific cause of failure based on past cases.

[0747] "Means for calculating" refers to a program that has the function of calculating the probability of occurrence of a failure cause using statistical data.

[0748] "Cause of failure" refers to the factors that cause a specific malfunction or abnormality to occur.

[0749] "Means for estimating" refers to a program that has the function of estimating the most likely cause of a failure based on the calculated probability of occurrence.

[0750] "Solution method" refers to the procedure for instructing the optimal repair or countermeasure for the estimated cause of the failure.

[0751] "Means for generating" refers to a program that has the function of automatically generating a solution based on the estimated cause of the failure.

[0752] "Presentation means" refers to a display device or program for showing the generated solution to the user.

[0753] "Means for collaboration" refers to devices or programs that have the function of communicating with factory robots and instructing their operations in order to implement the proposed solution.

[0754] "Feedback" refers to the results and opinions of users who implement the solutions provided.

[0755] A "factory robot" refers to a mechanical device that performs work automatically within a factory.

[0756] "Means for recording" refers to a program that has the function of storing feedback information in a database.

[0757] MODE FOR CARRYING OUT THE INVENTION

[0758] This invention relates to a fault diagnosis system for factory robots that estimates the cause of a fault based on malfunction information provided by the user and proposes the optimal solution. This system is composed of a user terminal, a server, and a database, and realizes factory automation and efficient fault response.

[0759] Hardware and software used

[0760] An embodiment of the present invention uses the following hardware and software:

[0761] Hardware: smartphones, tablets, factory robots, servers

[0762] Software: Programming languages ​​(Python, JavaScript), databases (MySQL, PostgreSQL), AI models (GPT-3, BERT), NLP engines (spaCy, NLTK), cloud services (AWS, Google Cloud)

[0763] Data processing and calculation flow

[0764] 1. User operations

[0765] Users use smartphones or tablets to input malfunction information for factory robots. The malfunction information includes the product name, model number, details of the problem, the circumstances under which the problem occurred, and error messages. This information is then sent from the user's device to the server.

[0766] 2. Data Reception and Analysis

[0767] The server receives the defect information sent from the user terminal. The received defect information is analyzed using a natural language processing engine (e.g., spaCy) to extract important keywords and patterns.

[0768] 3. Database Verification

[0769] Based on the analyzed data, the server searches the database for similar past cases, referencing past failure cases and response records, and generates statistical data based on the search results.

[0770] 4. Estimation of the cause of the failure

[0771] The server uses the generated statistical data to calculate the probability of occurrence of each cause of failure, and estimates the cause with the highest probability.

[0772] 5. Solution Generation

[0773] Based on the estimated cause of the failure, a generative AI model (e.g., GPT-3) is used to generate an optimal solution, along with a list of required replacement parts.

[0774] 6. Presentation to the User

[0775] The server sends the generated solution and replacement part list to the user's terminal and presents them to the user, who then executes the proposed solution.

[0776] 7. Collecting and recording feedback

[0777] The results of the user's execution are sent back to the server from the terminal, and the server receives and records this feedback information. This data is stored in a database and will be used for future failure analysis.

[0778] Specific examples

[0779] For example, if a robot arm in a factory stops working, the user inputs the following information about the problem into a tablet: "The robot arm won't move. There's an abnormal noise. Error code 5678 is displayed." Based on this information, the server analyzes it using an NLP engine and performs a database search. Statistical data is generated from the search results, and after calculating the probability of occurrence, it is estimated that the most likely cause is "motor failure." A generative AI model (e.g., GPT-3) is used to suggest "replacing the motor" as a solution. The user replaces the motor and re-enters the results into the system. This series of steps enables factory robot failures to be handled quickly and accurately.

[0780] Prompt Sentence Examples

[0781] "We are experiencing the following problems with our factory robot: The robot arm does not move, there is an abnormal noise, and error code 5678 is displayed." Please suggest the best solution based on this situation."

[0782] As a result, the present invention realizes rapid and accurate failure analysis and presentation of solutions based on the defect information input by the user.

[0783] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0784] Program processing steps and specific operations

[0785] Step 1:

[0786] The terminal receives the defect information input by the user. The user inputs the product name, model number, details of the defect, the circumstances when the defect occurred, and an error message into the input form on the terminal. This input data is sent from the terminal to the server.

[0787] Input: Product name, model number, problem details, circumstances under which it occurred, error message

[0788] Output: Defect information data sent to the server

[0789] Step 2:

[0790] The server receives the defect information data sent from the device. The received data is analyzed using a natural language processing engine (e.g., spaCy). During the analysis, important keywords and patterns are extracted and used in the next step, a database search.

[0791] Input: Received defect information data

[0792] Output: Extracted keywords and patterns

[0793] Step 3:

[0794] The server then searches the database based on the analysis results. The search locates similar cases from past failure cases and response records. In this process, a list of related cases and statistical data is generated.

[0795] Input: Extracted keywords or patterns

[0796] Output: List of similar cases and statistical data

[0797] Step 4:

[0798] The server uses the generated statistical data to calculate the probability of occurrence of the cause of the failure, and estimates the most likely cause of the failure based on the calculated probability of occurrence.

[0799] Input: List of similar cases and statistical data

[0800] Output: Estimated cause of failure and its probability of occurrence

[0801] Step 5:

[0802] The server uses a generative AI model (e.g., GPT-3) to generate an optimal solution based on the estimated cause of the failure, while simultaneously generating a list of required replacement parts.

[0803] Input: Estimated cause of failure and its probability of occurrence

[0804] Output: Solution and replacement part list

[0805] Step 6:

[0806] The server sends the generated solutions and a list of replacement parts to the user's terminal and presents them to the user, who then checks and implements the solutions.

[0807] Input: Solution and replacement part list

[0808] Output: A list of solutions and replacement parts sent to the user's device

[0809] Step 7:

[0810] The feedback obtained based on the solution method the user performs is sent from the terminal to the server. The server receives and records this feedback and stores it in a database. This feedback data is used for future fault analysis and resolution.

[0811] Input: Feedback of the results of the user's actions

[0812] Output: Feedback information recorded in a database

[0813] This allows for rapid and accurate diagnosis and resolution of factory robot failures through a series of steps.

[0814] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0815] The present invention is a system that combines a system in which a user inputs product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution, with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[0816] Overall system configuration

[0817] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[0818] User Roles

[0819] The user mainly performs the following operations:

[0820] 1. Enter the defect information:

[0821] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the device.

[0822] The emotion engine also simultaneously monitors the user's input information and behavior and analyzes emotions.

[0823] 2. Check the solution:

[0824] The user checks the solution sent from the server on the terminal.

[0825] Content optimized by the emotion engine is presented.

[0826] 3. Response implementation and feedback:

[0827] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[0828] Server Roles

[0829] The server is the center of the system and performs the following tasks:

[0830] 1. Data reception and analysis:

[0831] The server receives the defect information sent from the user terminal and launches a natural language processing (NLP) engine to analyze the received data.

[0832] Extract important keywords and phrases.

[0833] 2. Database Matching:

[0834] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[0835] 3. Applying statistical data and estimating the cause of failure:

[0836] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[0837] 4. Generate a solution:

[0838] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution.

[0839] It also generates a list of required replacement parts.

[0840] 5. Solution suggestions:

[0841] The generated solution and the required parts list are sent to the user's terminal.

[0842] The content presented is optimized based on the analysis results of the emotion engine.

[0843] 6. Receiving and recording feedback:

[0844] Receive feedback from users, analyze the data and record it in a database.

[0845] The role of the emotional engine

[0846] The Emotion Engine is responsible for:

[0847] 1. User Emotion Recognition:

[0848] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[0849] 2. Optimizing the solution:

[0850] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[0851] 3. Recording emotional data:

[0852] The recognized user emotion data is recorded in a database and used for future analysis.

[0853] The role of the database

[0854] The database stores the following information:

[0855] 1. Past failure data:

[0856] Past failure cases and their corresponding records.

[0857] 2. Statistics:

[0858] Basic data for calculating the probability of occurrence of a failure cause.

[0859] 3. Solution and parts list:

[0860] Information on the optimal solution and required parts for each estimated cause of failure.

[0861] 4. Feedback Information:

[0862] Repair results and user feedback information.

[0863] 5. Emotional Data:

[0864] Record the results of user sentiment analysis and use the data for future reference.

[0865] Specific examples

[0866] For example, if your air conditioner is not cooling properly:

[0867] 1. User Action:

[0868] The user types into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes that the user is frustrated based on the input data and typing speed.

[0869] 2. Server Analysis:

[0870] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working." The emotion engine analyzes the speed of input and the parts that are emphasized.

[0871] 3. Database Search:

[0872] The server searches the database based on these keywords and refers to similar past cases.

[0873] 4. Statistical Data Application:

[0874] Based on the statistical data obtained from the search, the probability of refrigerant shortage, compressor failure, and filter blockage occurring is calculated.

[0875] 5. Estimation of the cause of the failure:

[0876] The most likely cause is a lack of refrigerant.

[0877] 6. Generate a solution:

[0878] The solution is to add refrigerant and clean the filter. The emotion engine reflects the user's frustration and presents the solution in a format that is as simple and quick to respond as possible.

[0879] 7. User execution and feedback:

[0880] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[0881] 8. Server Records:

[0882] The server records this feedback in a database, along with emotional data, for future analysis.

[0883] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a malfunction without requiring specialized knowledge, and also enables responses that take into consideration the user's feelings.

[0884] The processing flow will be explained below.

[0885] Step 1:

[0886] The user accesses the input form on the device and enters product defect information, including the product name, model number, detailed problem description, circumstances at the time of the occurrence, and error message. As the user types, the emotion engine analyzes the user's typing speed and writing style to evaluate the user's emotions.

[0887] Step 2:

[0888] When the user completes the input and presses the send button, the device sends the defect information and emotion data to the server, including metadata reflecting the user's emotion.

[0889] Step 3:

[0890] The server launches a natural language processing engine to analyze the malfunction information received from the device. The server extracts important keywords and phrases to identify the problem. For example, it extracts keywords such as "not cooling," "one hour later," and "air conditioning not working."

[0891] Step 4:

[0892] The server searches a database for similar cases based on the analyzed keywords and phrases. The server checks whether a similar problem has already been reported.

[0893] Step 5:

[0894] Based on the search results, the server uses statistical data to calculate the probability of each cause of failure, such as a refrigerant shortage, compressor failure, or clogged filter.

[0895] Step 6:

[0896] The server evaluates the probability of occurrence of each cause and estimates the most likely cause of the failure, for example, a refrigerant shortage.

[0897] Step 7:

[0898] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, generating specific steps to add refrigerant and clean the filter.

[0899] Step 8:

[0900] The server compiles the generated list of solutions and replacement parts and optimizes it based on the user's emotional data, for example prioritizing simple and fast solutions if the user is frustrated.

[0901] Step 9:

[0902] The server sends the optimized solution and parts list to the user's device, where the user can review and understand the specific steps to take.

[0903] Step 10:

[0904] The user performs repairs or part replacements based on the proposed solutions, for example, the user adds refrigerant and cleans the filter.

[0905] Step 11:

[0906] The user inputs the results of the repair into the terminal and sends them to the server. For example, the user sends feedback such as, "After adding the refrigerant, the air conditioner worked normally."

[0907] Step 12:

[0908] The server receives feedback from users, analyzes the data, and records it in a database. At the same time, it also records the user's emotional data for future analysis.

[0909] In this way, this system is not only able to quickly and appropriately identify the cause of a malfunction and propose a solution, but also to respond in a way that takes the user's feelings into consideration.

[0910] Example 2

[0911] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0912] Conventional fault diagnosis systems have difficulty efficiently analyzing malfunction information from users and identifying the cause of the malfunction. They also lack the ability to recognize users' emotions and stress levels and provide optimal solutions, potentially resulting in a poor user experience. Furthermore, the solutions they generate often do not provide a list of replacement parts needed, making it difficult for users to prepare for repairs. These issues make it difficult to provide prompt and appropriate responses.

[0913] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: means for receiving malfunction information input by a user; means for analyzing the received malfunction information using natural language processing; means for searching a database for similar past cases based on the analysis results; means for calculating the occurrence probability of a malfunction cause using statistical data from the search results; means for estimating the malfunction cause based on the calculated probability; means for generating an optimal solution based on the estimated malfunction cause using a generative AI model; means for presenting the generated solution to the user; means for analyzing the user's input information and behavior and recognizing emotions; and means for optimizing the presented solution based on the recognized emotions. This makes it possible to quickly and appropriately identify the malfunction cause and provide an efficient solution. Furthermore, it is possible to respond in consideration of the user's emotions, thereby improving the user experience.

[0914] A "user" is an entity that uses the system to input product defect information and implement suggested solutions.

[0915] A "terminal" is an electronic device that a user uses to input defect information, check solutions, and provide feedback on the results of the solutions.

[0916] The "server" is a central processing unit that receives defect information from users, analyzes it, and provides the information.

[0917] "Problem information" is information that indicates details of a problem that has occurred in a product, and is entered by the user.

[0918] "Natural language processing" is a technology that allows computers to understand human language.

[0919] "Analysis" is the process of interpreting defect information and extracting important keywords and phrases.

[0920] A "database" is an information system that stores past failure cases, statistical data, solutions, replacement part information, and so on.

[0921] "Statistical data" is numerical information such as the probability of an event occurring, calculated based on past data.

[0922] "Cause of failure" refers to the factor that causes the product to not function properly.

[0923] "Occurrence probability" is a numerical value that indicates the degree of possibility that a particular cause of failure will occur.

[0924] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate optimal solutions from specific inputs.

[0925] A "solution" is a specific procedure or measure to resolve a specific cause of a failure.

[0926] An "emotion engine" is a system that analyzes the user's input information and operational behavior and recognizes the user's emotions.

[0927] "Emotions" refer to the psychological state that a user feels while using a system, and include feelings of frustration and satisfaction.

[0928] "Recognition" is the process by which the system identifies the user's emotions.

[0929] "Optimization" means adjusting the solution presentation based on the user's feelings to make it more acceptable.

[0930] The "list of replacement parts" is a list of parts required to eliminate the cause of the failure.

[0931] "Feedback" is information that a user provides to the system as a result of implementing a solution.

[0932] This invention combines a system that allows users to input product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution with an emotion engine that recognizes the user's emotions. This system is implemented using the following hardware and software.

[0933] Overall system configuration

[0934] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[0935] User Roles

[0936] The user uses the terminal to:

[0937] 1. Enter the defect information:

[0938] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form. The emotion engine simultaneously monitors the user's input speed, choices, etc., and analyzes their emotions.

[0939] 2. Check the solution:

[0940] The user checks the solution sent from the server on their device, and the solution optimized by the emotion engine is presented.

[0941] 3. Response implementation and feedback:

[0942] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[0943] Server Roles

[0944] The server is the center of the system and performs the following tasks:

[0945] 1. Data reception and analysis:

[0946] The server receives the defect information sent from the user's device and analyzes it using a natural language processing (NLP) engine to extract important keywords and phrases.

[0947] 2. Database Matching:

[0948] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[0949] 3. Applying statistical data and estimating the cause of failure:

[0950] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each failure cause, and the failure cause with the highest probability is estimated.

[0951] 4. Generate a solution:

[0952] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution, along with a list of required replacement parts.

[0953] 5. Solution suggestions:

[0954] The generated solution and required parts list are sent to the user's device, and the content presented is optimized based on the analysis results of the emotion engine.

[0955] 6. Receiving and recording feedback:

[0956] Receive feedback from users, analyze the data and record it in a database.

[0957] The role of the emotional engine

[0958] The Emotion Engine is responsible for:

[0959] 1. User Emotion Recognition:

[0960] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[0961] 2. Optimizing the solution:

[0962] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[0963] 3. Recording emotional data:

[0964] The recognized user emotion data is recorded in a database and used for future analysis.

[0965] The role of the database

[0966] The database stores the following information:

[0967] 1. Past failure data:

[0968] Store records of past failure cases and their responses.

[0969] 2. Statistics:

[0970] Contains basic data for calculating the probability of occurrence of a failure cause.

[0971] 3. Solution and parts list:

[0972] Save information on optimal solutions and required parts.

[0973] 4. Feedback Information:

[0974] Stores the results of repairs and feedback information from users.

[0975] 5. Emotional Data:

[0976] Records the results of user sentiment analysis and includes data for future reference.

[0977] Specific examples

[0978] For example, if your air conditioner is not cooling properly:

[0979] 1. User Action:

[0980] The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes the user's frustration.

[0981] 2. Server Analysis:

[0982] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working."

[0983] 3. Database Search:

[0984] The server searches the database based on these keywords and refers to similar past cases.

[0985] 4. Statistical Data Application:

[0986] Based on statistical data, the probability of refrigerant shortage, compressor failure, and filter clogging is calculated.

[0987] 5. Estimation of the cause of the failure:

[0988] The most likely cause is a lack of refrigerant.

[0989] 6. Generate a solution:

[0990] A solution is generated suggesting adding refrigerant and cleaning the filter. The emotion engine reflects the user's frustration and presents the solution in a simple, immediate format.

[0991] 7. User execution and feedback:

[0992] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[0993] 8. Server Records:

[0994] The server records the feedback information and emotion data in a database.

[0995] Prompt Sentence Examples

[0996] "We have received a report that the air conditioner is not cooling. It has been observed that the cooling stops working after an hour of use and the user seems to be frustrated. Please suggest the cause of the problem and how to fix it."

[0997] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0998] Step 1: Enter your defect information

[0999] The user enters information about the problem, such as the product name, model number, details of the problem, the circumstances under which it occurred, and any error messages, into the device's input form. Once the input is complete, the user presses the send button. The emotion engine simultaneously monitors the user's input speed and operating behavior to recognize emotions. For example, if the user enters "The air conditioner isn't cooling. After about an hour of use, the cooling stopped working," the emotion of irritation will be recognized.

[1000] Input: Product name, model number, problem details, occurrence status, error message

[1001] Output: Input defect information, user emotion data

[1002] Step 2: Submit your defect report

[1003] The device packages the entered defect information in JSON format and sends it to the server. The data is sent to the endpoint as a POST request.

[1004] Input: Entered defect information, user emotion data

[1005] Output: Defect information sent to the server, user emotion data

[1006] Step 3: Data reception and analysis

[1007] The server receives the malfunction information sent from the device. The received data is passed to a natural language processing (NLP) engine, which extracts important keywords and phrases. For example, keywords such as "not cooling," "after one hour," and "air conditioning not working" are extracted.

[1008] Input: Error information received from the device

[1009] Output: Extracted keywords and phrases

[1010] Step 4: Search the database

[1011] The server searches the database based on the extracted keywords and phrases. It compares them with past malfunction cases and extracts similar cases. For example, it searches for past malfunction cases related to "not cooling" and retrieves cases such as refrigerant shortages and clogged filters.

[1012] Input: Extracted keywords and phrases

[1013] Output: Similar cases

[1014] Step 5: Identify the cause of the failure

[1015] The server applies statistical data based on similar cases retrieved from the database to calculate the probability of occurrence of each failure cause. Based on the calculated probabilities, it estimates the most likely cause of the failure. For example, it estimates that a refrigerant shortage is the most likely cause.

[1016] Input: Similar cases

[1017] Output: Estimated cause of failure, occurrence probability

[1018] Step 6: Generate a solution

[1019] The server runs a generative AI model to generate solutions based on the estimated cause of the failure, such as adding refrigerant and cleaning the filter, and also generates a list of required replacement parts.

[1020] Input: Estimated cause of failure, occurrence probability

[1021] Output: Generated solution, replacement parts list

[1022] Step 7: Optimize and present the solution

[1023] The server optimizes the solution based on the user's emotions, recognized using an emotion engine. For example, if the user is frustrated, the server presents a solution as a simple and quick procedure. The server then sends the optimized solution and parts list to the user's device.

[1024] Input: Generated solutions, replacement parts list, user sentiment data

[1025] Output: Optimized solution, replacement parts list

[1026] Step 8: Receive and record feedback

[1027] The user repairs or replaces parts based on the proposed solutions. The results are entered into the terminal and sent as feedback to the server. The server receives this feedback information, analyzes it, and records it in a database. Emotional data is also recorded.

[1028] Input: Repair execution results, feedback information

[1029] Output: Feedback information and emotion data recorded in the database

[1030] (Application example 2)

[1031] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1032] Conventional product defect analysis systems do not take into account the user's emotions, and therefore are unable to reduce the stress and frustration experienced when users input defect information or receive solutions. In addition, because generative AI models are not adequately used to propose optimal solutions, the solutions presented are not always effective, making it difficult to quickly resolve problems.

[1033] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1034] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for recognizing emotions, means for generating an optimal solution, and means for presenting the generated solution to the user. This reduces stress by taking the user's emotions into consideration, and enables effective and rapid problem solving using a generative AI model.

[1035] The "means for receiving defect information input by the user" is a function for transmitting information about product or system defects input by the user through a terminal to the server, and for the server to receive this information.

[1036] "Means for analyzing using natural language processing" refers to a technology that receives defect information entered by a user as text data, analyzes the text data, and extracts important keywords and phrases.

[1037] The "means for searching for similar past cases from a database" is a function for searching for similar past recorded cases of defects in a database based on analyzed defect information.

[1038] "Means for recognizing emotions" refers to technology that analyzes the user's input information and operation behavior, and recognizes the user's current emotions (for example, irritation, confusion, tension, etc.).

[1039] The "means for generating an optimal solution" is a function that uses a generative AI model to generate an optimal solution based on the analysis results and the estimated cause of the failure, and then proposes that solution.

[1040] The "means for presenting the generated solution to the user" is a function for displaying the generated solution to the user, and aims to present it in a format that is easy for the user to understand.

[1041] A "generative AI model" is an algorithm that uses artificial intelligence to generate new solutions or suggestions based on input information.

[1042] A "prompt" is a document containing instructions or questions that are input to a generative AI model, and serves as the basis for the AI ​​to propose the optimal solution.

[1043] The present invention is a system that allows a user to input malfunction information about security hardware (e.g., smart locks or surveillance cameras), identifies the cause of the malfunction based on that information, and proposes the optimal solution. It also combines a function that analyzes the user's emotions and provides optimal support based on those emotions. Specific embodiments of this system are described below.

[1044] Overall system configuration

[1045] The system mainly consists of a user's device, a server, a database, and an emotion recognition engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion recognition engine then analyzes the user's input information and behavior, recognizes their emotions, and optimizes the solution suggestions it presents.

[1046] User Roles

[1047] The user mainly performs the following operations:

[1048] 1. Enter the defect information:

[1049] The user enters details of the security hardware problem into the device's input form. For example, if the smart lock is unresponsive, the user can write, "The smart lock is unresponsive. Nothing happens when I press the button." The emotion recognition engine simultaneously monitors the user's input information and behavior and analyzes their emotions.

[1050] 2. Check the solution:

[1051] The user then checks the solution sent from the server on their device, and the content is optimized using an emotion recognition engine. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[1052] 3. Response implementation and feedback:

[1053] The user then takes action based on the proposed solution and inputs the results into the device, which is then sent to the server, which records this feedback in a database for future analysis.

[1054] Server Roles

[1055] The server is the center of the system and performs the following tasks:

[1056] 1. Data reception and analysis:

[1057] The server receives the defect information sent from the user device and starts a natural language processing engine (NLP engine) to analyze the received data. For example, it uses an NLPProcessor to extract important keywords and phrases.

[1058] 2. Database Matching:

[1059] Based on the analysis results, the database of past failures is checked to search for similar cases. This is done using the Database module.

[1060] 3. Applying statistical data and estimating the cause of failure:

[1061] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[1062] 4. Generate a solution:

[1063] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution. The optimal solution is generated using a prompt sentence as input. For example, the following prompt sentence is used:

[1064] - Example prompt: "My smart lock is unresponsive. The situation is as follows: dead battery. Please suggest the best solution."

[1065] If necessary, a list of parts required for the solution is automatically generated.

[1066] 5. Solution suggestions:

[1067] The generated solution and required parts list are sent to the user's device, and the presentation content is optimized based on the analysis results of the emotion recognition engine.

[1068] 6. Receiving and recording feedback:

[1069] It receives feedback from users, analyzes the data, and records it in a database. Emotional data is also recorded at the same time and used for future analysis.

[1070] The role of emotion recognition engines

[1071] The emotion recognition engine is responsible for:

[1072] 1. User Emotion Recognition:

[1073] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[1074] 2. Optimizing the solution:

[1075] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[1076] 3. Recording emotional data:

[1077] The recognized user emotion data is recorded in a database and used for future analysis.

[1078] In this way, the system allows users to quickly and appropriately resolve problems, and also provides a response that takes emotion into consideration.

[1079] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1080] Step 1:

[1081] The user enters the defect information

[1082] The user enters details of the security hardware problem (e.g., a malfunction of a smart lock) in text format into an input form on the device, and the entered information is sent to the server.

[1083] (Input): User's defect information

[1084] (Output): Raw data sent to the server

[1085] Step 2:

[1086] The server receives the failure information

[1087] The server receives the defect information sent from the user's device. Since the received data cannot be analyzed as is, it is analyzed using a natural language processing (NLP) engine.

[1088] (Input): Malfunction information sent from the device

[1089] (Output): Defect information to be passed to natural language processing

[1090] Step 3:

[1091] Analysis of defect information using natural language processing

[1092] The server's NLP engine analyzes the received malfunction information and extracts important keywords and phrases, such as "smart lock," "unresponsive," and "nothing happens when I press the button."

[1093] (Input): Error information sent to the server

[1094] (Output): Parsed keywords and phrases

[1095] Step 4:

[1096] Emotion recognition engine for analyzing user emotions

[1097] The emotion recognition engine analyzes the user's emotions based on their input information and operational behavior (e.g., typing speed). For example, it can detect that the user is irritated based on their typing speed and the text.

[1098] (Input): User input information and operation behavior

[1099] (Output): Parsed user sentiment

[1100] Step 5:

[1101] Search for similar past cases from the database

[1102] The server searches a database for similar past cases based on the analyzed keywords and queries related cases.

[1103] (Input): Parsed keyword

[1104] (Output): Past similar cases

[1105] Step 6:

[1106] Calculating the probability of failure causes using statistical data

[1107] Based on the search results, the server uses statistical data to calculate the probability of the cause of the failure. For example, it estimates that the smart lock's battery is most likely dead.

[1108] (Input): Past similar cases

[1109] (Output): Probability of each failure cause

[1110] Step 7:

[1111] Estimation of the cause of failure

[1112] The server estimates the most likely cause of the failure based on the calculated occurrence probability.

[1113] (Input): Probability of each failure cause

[1114] (Output): Estimated cause of failure

[1115] Step 8:

[1116] Generating solutions using generative AI models

[1117] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure. For example, the server inputs a prompt statement such as "The smart lock is unresponsive. The situation is as follows: the battery is dead. Please suggest the optimal solution." into the generative AI model to obtain a solution.

[1118] (Input): Estimated cause of failure and prompt statement

[1119] (Output): The optimal solution generated

[1120] Step 9:

[1121] Provide the user with a solution

[1122] The server optimizes the generated solution based on the user's emotions and sends it to the user's device. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[1123] (Input): Generated solutions and analyzed user sentiment

[1124] (Output): Optimized solution

[1125] Step 10:

[1126] Receiving feedback and recording it in a database

[1127] The user tries the proposed solutions and inputs the results into the terminal and sends them to the server. The server receives this feedback and records it in a database. The analyzed emotional data is also recorded at the same time.

[1128] (Input): User feedback

[1129] (Output): Feedback and emotion data recorded in a database

[1130] The above processing steps allow the user to quickly and appropriately resolve the problem, and also realize a response that takes into consideration the user's feelings.

[1131] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1132] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1133] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1134] [Third embodiment]

[1135] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1136] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1137] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1138] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1139] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1140] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1141] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1142] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1143] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1144] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1145] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1146] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1147] The present invention is a system that allows a user to input product defect information, identifies the cause of the failure based on that information, and proposes an optimal solution. Specific embodiments of this system will be described below.

[1148] Overall system configuration

[1149] The system mainly consists of a user's terminal, a server, and a database. The user uses the terminal to input information about the problem, which is then sent to the server. The server processes the information, compares it with the database, deduces the cause of the problem, and generates a solution.

[1150] User Roles

[1151] The user mainly performs the following operations:

[1152] 1. Entering defect information: The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form.

[1153] 2. Confirmation of the solution: The user confirms the solution sent from the server on the device.

[1154] 3. Implementing the solution and providing feedback: The user carries out the repair or part replacement based on the proposed solution, and then enters the results into the terminal again and sends them to the server.

[1155] Server Roles

[1156] The server is the central part of the system and performs the following tasks:

[1157] 1. Data reception and analysis: The server receives the defect information sent from the user's device and analyzes it using a natural language processing engine. Important keywords and patterns are extracted to identify the problem.

[1158] 2. Database collation: Based on the analysis results, the results are collated with a database of past failures, similar cases are searched for, and statistical data is generated.

[1159] 3. Estimation of the cause of failure: Based on the statistical data obtained from the search results, the probability of occurrence of the cause of failure is calculated and the cause with the highest probability is estimated.

[1160] 4. Solution Generation: Based on the estimated cause of the failure, generative AI is used to generate the optimal solution, as well as a list of required replacement parts.

[1161] 5. Send Proposal: Send the generated solution and parts list to the user's device.

[1162] 6. Receiving and recording feedback: Receive feedback from users and record it in a database. This data will be used to identify the cause of future failures.

[1163] The role of the database

[1164] The database stores the following information:

[1165] 1. Past failure data: Records of past failures and their responses.

[1166] 2. Statistical data: Basic data for calculating the probability of occurrence of failure causes.

[1167] 3. Solutions and parts list: Information on the best solutions and required parts for each estimated cause of failure.

[1168] 4. Feedback information: Results after repair and feedback information from users.

[1169] Specific examples

[1170] For example, if your air conditioner is not cooling properly:

[1171] 1. User operation: The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[1172] 2. Server analysis: The server extracts the keywords "not cooling," "1 hour," and "air conditioning not working."

[1173] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[1174] 4. Application of statistical data: Based on statistical data obtained from past cases, the probability of refrigerant shortage, compressor failure, and filter clogging occurring is calculated.

[1175] 5. Estimation of the cause of failure: It is estimated that the most likely cause is a lack of refrigerant.

[1176] 6. Solution generation: The AI ​​generates a solution, adding refrigerant and cleaning the filter, and presents it to the user.

[1177] 7. User execution and feedback: The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[1178] 8. Server Recording: The server records this feedback in a database for future analysis.

[1179] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a failure, even without specialized knowledge.

[1180] The processing flow will be explained below.

[1181] Step 1:

[1182] The user accesses the device's input form and enters product defect information, including the product name, model number, detailed problem description, the circumstances under which the problem occurred, and any error messages.

[1183] Step 2:

[1184] When the user has finished entering the defect information in the input form, the terminal sends the information to the server, where it is converted into an appropriate data format before being sent.

[1185] Step 3:

[1186] The server receives the defect information sent from the device and starts a natural language processing (NLP) engine to analyze the received data.

[1187] Step 4:

[1188] The server uses a natural language processing engine to analyze the malfunction information and extract important keywords and phrases, such as "not cooling," "after one hour," and "air conditioning not working."

[1189] Step 5:

[1190] The server then searches the database based on the analyzed keywords and phrases to find similar past cases. For example, it extracts cases that include past reports of the product not cooling down.

[1191] Step 6:

[1192] The server uses statistical data based on the data obtained from the search results to calculate the probability of occurrence of each cause of failure, such as refrigerant shortage, compressor failure, and filter clogging.

[1193] Step 7:

[1194] The server estimates the most likely cause of the failure based on the calculated probability of occurrence. For example, it estimates that a refrigerant shortage is the most likely cause.

[1195] Step 8:

[1196] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, such as adding refrigerant or cleaning the filter.

[1197] Step 9:

[1198] The server compiles the generated solution and a list of required replacement parts and sends it to the user's device, where the user can view it.

[1199] Step 10:

[1200] The user then performs repairs or part replacements based on the proposed solutions, for example adding refrigerant and cleaning the filters.

[1201] Step 11:

[1202] The user inputs the results of the repairs they performed into the terminal and sends them to the server, for example, sending feedback such as "After adding the refrigerant, the air conditioner worked normally."

[1203] Step 12:

[1204] The server receives feedback from users, analyzes the data, and records it in a database. This information is used to identify the cause of future failures and improve solutions.

[1205] Example 1

[1206] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1207] Conventional systems for identifying the cause of a failure and presenting solutions have had difficulty accurately analyzing failure information from users and quickly and effectively presenting appropriate solutions. Furthermore, because user feedback could not be reflected in the next failure analysis, the learning and evolution of the entire system was delayed. This resulted in delays in troubleshooting product failures and reduced user convenience.

[1208] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1209] In this invention, the server includes means for receiving failure information, means for analyzing the information using natural language processing, means for searching for similar cases in a database, means for estimating the cause of the failure using statistical data, means for generating an optimal solution using a generative model, means for presenting the solution to the user, and means for receiving feedback and recording the feedback in the database. This makes it possible to quickly and accurately identify the cause of the failure based on detailed failure information from the user and provide an optimal solution. Furthermore, by reflecting user feedback in the system, learning and evolution of the entire system are promoted.

[1210] "Defect information" is information in which the user specifically and in detail describes abnormalities or problems with the product.

[1211] "Natural language processing" is a technology that uses computers to analyze natural language and understand its meaning and structure.

[1212] A "database" is a collection of data that systematically stores past failure cases and related data and manages them in a searchable format.

[1213] "Similar cases" are records of similar malfunctions or problems that have occurred in the past.

[1214] "Statistical data" refers to data for calculating the probability of occurrence of a failure cause based on numerical data obtained from a database.

[1215] The "cause of failure" is the fundamental cause of a product defect or abnormality.

[1216] A "generative model" is an algorithm that uses AI technology to generate an appropriate solution from given data.

[1217] A "solution" is a specific procedure or action proposed to solve a problem based on the estimated cause of the failure.

[1218] "Replacement Part" means a part or component required to resolve a defect.

[1219] "Feedback" is the evaluation or information provided by a user based on the results or experience of the solution they have implemented.

[1220] A "server" is a computer system that receives, analyzes, and stores data via a network, and provides the processed results to a client terminal.

[1221] A "user terminal" is a device such as a computer or smartphone that a user uses to input information or receive data from a server.

[1222] Overall system configuration

[1223] The system of the present invention mainly consists of a user terminal, a server, and a database. The user inputs fault information using the terminal, and the information is sent to the server. The server processes the information, compares it with the database, deduces the cause of the fault, and generates a solution.

[1224] Role of user terminal

[1225] The user terminal is the device that inputs information about the problem and receives and checks solutions from the server. Specifically, the user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the terminal. This information is then sent to the server. User terminals can be PCs or smartphones.

[1226] Server Roles

[1227] The server is the central part of the system and performs a variety of processes. First, it receives the failure information sent from the user's device. Next, it analyzes the information using a natural language processing engine (such as SpaCy or BERT) and extracts important keywords, which allows it to identify the problem. The server then searches a database based on the extracted keywords and refers to similar cases. The server estimates the cause of the failure based on statistical data obtained from past failure data. It then uses a generative AI model (such as a GPT model) to generate an optimal solution. The server also generates a list of required replacement parts and sends this to the user's device. Finally, it receives feedback from the user and records it in a database.

[1228] The role of the database

[1229] The database stores past failure cases and their corresponding records, statistical data, solutions, replacement parts lists, feedback information, etc. This provides the server with the basic data to quickly identify the cause of the failure and generate an appropriate solution.

[1230] Specific examples

[1231] For example, the following shows the flow when the user inputs "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[1232] 1. User operation: The user enters the defect information into the input form on the terminal.

[1233] 2. Server analysis: The server extracts the keywords "not cooling," "one hour later," and "air conditioning not working."

[1234] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[1235] 4. Estimation of failure causes: Calculate the probability of occurrence of causes such as refrigerant shortage, compressor failure, and filter blockage from statistical data. Estimate the cause with the highest probability (e.g., refrigerant shortage).

[1236] 5. Solution generation: The AI ​​generates solutions, "add refrigerant" and "clean the filter," and presents them to the user.

[1237] 6. User execution and feedback: The user executes the proposed solution and enters the results into the terminal.

[1238] 7. Server Recording: The server receives the feedback and records it in a database.

[1239] Prompt Sentence Examples

[1240] "My air conditioner isn't cooling. If it stops working after about an hour of use, what could be the cause of the problem? Please tell me what to do."

[1241] This system allows users to quickly and accurately identify the cause of a failure and find the optimal solution simply by entering detailed information about the product's malfunction.In addition, accumulating feedback from users will improve the analytical capabilities and scope of application of the entire system.

[1242] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1243] Step 1:

[1244] The user inputs the defect information.

[1245] Input: Product name, model number, problem description, circumstances under which it occurred, error messages, etc.

[1246] Specific operation: The user enters information such as "The air conditioner is not cooling. After about an hour of use, the cooling stops working" into the input form on the device.

[1247] Output: The entered defect information is saved on the terminal and later sent to the server.

[1248] Step 2:

[1249] The device sends the defect information to the server.

[1250] Input: The defect information entered by the user.

[1251] Specific operation: The device automatically sends the saved defect information to the server's API.

[1252] Output: The failure information is passed to the server.

[1253] Step 3:

[1254] The server receives and analyzes the defect information.

[1255] Input: The error information sent from the device.

[1256] Specific operation: The server uses a natural language processing engine (e.g., SpaCy, BERT) to analyze the malfunction information and extract important keywords (e.g., "not cooling," "after one hour," "air conditioning not working").

[1257] Output: Extracted keyword list.

[1258] Step 4:

[1259] The server searches the database based on the keyword list.

[1260] Input: Extracted keyword list.

[1261] Specific operation: The server searches the database and retrieves past similar cases.

[1262] Output: A list of similar cases.

[1263] Step 5:

[1264] The server uses statistical data from similar cases to estimate the cause of the failure.

[1265] Input: A list of similar cases.

[1266] Specific operation: Based on the statistical data, the server calculates the occurrence probability of each cause of failure (e.g., refrigerant shortage, compressor failure, clogged filter).

[1267] Output: Probability of occurrence of each failure cause.

[1268] Step 6:

[1269] The server uses a generative model based on the most probable causes of failure to generate an optimal solution.

[1270] Input: Probability of occurrence of failure cause.

[1271] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate the optimal solution for the most probable cause of the failure, for example, "add refrigerant" and "clean the filter."

[1272] Output: A list of solutions.

[1273] Step 7:

[1274] The server transmits the generated solution and a list of required replacement parts to the user terminal.

[1275] Input: List of solutions, list of replacement parts needed.

[1276] Specific operation: The server compiles the solution and replacement parts list and sends them to the user terminal.

[1277] Output: Solution and parts list displayed on the user's terminal.

[1278] Step 8:

[1279] The user executes the solution and enters the results into the terminal.

[1280] Input: Solution and replacement parts list sent from the server.

[1281] Specific actions: The user takes problem-solving actions according to the presented steps and inputs the results (e.g., success or failure of the solution) into the terminal.

[1282] Output: The feedback information is stored on the device and later sent to the server.

[1283] Step 9:

[1284] The server receives the feedback from the user and records it in a database.

[1285] Input: Feedback information entered by the user into the device.

[1286] Specific operation: The server receives the feedback sent by the device and stores it in a database.

[1287] Output: Feedback information is recorded in a database.

[1288] Through these steps, users can input product defect information, the server performs analysis and estimation, and provides the user with the optimal solution. By incorporating feedback, the accuracy and efficiency of the entire system are improved.

[1289] (Application example 1)

[1290] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1291] Conventional fault diagnosis systems estimate the cause of a fault based on fault information entered by the user and provide solutions. However, as factories become more automated, a faster and more advanced response is required. However, these systems rely on manual user operation, making it difficult to respond in real time or collect feedback. In particular, when a factory robot breaks down, the manual input of fault information is cumbersome, making it difficult to respond quickly. Therefore, a new system that can diagnose faults and provide a rapid response for factory robots is needed.

[1292] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1293] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the received defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for calculating the probability of occurrence of a failure cause using statistical data from the search results, means for estimating the failure cause based on the calculated probability, means for generating an optimal solution based on the estimated failure cause, means for presenting the generated solution to the user, means for coordinating with a factory robot to execute the presented solution, and means for receiving feedback from the user via the factory robot and recording it in the database, thereby enabling factory robot failure diagnosis and rapid response.

[1294] "User" refers to the factory technician or operator who inputs the defect information and implements the proposed solution.

[1295] "Fault information" is information that provides detailed explanations of failures and abnormalities that occur in factory robots and other equipment.

[1296] "Means for receiving" refers to a device or program that has the function of sending and receiving defect information entered by a user to a server.

[1297] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning.

[1298] "Means for analyzing" refers to a program or device that has the function of analyzing received defect information using natural language processing technology.

[1299] "Database" refers to a computer system that stores and manages past failure cases and solutions.

[1300] "Searching means" refers to a program that has the function of searching for similar past cases from a database based on the analysis results.

[1301] "Statistical data" refers to data for calculating the probability of occurrence of a specific cause of failure based on past cases.

[1302] "Means for calculating" refers to a program that has the function of calculating the probability of occurrence of a failure cause using statistical data.

[1303] "Cause of failure" refers to the factors that cause a specific malfunction or abnormality to occur.

[1304] "Means for estimating" refers to a program that has the function of estimating the most likely cause of a failure based on the calculated probability of occurrence.

[1305] "Solution method" refers to the procedure for instructing the optimal repair or countermeasure for the estimated cause of the failure.

[1306] "Means for generating" refers to a program that has the function of automatically generating a solution based on the estimated cause of the failure.

[1307] "Presentation means" refers to a display device or program for showing the generated solution to the user.

[1308] "Means for collaboration" refers to devices or programs that have the function of communicating with factory robots and instructing their operations in order to implement the proposed solution.

[1309] "Feedback" refers to the results and opinions of users who implement the solutions provided.

[1310] A "factory robot" refers to a mechanical device that performs work automatically within a factory.

[1311] "Means for recording" refers to a program that has the function of storing feedback information in a database.

[1312] MODE FOR CARRYING OUT THE INVENTION

[1313] This invention relates to a fault diagnosis system for factory robots that estimates the cause of a fault based on malfunction information provided by the user and proposes the optimal solution. This system is composed of a user terminal, a server, and a database, and realizes factory automation and efficient fault response.

[1314] Hardware and software used

[1315] An embodiment of the present invention uses the following hardware and software:

[1316] Hardware: smartphones, tablets, factory robots, servers

[1317] Software: Programming languages ​​(Python, JavaScript), databases (MySQL, PostgreSQL), AI models (GPT-3, BERT), NLP engines (spaCy, NLTK), cloud services (AWS, Google Cloud)

[1318] Data processing and calculation flow

[1319] 1. User operations

[1320] Users use smartphones or tablets to input malfunction information for factory robots. The malfunction information includes the product name, model number, details of the problem, the circumstances under which the problem occurred, and error messages. This information is then sent from the user's device to the server.

[1321] 2. Data Reception and Analysis

[1322] The server receives the defect information sent from the user terminal. The received defect information is analyzed using a natural language processing engine (e.g., spaCy) to extract important keywords and patterns.

[1323] 3. Database Verification

[1324] Based on the analyzed data, the server searches the database for similar past cases, referencing past failure cases and response records, and generates statistical data based on the search results.

[1325] 4. Estimation of the cause of the failure

[1326] The server uses the generated statistical data to calculate the probability of occurrence of each cause of failure, and estimates the cause with the highest probability.

[1327] 5. Solution Generation

[1328] Based on the estimated cause of the failure, a generative AI model (e.g., GPT-3) is used to generate an optimal solution, along with a list of required replacement parts.

[1329] 6. Presentation to the User

[1330] The server sends the generated solution and replacement part list to the user's terminal and presents them to the user, who then executes the proposed solution.

[1331] 7. Collecting and recording feedback

[1332] The results of the user's execution are sent back to the server from the terminal, and the server receives and records this feedback information. This data is stored in a database and will be used for future failure analysis.

[1333] Specific examples

[1334] For example, if a robot arm in a factory stops working, the user inputs the following information about the problem into a tablet: "The robot arm won't move. There's an abnormal noise. Error code 5678 is displayed." Based on this information, the server analyzes it using an NLP engine and performs a database search. Statistical data is generated from the search results, and after calculating the probability of occurrence, it is estimated that the most likely cause is "motor failure." A generative AI model (e.g., GPT-3) is used to suggest "replacing the motor" as a solution. The user replaces the motor and re-enters the results into the system. This series of steps enables factory robot failures to be handled quickly and accurately.

[1335] Prompt Sentence Examples

[1336] "We are experiencing the following problems with our factory robot: The robot arm does not move, there is an abnormal noise, and error code 5678 is displayed." Please suggest the best solution based on this situation."

[1337] As a result, the present invention realizes rapid and accurate failure analysis and presentation of solutions based on the defect information input by the user.

[1338] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1339] Program processing steps and specific operations

[1340] Step 1:

[1341] The terminal receives the defect information input by the user. The user inputs the product name, model number, details of the defect, the circumstances when the defect occurred, and an error message into the input form on the terminal. This input data is sent from the terminal to the server.

[1342] Input: Product name, model number, problem details, circumstances under which it occurred, error message

[1343] Output: Defect information data sent to the server

[1344] Step 2:

[1345] The server receives the defect information data sent from the device. The received data is analyzed using a natural language processing engine (e.g., spaCy). During the analysis, important keywords and patterns are extracted and used in the next step, a database search.

[1346] Input: Received defect information data

[1347] Output: Extracted keywords and patterns

[1348] Step 3:

[1349] The server then searches the database based on the analysis results. The search locates similar cases from past failure cases and response records. In this process, a list of related cases and statistical data is generated.

[1350] Input: Extracted keywords or patterns

[1351] Output: List of similar cases and statistical data

[1352] Step 4:

[1353] The server uses the generated statistical data to calculate the probability of occurrence of the cause of the failure, and estimates the most likely cause of the failure based on the calculated probability of occurrence.

[1354] Input: List of similar cases and statistical data

[1355] Output: Estimated cause of failure and its probability of occurrence

[1356] Step 5:

[1357] The server uses a generative AI model (e.g., GPT-3) to generate an optimal solution based on the estimated cause of the failure, while simultaneously generating a list of required replacement parts.

[1358] Input: Estimated cause of failure and its probability of occurrence

[1359] Output: Solution and replacement part list

[1360] Step 6:

[1361] The server sends the generated solutions and a list of replacement parts to the user's terminal and presents them to the user, who then checks and implements the solutions.

[1362] Input: Solution and replacement part list

[1363] Output: A list of solutions and replacement parts sent to the user's device

[1364] Step 7:

[1365] The feedback obtained based on the solution method the user performs is sent from the terminal to the server. The server receives and records this feedback and stores it in a database. This feedback data is used for future fault analysis and resolution.

[1366] Input: Feedback of the results of the user's actions

[1367] Output: Feedback information recorded in a database

[1368] This allows for rapid and accurate diagnosis and resolution of factory robot failures through a series of steps.

[1369] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1370] The present invention is a system that combines a system in which a user inputs product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution, with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[1371] Overall system configuration

[1372] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[1373] User Roles

[1374] The user mainly performs the following operations:

[1375] 1. Enter the defect information:

[1376] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the device.

[1377] The emotion engine also simultaneously monitors the user's input information and behavior and analyzes emotions.

[1378] 2. Check the solution:

[1379] The user checks the solution sent from the server on the terminal.

[1380] Content optimized by the emotion engine is presented.

[1381] 3. Response implementation and feedback:

[1382] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[1383] Server Roles

[1384] The server is the center of the system and performs the following tasks:

[1385] 1. Data reception and analysis:

[1386] The server receives the defect information sent from the user terminal and launches a natural language processing (NLP) engine to analyze the received data.

[1387] Extract important keywords and phrases.

[1388] 2. Database Matching:

[1389] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[1390] 3. Applying statistical data and estimating the cause of failure:

[1391] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[1392] 4. Generate a solution:

[1393] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution.

[1394] It also generates a list of required replacement parts.

[1395] 5. Solution suggestions:

[1396] The generated solution and the required parts list are sent to the user's terminal.

[1397] The content presented is optimized based on the analysis results of the emotion engine.

[1398] 6. Receiving and recording feedback:

[1399] Receive feedback from users, analyze the data and record it in a database.

[1400] The role of the emotional engine

[1401] The Emotion Engine is responsible for:

[1402] 1. User Emotion Recognition:

[1403] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[1404] 2. Optimizing the solution:

[1405] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[1406] 3. Recording emotional data:

[1407] The recognized user emotion data is recorded in a database and used for future analysis.

[1408] The role of the database

[1409] The database stores the following information:

[1410] 1. Past failure data:

[1411] Past failure cases and their corresponding records.

[1412] 2. Statistics:

[1413] Basic data for calculating the probability of occurrence of a failure cause.

[1414] 3. Solution and parts list:

[1415] Information on the optimal solution and required parts for each estimated cause of failure.

[1416] 4. Feedback Information:

[1417] Repair results and user feedback information.

[1418] 5. Emotional Data:

[1419] Record the results of user sentiment analysis and use the data for future reference.

[1420] Specific examples

[1421] For example, if your air conditioner is not cooling properly:

[1422] 1. User Action:

[1423] The user types into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes that the user is frustrated based on the input data and typing speed.

[1424] 2. Server Analysis:

[1425] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working." The emotion engine analyzes the speed of input and the parts that are emphasized.

[1426] 3. Database Search:

[1427] The server searches the database based on these keywords and refers to similar past cases.

[1428] 4. Statistical Data Application:

[1429] Based on the statistical data obtained from the search, the probability of refrigerant shortage, compressor failure, and filter blockage occurring is calculated.

[1430] 5. Estimation of the cause of the failure:

[1431] The most likely cause is a lack of refrigerant.

[1432] 6. Generate a solution:

[1433] The solution is to add refrigerant and clean the filter. The emotion engine reflects the user's frustration and presents the solution in a format that is as simple and quick to respond as possible.

[1434] 7. User execution and feedback:

[1435] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[1436] 8. Server Records:

[1437] The server records this feedback in a database, along with emotional data, for future analysis.

[1438] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a malfunction without requiring specialized knowledge, and also enables responses that take into consideration the user's feelings.

[1439] The processing flow will be explained below.

[1440] Step 1:

[1441] The user accesses the input form on the device and enters product defect information, including the product name, model number, detailed problem description, circumstances at the time of the occurrence, and error message. As the user types, the emotion engine analyzes the user's typing speed and writing style to evaluate the user's emotions.

[1442] Step 2:

[1443] When the user completes the input and presses the send button, the device sends the defect information and emotion data to the server, including metadata reflecting the user's emotion.

[1444] Step 3:

[1445] The server launches a natural language processing engine to analyze the malfunction information received from the device. The server extracts important keywords and phrases to identify the problem. For example, it extracts keywords such as "not cooling," "one hour later," and "air conditioning not working."

[1446] Step 4:

[1447] The server searches a database for similar cases based on the analyzed keywords and phrases. The server checks whether a similar problem has already been reported.

[1448] Step 5:

[1449] Based on the search results, the server uses statistical data to calculate the probability of each cause of failure, such as a refrigerant shortage, compressor failure, or clogged filter.

[1450] Step 6:

[1451] The server evaluates the probability of occurrence of each cause and estimates the most likely cause of the failure, for example, a refrigerant shortage.

[1452] Step 7:

[1453] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, generating specific steps to add refrigerant and clean the filter.

[1454] Step 8:

[1455] The server compiles the generated list of solutions and replacement parts and optimizes it based on the user's emotional data, for example prioritizing simple and fast solutions if the user is frustrated.

[1456] Step 9:

[1457] The server sends the optimized solution and parts list to the user's device, where the user can review and understand the specific steps to take.

[1458] Step 10:

[1459] The user performs repairs or part replacements based on the proposed solutions, for example, the user adds refrigerant and cleans the filter.

[1460] Step 11:

[1461] The user inputs the results of the repair into the terminal and sends them to the server. For example, the user sends feedback such as, "After adding the refrigerant, the air conditioner worked normally."

[1462] Step 12:

[1463] The server receives feedback from users, analyzes the data, and records it in a database. At the same time, it also records the user's emotional data for future analysis.

[1464] In this way, this system is not only able to quickly and appropriately identify the cause of a malfunction and propose a solution, but also to respond in a way that takes the user's feelings into consideration.

[1465] Example 2

[1466] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1467] Conventional fault diagnosis systems have difficulty efficiently analyzing malfunction information from users and identifying the cause of the malfunction. They also lack the ability to recognize users' emotions and stress levels and provide optimal solutions, potentially resulting in a poor user experience. Furthermore, the solutions they generate often do not provide a list of replacement parts needed, making it difficult for users to prepare for repairs. These issues make it difficult to provide prompt and appropriate responses.

[1468] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: means for receiving malfunction information input by a user; means for analyzing the received malfunction information using natural language processing; means for searching a database for similar past cases based on the analysis results; means for calculating the occurrence probability of a malfunction cause using statistical data from the search results; means for estimating the malfunction cause based on the calculated probability; means for generating an optimal solution based on the estimated malfunction cause using a generative AI model; means for presenting the generated solution to the user; means for analyzing the user's input information and behavior and recognizing emotions; and means for optimizing the presented solution based on the recognized emotions. This makes it possible to quickly and appropriately identify the malfunction cause and provide an efficient solution. Furthermore, it is possible to respond in consideration of the user's emotions, thereby improving the user experience.

[1469] A "user" is an entity that uses the system to input product defect information and implement suggested solutions.

[1470] A "terminal" is an electronic device that a user uses to input defect information, check solutions, and provide feedback on the results of the solutions.

[1471] The "server" is a central processing unit that receives defect information from users, analyzes it, and provides the information.

[1472] "Problem information" is information that indicates details of a problem that has occurred in a product, and is entered by the user.

[1473] "Natural language processing" is a technology that allows computers to understand human language.

[1474] "Analysis" is the process of interpreting defect information and extracting important keywords and phrases.

[1475] A "database" is an information system that stores past failure cases, statistical data, solutions, replacement part information, and so on.

[1476] "Statistical data" is numerical information such as the probability of an event occurring, calculated based on past data.

[1477] "Cause of failure" refers to the factor that causes the product to not function properly.

[1478] "Occurrence probability" is a numerical value that indicates the degree of possibility that a particular cause of failure will occur.

[1479] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate optimal solutions from specific inputs.

[1480] A "solution" is a specific procedure or measure to resolve a specific cause of a failure.

[1481] An "emotion engine" is a system that analyzes the user's input information and operational behavior and recognizes the user's emotions.

[1482] "Emotions" refer to the psychological state that a user feels while using a system, and include feelings of frustration and satisfaction.

[1483] "Recognition" is the process by which the system identifies the user's emotions.

[1484] "Optimization" means adjusting the solution presentation based on the user's feelings to make it more acceptable.

[1485] The "list of replacement parts" is a list of parts required to eliminate the cause of the failure.

[1486] "Feedback" is information that a user provides to the system as a result of implementing a solution.

[1487] This invention combines a system that allows users to input product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution with an emotion engine that recognizes the user's emotions. This system is implemented using the following hardware and software.

[1488] Overall system configuration

[1489] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[1490] User Roles

[1491] The user uses the terminal to:

[1492] 1. Enter the defect information:

[1493] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form. The emotion engine simultaneously monitors the user's input speed, choices, etc., and analyzes their emotions.

[1494] 2. Check the solution:

[1495] The user checks the solution sent from the server on their device, and the solution optimized by the emotion engine is presented.

[1496] 3. Response implementation and feedback:

[1497] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[1498] Server Roles

[1499] The server is the center of the system and performs the following tasks:

[1500] 1. Data reception and analysis:

[1501] The server receives the defect information sent from the user's device and analyzes it using a natural language processing (NLP) engine to extract important keywords and phrases.

[1502] 2. Database Matching:

[1503] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[1504] 3. Applying statistical data and estimating the cause of failure:

[1505] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each failure cause, and the failure cause with the highest probability is estimated.

[1506] 4. Generate a solution:

[1507] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution, along with a list of required replacement parts.

[1508] 5. Solution suggestions:

[1509] The generated solution and required parts list are sent to the user's device, and the content presented is optimized based on the analysis results of the emotion engine.

[1510] 6. Receiving and recording feedback:

[1511] Receive feedback from users, analyze the data and record it in a database.

[1512] The role of the emotional engine

[1513] The Emotion Engine is responsible for:

[1514] 1. User Emotion Recognition:

[1515] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[1516] 2. Optimizing the solution:

[1517] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[1518] 3. Recording emotional data:

[1519] The recognized user emotion data is recorded in a database and used for future analysis.

[1520] The role of the database

[1521] The database stores the following information:

[1522] 1. Past failure data:

[1523] Store records of past failure cases and their responses.

[1524] 2. Statistics:

[1525] Contains basic data for calculating the probability of occurrence of a failure cause.

[1526] 3. Solution and parts list:

[1527] Save information on optimal solutions and required parts.

[1528] 4. Feedback Information:

[1529] Stores the results of repairs and feedback information from users.

[1530] 5. Emotional Data:

[1531] Records the results of user sentiment analysis and includes data for future reference.

[1532] Specific examples

[1533] For example, if your air conditioner is not cooling properly:

[1534] 1. User Action:

[1535] The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes the user's frustration.

[1536] 2. Server Analysis:

[1537] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working."

[1538] 3. Database Search:

[1539] The server searches the database based on these keywords and refers to similar past cases.

[1540] 4. Statistical Data Application:

[1541] Based on statistical data, the probability of refrigerant shortage, compressor failure, and filter clogging is calculated.

[1542] 5. Estimation of the cause of the failure:

[1543] The most likely cause is a lack of refrigerant.

[1544] 6. Generate a solution:

[1545] A solution is generated suggesting adding refrigerant and cleaning the filter. The emotion engine reflects the user's frustration and presents the solution in a simple, immediate format.

[1546] 7. User execution and feedback:

[1547] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[1548] 8. Server Records:

[1549] The server records the feedback information and emotion data in a database.

[1550] Prompt Sentence Examples

[1551] "We have received a report that the air conditioner is not cooling. It has been observed that the cooling stops working after an hour of use and the user seems to be frustrated. Please suggest the cause of the problem and how to fix it."

[1552] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1553] Step 1: Enter your defect information

[1554] The user enters information about the problem, such as the product name, model number, details of the problem, the circumstances under which it occurred, and any error messages, into the device's input form. Once the input is complete, the user presses the send button. The emotion engine simultaneously monitors the user's input speed and operating behavior to recognize emotions. For example, if the user enters "The air conditioner isn't cooling. After about an hour of use, the cooling stopped working," the emotion of irritation will be recognized.

[1555] Input: Product name, model number, problem details, occurrence status, error message

[1556] Output: Input defect information, user emotion data

[1557] Step 2: Submit your defect report

[1558] The device packages the entered defect information in JSON format and sends it to the server. The data is sent to the endpoint as a POST request.

[1559] Input: Entered defect information, user emotion data

[1560] Output: Defect information sent to the server, user emotion data

[1561] Step 3: Data reception and analysis

[1562] The server receives the malfunction information sent from the device. The received data is passed to a natural language processing (NLP) engine, which extracts important keywords and phrases. For example, keywords such as "not cooling," "after one hour," and "air conditioning not working" are extracted.

[1563] Input: Error information received from the device

[1564] Output: Extracted keywords and phrases

[1565] Step 4: Search the database

[1566] The server searches the database based on the extracted keywords and phrases. It compares them with past malfunction cases and extracts similar cases. For example, it searches for past malfunction cases related to "not cooling" and retrieves cases such as refrigerant shortages and clogged filters.

[1567] Input: Extracted keywords and phrases

[1568] Output: Similar cases

[1569] Step 5: Identify the cause of the failure

[1570] The server applies statistical data based on similar cases retrieved from the database to calculate the probability of occurrence of each failure cause. Based on the calculated probabilities, it estimates the most likely cause of the failure. For example, it estimates that a refrigerant shortage is the most likely cause.

[1571] Input: Similar cases

[1572] Output: Estimated cause of failure, occurrence probability

[1573] Step 6: Generate a solution

[1574] The server runs a generative AI model to generate solutions based on the estimated cause of the failure, such as adding refrigerant and cleaning the filter, and also generates a list of required replacement parts.

[1575] Input: Estimated cause of failure, occurrence probability

[1576] Output: Generated solution, replacement parts list

[1577] Step 7: Optimize and present the solution

[1578] The server optimizes the solution based on the user's emotions, recognized using an emotion engine. For example, if the user is frustrated, the server presents a solution as a simple and quick procedure. The server then sends the optimized solution and parts list to the user's device.

[1579] Input: Generated solutions, replacement parts list, user sentiment data

[1580] Output: Optimized solution, replacement parts list

[1581] Step 8: Receive and record feedback

[1582] The user repairs or replaces parts based on the proposed solutions. The results are entered into the terminal and sent as feedback to the server. The server receives this feedback information, analyzes it, and records it in a database. Emotional data is also recorded.

[1583] Input: Repair execution results, feedback information

[1584] Output: Feedback information and emotion data recorded in the database

[1585] (Application example 2)

[1586] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1587] Conventional product defect analysis systems do not take into account the user's emotions, and therefore are unable to reduce the stress and frustration experienced when users input defect information or receive solutions. In addition, because generative AI models are not adequately used to propose optimal solutions, the solutions presented are not always effective, making it difficult to quickly resolve problems.

[1588] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1589] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for recognizing emotions, means for generating an optimal solution, and means for presenting the generated solution to the user. This reduces stress by taking the user's emotions into consideration, and enables effective and rapid problem solving using a generative AI model.

[1590] The "means for receiving defect information input by the user" is a function for transmitting information about product or system defects input by the user through a terminal to the server, and for the server to receive this information.

[1591] "Means for analyzing using natural language processing" refers to a technology that receives defect information entered by a user as text data, analyzes the text data, and extracts important keywords and phrases.

[1592] The "means for searching for similar past cases from a database" is a function for searching for similar past recorded cases of defects in a database based on analyzed defect information.

[1593] "Means for recognizing emotions" refers to technology that analyzes the user's input information and operation behavior, and recognizes the user's current emotions (for example, irritation, confusion, tension, etc.).

[1594] The "means for generating an optimal solution" is a function that uses a generative AI model to generate an optimal solution based on the analysis results and the estimated cause of the failure, and then proposes that solution.

[1595] The "means for presenting the generated solution to the user" is a function for displaying the generated solution to the user, and aims to present it in a format that is easy for the user to understand.

[1596] A "generative AI model" is an algorithm that uses artificial intelligence to generate new solutions or suggestions based on input information.

[1597] A "prompt" is a document containing instructions or questions that are input to a generative AI model, and serves as the basis for the AI ​​to propose the optimal solution.

[1598] The present invention is a system that allows a user to input malfunction information about security hardware (e.g., smart locks or surveillance cameras), identifies the cause of the malfunction based on that information, and proposes the optimal solution. It also combines a function that analyzes the user's emotions and provides optimal support based on those emotions. Specific embodiments of this system are described below.

[1599] Overall system configuration

[1600] The system mainly consists of a user's device, a server, a database, and an emotion recognition engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion recognition engine then analyzes the user's input information and behavior, recognizes their emotions, and optimizes the solution suggestions it presents.

[1601] User Roles

[1602] The user mainly performs the following operations:

[1603] 1. Enter the defect information:

[1604] The user enters details of the security hardware problem into the device's input form. For example, if the smart lock is unresponsive, the user can write, "The smart lock is unresponsive. Nothing happens when I press the button." The emotion recognition engine simultaneously monitors the user's input information and behavior and analyzes their emotions.

[1605] 2. Check the solution:

[1606] The user then checks the solution sent from the server on their device, and the content is optimized using an emotion recognition engine. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[1607] 3. Response implementation and feedback:

[1608] The user then takes action based on the proposed solution and inputs the results into the device, which is then sent to the server, which records this feedback in a database for future analysis.

[1609] Server Roles

[1610] The server is the center of the system and performs the following tasks:

[1611] 1. Data reception and analysis:

[1612] The server receives the defect information sent from the user device and starts a natural language processing engine (NLP engine) to analyze the received data. For example, it uses an NLPProcessor to extract important keywords and phrases.

[1613] 2. Database Matching:

[1614] Based on the analysis results, the database of past failures is checked to search for similar cases. This is done using the Database module.

[1615] 3. Applying statistical data and estimating the cause of failure:

[1616] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[1617] 4. Generate a solution:

[1618] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution. The optimal solution is generated using a prompt sentence as input. For example, the following prompt sentence is used:

[1619] - Example prompt: "My smart lock is unresponsive. The situation is as follows: dead battery. Please suggest the best solution."

[1620] If necessary, a list of parts required for the solution is automatically generated.

[1621] 5. Solution suggestions:

[1622] The generated solution and required parts list are sent to the user's device, and the presentation content is optimized based on the analysis results of the emotion recognition engine.

[1623] 6. Receiving and recording feedback:

[1624] It receives feedback from users, analyzes the data, and records it in a database. Emotional data is also recorded at the same time and used for future analysis.

[1625] The role of emotion recognition engines

[1626] The emotion recognition engine is responsible for:

[1627] 1. User Emotion Recognition:

[1628] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[1629] 2. Optimizing the solution:

[1630] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[1631] 3. Recording emotional data:

[1632] The recognized user emotion data is recorded in a database and used for future analysis.

[1633] In this way, the system allows users to quickly and appropriately resolve problems, and also provides a response that takes emotion into consideration.

[1634] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1635] Step 1:

[1636] The user enters the defect information

[1637] The user enters details of the security hardware problem (e.g., a malfunction of a smart lock) in text format into an input form on the device, and the entered information is sent to the server.

[1638] (Input): User's defect information

[1639] (Output): Raw data sent to the server

[1640] Step 2:

[1641] The server receives the failure information

[1642] The server receives the defect information sent from the user's device. Since the received data cannot be analyzed as is, it is analyzed using a natural language processing (NLP) engine.

[1643] (Input): Malfunction information sent from the device

[1644] (Output): Defect information to be passed to natural language processing

[1645] Step 3:

[1646] Analysis of defect information using natural language processing

[1647] The server's NLP engine analyzes the received malfunction information and extracts important keywords and phrases, such as "smart lock," "unresponsive," and "nothing happens when I press the button."

[1648] (Input): Error information sent to the server

[1649] (Output): Parsed keywords and phrases

[1650] Step 4:

[1651] Emotion recognition engine for analyzing user emotions

[1652] The emotion recognition engine analyzes the user's emotions based on their input information and operational behavior (e.g., typing speed). For example, it can detect that the user is irritated based on their typing speed and the text.

[1653] (Input): User input information and operation behavior

[1654] (Output): Parsed user sentiment

[1655] Step 5:

[1656] Search for similar past cases from the database

[1657] The server searches a database for similar past cases based on the analyzed keywords and queries related cases.

[1658] (Input): Parsed keyword

[1659] (Output): Past similar cases

[1660] Step 6:

[1661] Calculating the probability of failure causes using statistical data

[1662] Based on the search results, the server uses statistical data to calculate the probability of the cause of the failure. For example, it estimates that the smart lock's battery is most likely dead.

[1663] (Input): Past similar cases

[1664] (Output): Probability of each failure cause

[1665] Step 7:

[1666] Estimation of the cause of failure

[1667] The server estimates the most likely cause of the failure based on the calculated occurrence probability.

[1668] (Input): Probability of each failure cause

[1669] (Output): Estimated cause of failure

[1670] Step 8:

[1671] Generating solutions using generative AI models

[1672] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure. For example, the server inputs a prompt statement such as "The smart lock is unresponsive. The situation is as follows: the battery is dead. Please suggest the optimal solution." into the generative AI model to obtain a solution.

[1673] (Input): Estimated cause of failure and prompt statement

[1674] (Output): The optimal solution generated

[1675] Step 9:

[1676] Provide the user with a solution

[1677] The server optimizes the generated solution based on the user's emotions and sends it to the user's device. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[1678] (Input): Generated solutions and analyzed user sentiment

[1679] (Output): Optimized solution

[1680] Step 10:

[1681] Receiving feedback and recording it in a database

[1682] The user tries the proposed solutions and inputs the results into the terminal and sends them to the server. The server receives this feedback and records it in a database. The analyzed emotional data is also recorded at the same time.

[1683] (Input): User feedback

[1684] (Output): Feedback and emotion data recorded in a database

[1685] The above processing steps allow the user to quickly and appropriately resolve the problem, and also realize a response that takes into consideration the user's feelings.

[1686] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1687] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1688] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1689] [Fourth embodiment]

[1690] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1691] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1692] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1693] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1694] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1695] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1696] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1697] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1698] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1699] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1700] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1701] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1702] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1703] The present invention is a system that allows a user to input product defect information, identifies the cause of the failure based on that information, and proposes an optimal solution. Specific embodiments of this system will be described below.

[1704] Overall system configuration

[1705] The system mainly consists of a user's terminal, a server, and a database. The user uses the terminal to input information about the problem, which is then sent to the server. The server processes the information, compares it with the database, deduces the cause of the problem, and generates a solution.

[1706] User Roles

[1707] The user mainly performs the following operations:

[1708] 1. Entering defect information: The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form.

[1709] 2. Confirmation of the solution: The user confirms the solution sent from the server on the device.

[1710] 3. Implementing the solution and providing feedback: The user carries out the repair or part replacement based on the proposed solution, and then enters the results into the terminal again and sends them to the server.

[1711] Server Roles

[1712] The server is the central part of the system and performs the following tasks:

[1713] 1. Data reception and analysis: The server receives the defect information sent from the user's device and analyzes it using a natural language processing engine. Important keywords and patterns are extracted to identify the problem.

[1714] 2. Database collation: Based on the analysis results, the results are collated with a database of past failures, similar cases are searched for, and statistical data is generated.

[1715] 3. Estimation of the cause of failure: Based on the statistical data obtained from the search results, the probability of occurrence of the cause of failure is calculated and the cause with the highest probability is estimated.

[1716] 4. Solution Generation: Based on the estimated cause of the failure, generative AI is used to generate the optimal solution, as well as a list of required replacement parts.

[1717] 5. Send Proposal: Send the generated solution and parts list to the user's device.

[1718] 6. Receiving and recording feedback: Receive feedback from users and record it in a database. This data will be used to identify the cause of future failures.

[1719] The role of the database

[1720] The database stores the following information:

[1721] 1. Past failure data: Records of past failures and their responses.

[1722] 2. Statistical data: Basic data for calculating the probability of occurrence of failure causes.

[1723] 3. Solutions and parts list: Information on the best solutions and required parts for each estimated cause of failure.

[1724] 4. Feedback information: Results after repair and feedback information from users.

[1725] Specific examples

[1726] For example, if your air conditioner is not cooling properly:

[1727] 1. User operation: The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[1728] 2. Server analysis: The server extracts the keywords "not cooling," "1 hour," and "air conditioning not working."

[1729] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[1730] 4. Application of statistical data: Based on statistical data obtained from past cases, the probability of refrigerant shortage, compressor failure, and filter clogging occurring is calculated.

[1731] 5. Estimation of the cause of failure: It is estimated that the most likely cause is a lack of refrigerant.

[1732] 6. Solution generation: The AI ​​generates a solution, adding refrigerant and cleaning the filter, and presents it to the user.

[1733] 7. User execution and feedback: The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[1734] 8. Server Recording: The server records this feedback in a database for future analysis.

[1735] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a failure, even without specialized knowledge.

[1736] The processing flow will be explained below.

[1737] Step 1:

[1738] The user accesses the device's input form and enters product defect information, including the product name, model number, detailed problem description, the circumstances under which the problem occurred, and any error messages.

[1739] Step 2:

[1740] When the user has finished entering the defect information in the input form, the terminal sends the information to the server, where it is converted into an appropriate data format before being sent.

[1741] Step 3:

[1742] The server receives the defect information sent from the device and starts a natural language processing (NLP) engine to analyze the received data.

[1743] Step 4:

[1744] The server uses a natural language processing engine to analyze the malfunction information and extract important keywords and phrases, such as "not cooling," "after one hour," and "air conditioning not working."

[1745] Step 5:

[1746] The server then searches the database based on the analyzed keywords and phrases to find similar past cases. For example, it extracts cases that include past reports of the product not cooling down.

[1747] Step 6:

[1748] The server uses statistical data based on the data obtained from the search results to calculate the probability of occurrence of each cause of failure, such as refrigerant shortage, compressor failure, and filter clogging.

[1749] Step 7:

[1750] The server estimates the most likely cause of the failure based on the calculated probability of occurrence. For example, it estimates that a refrigerant shortage is the most likely cause.

[1751] Step 8:

[1752] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, such as adding refrigerant or cleaning the filter.

[1753] Step 9:

[1754] The server compiles the generated solution and a list of required replacement parts and sends it to the user's device, where the user can view it.

[1755] Step 10:

[1756] The user then performs repairs or part replacements based on the proposed solutions, for example adding refrigerant and cleaning the filters.

[1757] Step 11:

[1758] The user inputs the results of the repairs they performed into the terminal and sends them to the server, for example, sending feedback such as "After adding the refrigerant, the air conditioner worked normally."

[1759] Step 12:

[1760] The server receives feedback from users, analyzes the data, and records it in a database. This information is used to identify the cause of future failures and improve solutions.

[1761] Example 1

[1762] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1763] Conventional systems for identifying the cause of a failure and presenting solutions have had difficulty accurately analyzing failure information from users and quickly and effectively presenting appropriate solutions. Furthermore, because user feedback could not be reflected in the next failure analysis, the learning and evolution of the entire system was delayed. This resulted in delays in troubleshooting product failures and reduced user convenience.

[1764] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1765] In this invention, the server includes means for receiving failure information, means for analyzing the information using natural language processing, means for searching for similar cases in a database, means for estimating the cause of the failure using statistical data, means for generating an optimal solution using a generative model, means for presenting the solution to the user, and means for receiving feedback and recording the feedback in the database. This makes it possible to quickly and accurately identify the cause of the failure based on detailed failure information from the user and provide an optimal solution. Furthermore, by reflecting user feedback in the system, learning and evolution of the entire system are promoted.

[1766] "Defect information" is information in which the user specifically and in detail describes abnormalities or problems with the product.

[1767] "Natural language processing" is a technology that uses computers to analyze natural language and understand its meaning and structure.

[1768] A "database" is a collection of data that systematically stores past failure cases and related data and manages them in a searchable format.

[1769] "Similar cases" are records of similar malfunctions or problems that have occurred in the past.

[1770] "Statistical data" refers to data for calculating the probability of occurrence of a failure cause based on numerical data obtained from a database.

[1771] The "cause of failure" is the fundamental cause of a product defect or abnormality.

[1772] A "generative model" is an algorithm that uses AI technology to generate an appropriate solution from given data.

[1773] A "solution" is a specific procedure or action proposed to solve a problem based on the estimated cause of the failure.

[1774] "Replacement Part" means a part or component required to resolve a defect.

[1775] "Feedback" is the evaluation or information provided by a user based on the results or experience of the solution they have implemented.

[1776] A "server" is a computer system that receives, analyzes, and stores data via a network, and provides the processed results to a client terminal.

[1777] A "user terminal" is a device such as a computer or smartphone that a user uses to input information or receive data from a server.

[1778] Overall system configuration

[1779] The system of the present invention mainly consists of a user terminal, a server, and a database. The user inputs fault information using the terminal, and the information is sent to the server. The server processes the information, compares it with the database, deduces the cause of the fault, and generates a solution.

[1780] Role of user terminal

[1781] The user terminal is the device that inputs information about the problem and receives and checks solutions from the server. Specifically, the user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the terminal. This information is then sent to the server. User terminals can be PCs or smartphones.

[1782] Server Roles

[1783] The server is the central part of the system and performs a variety of processes. First, it receives the failure information sent from the user's device. Next, it analyzes the information using a natural language processing engine (such as SpaCy or BERT) and extracts important keywords, which allows it to identify the problem. The server then searches a database based on the extracted keywords and refers to similar cases. The server estimates the cause of the failure based on statistical data obtained from past failure data. It then uses a generative AI model (such as a GPT model) to generate an optimal solution. The server also generates a list of required replacement parts and sends this to the user's device. Finally, it receives feedback from the user and records it in a database.

[1784] The role of the database

[1785] The database stores past failure cases and their corresponding records, statistical data, solutions, replacement parts lists, feedback information, etc. This provides the server with the basic data to quickly identify the cause of the failure and generate an appropriate solution.

[1786] Specific examples

[1787] For example, the following shows the flow when the user inputs "The air conditioner is not cooling. After about an hour of use, the cooling stops working."

[1788] 1. User operation: The user enters the defect information into the input form on the terminal.

[1789] 2. Server analysis: The server extracts the keywords "not cooling," "one hour later," and "air conditioning not working."

[1790] 3. Database search: The server searches the database based on these keywords and refers to past similar cases.

[1791] 4. Estimation of failure causes: Calculate the probability of occurrence of causes such as refrigerant shortage, compressor failure, and filter blockage from statistical data. Estimate the cause with the highest probability (e.g., refrigerant shortage).

[1792] 5. Solution generation: The AI ​​generates solutions, "add refrigerant" and "clean the filter," and presents them to the user.

[1793] 6. User execution and feedback: The user executes the proposed solution and enters the results into the terminal.

[1794] 7. Server Recording: The server receives the feedback and records it in a database.

[1795] Prompt Sentence Examples

[1796] "My air conditioner isn't cooling. If it stops working after about an hour of use, what could be the cause of the problem? Please tell me what to do."

[1797] This system allows users to quickly and accurately identify the cause of a failure and find the optimal solution simply by entering detailed information about the product's malfunction.In addition, accumulating feedback from users will improve the analytical capabilities and scope of application of the entire system.

[1798] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1799] Step 1:

[1800] The user inputs the defect information.

[1801] Input: Product name, model number, problem description, circumstances under which it occurred, error messages, etc.

[1802] Specific operation: The user enters information such as "The air conditioner is not cooling. After about an hour of use, the cooling stops working" into the input form on the device.

[1803] Output: The entered defect information is saved on the terminal and later sent to the server.

[1804] Step 2:

[1805] The device sends the defect information to the server.

[1806] Input: The defect information entered by the user.

[1807] Specific operation: The device automatically sends the saved defect information to the server's API.

[1808] Output: The failure information is passed to the server.

[1809] Step 3:

[1810] The server receives and analyzes the defect information.

[1811] Input: The error information sent from the device.

[1812] Specific operation: The server uses a natural language processing engine (e.g., SpaCy, BERT) to analyze the malfunction information and extract important keywords (e.g., "not cooling," "after one hour," "air conditioning not working").

[1813] Output: Extracted keyword list.

[1814] Step 4:

[1815] The server searches the database based on the keyword list.

[1816] Input: Extracted keyword list.

[1817] Specific operation: The server searches the database and retrieves past similar cases.

[1818] Output: A list of similar cases.

[1819] Step 5:

[1820] The server uses statistical data from similar cases to estimate the cause of the failure.

[1821] Input: A list of similar cases.

[1822] Specific operation: Based on the statistical data, the server calculates the occurrence probability of each cause of failure (e.g., refrigerant shortage, compressor failure, clogged filter).

[1823] Output: Probability of occurrence of each failure cause.

[1824] Step 6:

[1825] The server uses a generative model based on the most probable causes of failure to generate an optimal solution.

[1826] Input: Probability of occurrence of failure cause.

[1827] Specific operation: The server uses a generative AI model (e.g., GPT model) to generate the optimal solution for the most probable cause of the failure, for example, "add refrigerant" and "clean the filter."

[1828] Output: A list of solutions.

[1829] Step 7:

[1830] The server transmits the generated solution and a list of required replacement parts to the user terminal.

[1831] Input: List of solutions, list of replacement parts needed.

[1832] Specific operation: The server compiles the solution and replacement parts list and sends them to the user terminal.

[1833] Output: Solution and parts list displayed on the user's terminal.

[1834] Step 8:

[1835] The user executes the solution and enters the results into the terminal.

[1836] Input: Solution and replacement parts list sent from the server.

[1837] Specific actions: The user takes problem-solving actions according to the presented steps and inputs the results (e.g., success or failure of the solution) into the terminal.

[1838] Output: The feedback information is stored on the device and later sent to the server.

[1839] Step 9:

[1840] The server receives the feedback from the user and records it in a database.

[1841] Input: Feedback information entered by the user into the device.

[1842] Specific operation: The server receives the feedback sent by the device and stores it in a database.

[1843] Output: Feedback information is recorded in a database.

[1844] Through these steps, users can input product defect information, the server performs analysis and estimation, and provides the user with the optimal solution. By incorporating feedback, the accuracy and efficiency of the entire system are improved.

[1845] (Application example 1)

[1846] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1847] Conventional fault diagnosis systems estimate the cause of a fault based on fault information entered by the user and provide solutions. However, as factories become more automated, a faster and more advanced response is required. However, these systems rely on manual user operation, making it difficult to respond in real time or collect feedback. In particular, when a factory robot breaks down, the manual input of fault information is cumbersome, making it difficult to respond quickly. Therefore, a new system that can diagnose faults and provide a rapid response for factory robots is needed.

[1848] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1849] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the received defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for calculating the probability of occurrence of a failure cause using statistical data from the search results, means for estimating the failure cause based on the calculated probability, means for generating an optimal solution based on the estimated failure cause, means for presenting the generated solution to the user, means for coordinating with a factory robot to execute the presented solution, and means for receiving feedback from the user via the factory robot and recording it in the database, thereby enabling factory robot failure diagnosis and rapid response.

[1850] "User" refers to the factory technician or operator who inputs the defect information and implements the proposed solution.

[1851] "Fault information" is information that provides detailed explanations of failures and abnormalities that occur in factory robots and other equipment.

[1852] "Means for receiving" refers to a device or program that has the function of sending and receiving defect information entered by a user to a server.

[1853] "Natural language processing" refers to the technology of analyzing human language and understanding its meaning.

[1854] "Means for analyzing" refers to a program or device that has the function of analyzing received defect information using natural language processing technology.

[1855] "Database" refers to a computer system that stores and manages past failure cases and solutions.

[1856] "Searching means" refers to a program that has the function of searching for similar past cases from a database based on the analysis results.

[1857] "Statistical data" refers to data for calculating the probability of occurrence of a specific cause of failure based on past cases.

[1858] "Means for calculating" refers to a program that has the function of calculating the probability of occurrence of a failure cause using statistical data.

[1859] "Cause of failure" refers to the factors that cause a specific malfunction or abnormality to occur.

[1860] "Means for estimating" refers to a program that has the function of estimating the most likely cause of a failure based on the calculated probability of occurrence.

[1861] "Solution method" refers to the procedure for instructing the optimal repair or countermeasure for the estimated cause of the failure.

[1862] "Means for generating" refers to a program that has the function of automatically generating a solution based on the estimated cause of the failure.

[1863] "Presentation means" refers to a display device or program for showing the generated solution to the user.

[1864] "Means for collaboration" refers to devices or programs that have the function of communicating with factory robots and instructing their operations in order to implement the proposed solution.

[1865] "Feedback" refers to the results and opinions of users who implement the solutions provided.

[1866] A "factory robot" refers to a mechanical device that performs work automatically within a factory.

[1867] "Means for recording" refers to a program that has the function of storing feedback information in a database.

[1868] MODE FOR CARRYING OUT THE INVENTION

[1869] This invention relates to a fault diagnosis system for factory robots that estimates the cause of a fault based on malfunction information provided by the user and proposes the optimal solution. This system is composed of a user terminal, a server, and a database, and realizes factory automation and efficient fault response.

[1870] Hardware and software used

[1871] An embodiment of the present invention uses the following hardware and software:

[1872] Hardware: smartphones, tablets, factory robots, servers

[1873] Software: Programming languages ​​(Python, JavaScript), databases (MySQL, PostgreSQL), AI models (GPT-3, BERT), NLP engines (spaCy, NLTK), cloud services (AWS, Google Cloud)

[1874] Data processing and calculation flow

[1875] 1. User operations

[1876] Users use smartphones or tablets to input malfunction information for factory robots. The malfunction information includes the product name, model number, details of the problem, the circumstances under which the problem occurred, and error messages. This information is then sent from the user's device to the server.

[1877] 2. Data Reception and Analysis

[1878] The server receives the defect information sent from the user terminal. The received defect information is analyzed using a natural language processing engine (e.g., spaCy) to extract important keywords and patterns.

[1879] 3. Database Verification

[1880] Based on the analyzed data, the server searches the database for similar past cases, referencing past failure cases and response records, and generates statistical data based on the search results.

[1881] 4. Estimation of the cause of the failure

[1882] The server uses the generated statistical data to calculate the probability of occurrence of each cause of failure, and estimates the cause with the highest probability.

[1883] 5. Solution Generation

[1884] Based on the estimated cause of the failure, a generative AI model (e.g., GPT-3) is used to generate an optimal solution, along with a list of required replacement parts.

[1885] 6. Presentation to the User

[1886] The server sends the generated solution and replacement part list to the user's terminal and presents them to the user, who then executes the proposed solution.

[1887] 7. Collecting and recording feedback

[1888] The results of the user's execution are sent back to the server from the terminal, and the server receives and records this feedback information. This data is stored in a database and will be used for future failure analysis.

[1889] Specific examples

[1890] For example, if a robot arm in a factory stops working, the user inputs the following information about the problem into a tablet: "The robot arm won't move. There's an abnormal noise. Error code 5678 is displayed." Based on this information, the server analyzes it using an NLP engine and performs a database search. Statistical data is generated from the search results, and after calculating the probability of occurrence, it is estimated that the most likely cause is "motor failure." A generative AI model (e.g., GPT-3) is used to suggest "replacing the motor" as a solution. The user replaces the motor and re-enters the results into the system. This series of steps enables factory robot failures to be handled quickly and accurately.

[1891] Prompt Sentence Examples

[1892] "We are experiencing the following problems with our factory robot: The robot arm does not move, there is an abnormal noise, and error code 5678 is displayed." Please suggest the best solution based on this situation."

[1893] As a result, the present invention realizes rapid and accurate failure analysis and presentation of solutions based on the defect information input by the user.

[1894] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1895] Program processing steps and specific operations

[1896] Step 1:

[1897] The terminal receives the defect information input by the user. The user inputs the product name, model number, details of the defect, the circumstances when the defect occurred, and an error message into the input form on the terminal. This input data is sent from the terminal to the server.

[1898] Input: Product name, model number, problem details, circumstances under which it occurred, error message

[1899] Output: Defect information data sent to the server

[1900] Step 2:

[1901] The server receives the defect information data sent from the device. The received data is analyzed using a natural language processing engine (e.g., spaCy). During the analysis, important keywords and patterns are extracted and used in the next step, a database search.

[1902] Input: Received defect information data

[1903] Output: Extracted keywords and patterns

[1904] Step 3:

[1905] The server then searches the database based on the analysis results. The search locates similar cases from past failure cases and response records. In this process, a list of related cases and statistical data is generated.

[1906] Input: Extracted keywords or patterns

[1907] Output: List of similar cases and statistical data

[1908] Step 4:

[1909] The server uses the generated statistical data to calculate the probability of occurrence of the cause of the failure, and estimates the most likely cause of the failure based on the calculated probability of occurrence.

[1910] Input: List of similar cases and statistical data

[1911] Output: Estimated cause of failure and its probability of occurrence

[1912] Step 5:

[1913] The server uses a generative AI model (e.g., GPT-3) to generate an optimal solution based on the estimated cause of the failure, while simultaneously generating a list of required replacement parts.

[1914] Input: Estimated cause of failure and its probability of occurrence

[1915] Output: Solution and replacement part list

[1916] Step 6:

[1917] The server sends the generated solutions and a list of replacement parts to the user's terminal and presents them to the user, who then checks and implements the solutions.

[1918] Input: Solution and replacement part list

[1919] Output: A list of solutions and replacement parts sent to the user's device

[1920] Step 7:

[1921] The feedback obtained based on the solution method the user performs is sent from the terminal to the server. The server receives and records this feedback and stores it in a database. This feedback data is used for future fault analysis and resolution.

[1922] Input: Feedback of the results of the user's actions

[1923] Output: Feedback information recorded in a database

[1924] This allows for rapid and accurate diagnosis and resolution of factory robot failures through a series of steps.

[1925] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1926] The present invention is a system that combines a system in which a user inputs product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution, with an emotion engine that recognizes the user's emotions. Specific embodiments of this system are described below.

[1927] Overall system configuration

[1928] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[1929] User Roles

[1930] The user mainly performs the following operations:

[1931] 1. Enter the defect information:

[1932] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the input form on the device.

[1933] The emotion engine also simultaneously monitors the user's input information and behavior and analyzes emotions.

[1934] 2. Check the solution:

[1935] The user checks the solution sent from the server on the terminal.

[1936] Content optimized by the emotion engine is presented.

[1937] 3. Response implementation and feedback:

[1938] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[1939] Server Roles

[1940] The server is the center of the system and performs the following tasks:

[1941] 1. Data reception and analysis:

[1942] The server receives the defect information sent from the user terminal and launches a natural language processing (NLP) engine to analyze the received data.

[1943] Extract important keywords and phrases.

[1944] 2. Database Matching:

[1945] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[1946] 3. Applying statistical data and estimating the cause of failure:

[1947] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[1948] 4. Generate a solution:

[1949] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution.

[1950] It also generates a list of required replacement parts.

[1951] 5. Solution suggestions:

[1952] The generated solution and the required parts list are sent to the user's terminal.

[1953] The content presented is optimized based on the analysis results of the emotion engine.

[1954] 6. Receiving and recording feedback:

[1955] Receive feedback from users, analyze the data and record it in a database.

[1956] The role of the emotional engine

[1957] The Emotion Engine is responsible for:

[1958] 1. User Emotion Recognition:

[1959] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[1960] 2. Optimizing the solution:

[1961] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[1962] 3. Recording emotional data:

[1963] The recognized user emotion data is recorded in a database and used for future analysis.

[1964] The role of the database

[1965] The database stores the following information:

[1966] 1. Past failure data:

[1967] Past failure cases and their corresponding records.

[1968] 2. Statistics:

[1969] Basic data for calculating the probability of occurrence of a failure cause.

[1970] 3. Solution and parts list:

[1971] Information on the optimal solution and required parts for each estimated cause of failure.

[1972] 4. Feedback Information:

[1973] Repair results and user feedback information.

[1974] 5. Emotional Data:

[1975] Record the results of user sentiment analysis and use the data for future reference.

[1976] Specific examples

[1977] For example, if your air conditioner is not cooling properly:

[1978] 1. User Action:

[1979] The user types into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes that the user is frustrated based on the input data and typing speed.

[1980] 2. Server Analysis:

[1981] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working." The emotion engine analyzes the speed of input and the parts that are emphasized.

[1982] 3. Database Search:

[1983] The server searches the database based on these keywords and refers to similar past cases.

[1984] 4. Statistical Data Application:

[1985] Based on the statistical data obtained from the search, the probability of refrigerant shortage, compressor failure, and filter blockage occurring is calculated.

[1986] 5. Estimation of the cause of the failure:

[1987] The most likely cause is a lack of refrigerant.

[1988] 6. Generate a solution:

[1989] The solution is to add refrigerant and clean the filter. The emotion engine reflects the user's frustration and presents the solution in a format that is as simple and quick to respond as possible.

[1990] 7. User execution and feedback:

[1991] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[1992] 8. Server Records:

[1993] The server records this feedback in a database, along with emotional data, for future analysis.

[1994] In this way, this system makes it possible to quickly and appropriately identify and resolve the cause of a malfunction without requiring specialized knowledge, and also enables responses that take into consideration the user's feelings.

[1995] The processing flow will be explained below.

[1996] Step 1:

[1997] The user accesses the input form on the device and enters product defect information, including the product name, model number, detailed problem description, circumstances at the time of the occurrence, and error message. As the user types, the emotion engine analyzes the user's typing speed and writing style to evaluate the user's emotions.

[1998] Step 2:

[1999] When the user completes the input and presses the send button, the device sends the defect information and emotion data to the server, including metadata reflecting the user's emotion.

[2000] Step 3:

[2001] The server launches a natural language processing engine to analyze the malfunction information received from the device. The server extracts important keywords and phrases to identify the problem. For example, it extracts keywords such as "not cooling," "one hour later," and "air conditioning not working."

[2002] Step 4:

[2003] The server searches a database for similar cases based on the analyzed keywords and phrases. The server checks whether a similar problem has already been reported.

[2004] Step 5:

[2005] Based on the search results, the server uses statistical data to calculate the probability of each cause of failure, such as a refrigerant shortage, compressor failure, or clogged filter.

[2006] Step 6:

[2007] The server evaluates the probability of occurrence of each cause and estimates the most likely cause of the failure, for example, a refrigerant shortage.

[2008] Step 7:

[2009] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure, generating specific steps to add refrigerant and clean the filter.

[2010] Step 8:

[2011] The server compiles the generated list of solutions and replacement parts and optimizes it based on the user's emotional data, for example prioritizing simple and fast solutions if the user is frustrated.

[2012] Step 9:

[2013] The server sends the optimized solution and parts list to the user's device, where the user can review and understand the specific steps to take.

[2014] Step 10:

[2015] The user performs repairs or part replacements based on the proposed solutions, for example, the user adds refrigerant and cleans the filter.

[2016] Step 11:

[2017] The user inputs the results of the repair into the terminal and sends them to the server. For example, the user sends feedback such as, "After adding the refrigerant, the air conditioner worked normally."

[2018] Step 12:

[2019] The server receives feedback from users, analyzes the data, and records it in a database. At the same time, it also records the user's emotional data for future analysis.

[2020] In this way, this system is not only able to quickly and appropriately identify the cause of a malfunction and propose a solution, but also to respond in a way that takes the user's feelings into consideration.

[2021] Example 2

[2022] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2023] Conventional fault diagnosis systems have difficulty efficiently analyzing malfunction information from users and identifying the cause of the malfunction. They also lack the ability to recognize users' emotions and stress levels and provide optimal solutions, potentially resulting in a poor user experience. Furthermore, the solutions they generate often do not provide a list of replacement parts needed, making it difficult for users to prepare for repairs. These issues make it difficult to provide prompt and appropriate responses.

[2024] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes: means for receiving malfunction information input by a user; means for analyzing the received malfunction information using natural language processing; means for searching a database for similar past cases based on the analysis results; means for calculating the occurrence probability of a malfunction cause using statistical data from the search results; means for estimating the malfunction cause based on the calculated probability; means for generating an optimal solution based on the estimated malfunction cause using a generative AI model; means for presenting the generated solution to the user; means for analyzing the user's input information and behavior and recognizing emotions; and means for optimizing the presented solution based on the recognized emotions. This makes it possible to quickly and appropriately identify the malfunction cause and provide an efficient solution. Furthermore, it is possible to respond in consideration of the user's emotions, thereby improving the user experience.

[2025] A "user" is an entity that uses the system to input product defect information and implement suggested solutions.

[2026] A "terminal" is an electronic device that a user uses to input defect information, check solutions, and provide feedback on the results of the solutions.

[2027] The "server" is a central processing unit that receives defect information from users, analyzes it, and provides the information.

[2028] "Problem information" is information that indicates details of a problem that has occurred in a product, and is entered by the user.

[2029] "Natural language processing" is a technology that allows computers to understand human language.

[2030] "Analysis" is the process of interpreting defect information and extracting important keywords and phrases.

[2031] A "database" is an information system that stores past failure cases, statistical data, solutions, replacement part information, and so on.

[2032] "Statistical data" is numerical information such as the probability of an event occurring, calculated based on past data.

[2033] "Cause of failure" refers to the factor that causes the product to not function properly.

[2034] "Occurrence probability" is a numerical value that indicates the degree of possibility that a particular cause of failure will occur.

[2035] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate optimal solutions from specific inputs.

[2036] A "solution" is a specific procedure or measure to resolve a specific cause of a failure.

[2037] An "emotion engine" is a system that analyzes the user's input information and operational behavior and recognizes the user's emotions.

[2038] "Emotions" refer to the psychological state that a user feels while using a system, and include feelings of frustration and satisfaction.

[2039] "Recognition" is the process by which the system identifies the user's emotions.

[2040] "Optimization" means adjusting the solution presentation based on the user's feelings to make it more acceptable.

[2041] The "list of replacement parts" is a list of parts required to eliminate the cause of the failure.

[2042] "Feedback" is information that a user provides to the system as a result of implementing a solution.

[2043] This invention combines a system that allows users to input product defect information, identifies the cause of the failure based on that information, and proposes the optimal solution with an emotion engine that recognizes the user's emotions. This system is implemented using the following hardware and software.

[2044] Overall system configuration

[2045] The system mainly consists of a user's device, a server, a database, and an emotion engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion engine then analyzes the user's input information and behavior, recognizes emotions, and optimizes the solution suggestions it presents.

[2046] User Roles

[2047] The user uses the terminal to:

[2048] 1. Enter the defect information:

[2049] The user enters the product name, model number, details of the problem, the circumstances under which it occurred, error messages, etc. into the device's input form. The emotion engine simultaneously monitors the user's input speed, choices, etc., and analyzes their emotions.

[2050] 2. Check the solution:

[2051] The user checks the solution sent from the server on their device, and the solution optimized by the emotion engine is presented.

[2052] 3. Response implementation and feedback:

[2053] The user carries out repairs or part replacements based on the presented solutions, and inputs the results of the repairs into the terminal and transmits them to the server.

[2054] Server Roles

[2055] The server is the center of the system and performs the following tasks:

[2056] 1. Data reception and analysis:

[2057] The server receives the defect information sent from the user's device and analyzes it using a natural language processing (NLP) engine to extract important keywords and phrases.

[2058] 2. Database Matching:

[2059] Based on the analysis results, the system compares them with a database of past failures to search for similar cases.

[2060] 3. Applying statistical data and estimating the cause of failure:

[2061] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each failure cause, and the failure cause with the highest probability is estimated.

[2062] 4. Generate a solution:

[2063] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution, along with a list of required replacement parts.

[2064] 5. Solution suggestions:

[2065] The generated solution and required parts list are sent to the user's device, and the content presented is optimized based on the analysis results of the emotion engine.

[2066] 6. Receiving and recording feedback:

[2067] Receive feedback from users, analyze the data and record it in a database.

[2068] The role of the emotional engine

[2069] The Emotion Engine is responsible for:

[2070] 1. User Emotion Recognition:

[2071] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[2072] 2. Optimizing the solution:

[2073] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[2074] 3. Recording emotional data:

[2075] The recognized user emotion data is recorded in a database and used for future analysis.

[2076] The role of the database

[2077] The database stores the following information:

[2078] 1. Past failure data:

[2079] Store records of past failure cases and their responses.

[2080] 2. Statistics:

[2081] Contains basic data for calculating the probability of occurrence of a failure cause.

[2082] 3. Solution and parts list:

[2083] Save information on optimal solutions and required parts.

[2084] 4. Feedback Information:

[2085] Stores the results of repairs and feedback information from users.

[2086] 5. Emotional Data:

[2087] Records the results of user sentiment analysis and includes data for future reference.

[2088] Specific examples

[2089] For example, if your air conditioner is not cooling properly:

[2090] 1. User Action:

[2091] The user inputs into the terminal, "The air conditioner is not cooling. After about an hour of use, the cooling stops working." The emotion engine recognizes the user's frustration.

[2092] 2. Server Analysis:

[2093] The server extracts the keywords "not cold," "one hour later," and "air conditioning not working."

[2094] 3. Database Search:

[2095] The server searches the database based on these keywords and refers to similar past cases.

[2096] 4. Statistical Data Application:

[2097] Based on statistical data, the probability of refrigerant shortage, compressor failure, and filter clogging is calculated.

[2098] 5. Estimation of the cause of the failure:

[2099] The most likely cause is a lack of refrigerant.

[2100] 6. Generate a solution:

[2101] A solution is generated suggesting adding refrigerant and cleaning the filter. The emotion engine reflects the user's frustration and presents the solution in a simple, immediate format.

[2102] 7. User execution and feedback:

[2103] The user adds refrigerant and cleans the filter, enters the results into the terminal, and sends them to the server.

[2104] 8. Server Records:

[2105] The server records the feedback information and emotion data in a database.

[2106] Prompt Sentence Examples

[2107] "We have received a report that the air conditioner is not cooling. It has been observed that the cooling stops working after an hour of use and the user seems to be frustrated. Please suggest the cause of the problem and how to fix it."

[2108] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2109] Step 1: Enter your defect information

[2110] The user enters information about the problem, such as the product name, model number, details of the problem, the circumstances under which it occurred, and any error messages, into the device's input form. Once the input is complete, the user presses the send button. The emotion engine simultaneously monitors the user's input speed and operating behavior to recognize emotions. For example, if the user enters "The air conditioner isn't cooling. After about an hour of use, the cooling stopped working," the emotion of irritation will be recognized.

[2111] Input: Product name, model number, problem details, occurrence status, error message

[2112] Output: Input defect information, user emotion data

[2113] Step 2: Submit your defect report

[2114] The device packages the entered defect information in JSON format and sends it to the server. The data is sent to the endpoint as a POST request.

[2115] Input: Entered defect information, user emotion data

[2116] Output: Defect information sent to the server, user emotion data

[2117] Step 3: Data reception and analysis

[2118] The server receives the malfunction information sent from the device. The received data is passed to a natural language processing (NLP) engine, which extracts important keywords and phrases. For example, keywords such as "not cooling," "after one hour," and "air conditioning not working" are extracted.

[2119] Input: Error information received from the device

[2120] Output: Extracted keywords and phrases

[2121] Step 4: Search the database

[2122] The server searches the database based on the extracted keywords and phrases. It compares them with past malfunction cases and extracts similar cases. For example, it searches for past malfunction cases related to "not cooling" and retrieves cases such as refrigerant shortages and clogged filters.

[2123] Input: Extracted keywords and phrases

[2124] Output: Similar cases

[2125] Step 5: Identify the cause of the failure

[2126] The server applies statistical data based on similar cases retrieved from the database to calculate the probability of occurrence of each failure cause. Based on the calculated probabilities, it estimates the most likely cause of the failure. For example, it estimates that a refrigerant shortage is the most likely cause.

[2127] Input: Similar cases

[2128] Output: Estimated cause of failure, occurrence probability

[2129] Step 6: Generate a solution

[2130] The server runs a generative AI model to generate solutions based on the estimated cause of the failure, such as adding refrigerant and cleaning the filter, and also generates a list of required replacement parts.

[2131] Input: Estimated cause of failure, occurrence probability

[2132] Output: Generated solution, replacement parts list

[2133] Step 7: Optimize and present the solution

[2134] The server optimizes the solution based on the user's emotions, recognized using an emotion engine. For example, if the user is frustrated, the server presents a solution as a simple and quick procedure. The server then sends the optimized solution and parts list to the user's device.

[2135] Input: Generated solutions, replacement parts list, user sentiment data

[2136] Output: Optimized solution, replacement parts list

[2137] Step 8: Receive and record feedback

[2138] The user repairs or replaces parts based on the proposed solutions. The results are entered into the terminal and sent as feedback to the server. The server receives this feedback information, analyzes it, and records it in a database. Emotional data is also recorded.

[2139] Input: Repair execution results, feedback information

[2140] Output: Feedback information and emotion data recorded in the database

[2141] (Application example 2)

[2142] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2143] Conventional product defect analysis systems do not take into account the user's emotions, and therefore are unable to reduce the stress and frustration experienced when users input defect information or receive solutions. In addition, because generative AI models are not adequately used to propose optimal solutions, the solutions presented are not always effective, making it difficult to quickly resolve problems.

[2144] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2145] In this invention, the server includes means for receiving defect information input by a user, means for analyzing the defect information using natural language processing, means for searching a database for similar past cases based on the analysis results, means for recognizing emotions, means for generating an optimal solution, and means for presenting the generated solution to the user. This reduces stress by taking the user's emotions into consideration, and enables effective and rapid problem solving using a generative AI model.

[2146] The "means for receiving defect information input by the user" is a function for transmitting information about product or system defects input by the user through a terminal to the server, and for the server to receive this information.

[2147] "Means for analyzing using natural language processing" refers to a technology that receives defect information entered by a user as text data, analyzes the text data, and extracts important keywords and phrases.

[2148] The "means for searching for similar past cases from a database" is a function for searching for similar past recorded cases of defects in a database based on analyzed defect information.

[2149] "Means for recognizing emotions" refers to technology that analyzes the user's input information and operation behavior, and recognizes the user's current emotions (for example, irritation, confusion, tension, etc.).

[2150] The "means for generating an optimal solution" is a function that uses a generative AI model to generate an optimal solution based on the analysis results and the estimated cause of the failure, and then proposes that solution.

[2151] The "means for presenting the generated solution to the user" is a function for displaying the generated solution to the user, and aims to present it in a format that is easy for the user to understand.

[2152] A "generative AI model" is an algorithm that uses artificial intelligence to generate new solutions or suggestions based on input information.

[2153] A "prompt" is a document containing instructions or questions that are input to a generative AI model, and serves as the basis for the AI ​​to propose the optimal solution.

[2154] The present invention is a system that allows a user to input malfunction information about security hardware (e.g., smart locks or surveillance cameras), identifies the cause of the malfunction based on that information, and proposes the optimal solution. It also combines a function that analyzes the user's emotions and provides optimal support based on those emotions. Specific embodiments of this system are described below.

[2155] Overall system configuration

[2156] The system mainly consists of a user's device, a server, a database, and an emotion recognition engine. The user uses the device to input malfunction information, which is then sent to the server. The server processes the information, compares it with the database, infers the cause of the malfunction, and generates a solution. The emotion recognition engine then analyzes the user's input information and behavior, recognizes their emotions, and optimizes the solution suggestions it presents.

[2157] User Roles

[2158] The user mainly performs the following operations:

[2159] 1. Enter the defect information:

[2160] The user enters details of the security hardware problem into the device's input form. For example, if the smart lock is unresponsive, the user can write, "The smart lock is unresponsive. Nothing happens when I press the button." The emotion recognition engine simultaneously monitors the user's input information and behavior and analyzes their emotions.

[2161] 2. Check the solution:

[2162] The user then checks the solution sent from the server on their device, and the content is optimized using an emotion recognition engine. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[2163] 3. Response implementation and feedback:

[2164] The user then takes action based on the proposed solution and inputs the results into the device, which is then sent to the server, which records this feedback in a database for future analysis.

[2165] Server Roles

[2166] The server is the center of the system and performs the following tasks:

[2167] 1. Data reception and analysis:

[2168] The server receives the defect information sent from the user device and starts a natural language processing engine (NLP engine) to analyze the received data. For example, it uses an NLPProcessor to extract important keywords and phrases.

[2169] 2. Database Matching:

[2170] Based on the analysis results, the database of past failures is checked to search for similar cases. This is done using the Database module.

[2171] 3. Applying statistical data and estimating the cause of failure:

[2172] Based on the data obtained from the search results, statistical data is used to calculate the probability of occurrence of each cause of failure, and the cause of failure with the highest probability is estimated.

[2173] 4. Generate a solution:

[2174] Based on the estimated cause of the failure, a generative AI model is used to generate the optimal solution. The optimal solution is generated using a prompt sentence as input. For example, the following prompt sentence is used:

[2175] - Example prompt: "My smart lock is unresponsive. The situation is as follows: dead battery. Please suggest the best solution."

[2176] If necessary, a list of parts required for the solution is automatically generated.

[2177] 5. Solution suggestions:

[2178] The generated solution and required parts list are sent to the user's device, and the presentation content is optimized based on the analysis results of the emotion recognition engine.

[2179] 6. Receiving and recording feedback:

[2180] It receives feedback from users, analyzes the data, and records it in a database. Emotional data is also recorded at the same time and used for future analysis.

[2181] The role of emotion recognition engines

[2182] The emotion recognition engine is responsible for:

[2183] 1. User Emotion Recognition:

[2184] It analyzes the user's input information, operation behavior, typing speed, etc. to recognize emotions.

[2185] 2. Optimizing the solution:

[2186] Based on the recognized user sentiment, the solution presentation is optimized and the information is presented in a format that is easy for the user to accept.

[2187] 3. Recording emotional data:

[2188] The recognized user emotion data is recorded in a database and used for future analysis.

[2189] In this way, the system allows users to quickly and appropriately resolve problems, and also provides a response that takes emotion into consideration.

[2190] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2191] Step 1:

[2192] The user enters the defect information

[2193] The user enters details of the security hardware problem (e.g., a malfunction of a smart lock) in text format into an input form on the device, and the entered information is sent to the server.

[2194] (Input): User's defect information

[2195] (Output): Raw data sent to the server

[2196] Step 2:

[2197] The server receives the failure information

[2198] The server receives the defect information sent from the user's device. Since the received data cannot be analyzed as is, it is analyzed using a natural language processing (NLP) engine.

[2199] (Input): Malfunction information sent from the device

[2200] (Output): Defect information to be passed to natural language processing

[2201] Step 3:

[2202] Analysis of defect information using natural language processing

[2203] The server's NLP engine analyzes the received malfunction information and extracts important keywords and phrases, such as "smart lock," "unresponsive," and "nothing happens when I press the button."

[2204] (Input): Error information sent to the server

[2205] (Output): Parsed keywords and phrases

[2206] Step 4:

[2207] Emotion recognition engine for analyzing user emotions

[2208] The emotion recognition engine analyzes the user's emotions based on their input information and operational behavior (e.g., typing speed). For example, it can detect that the user is irritated based on their typing speed and the text.

[2209] (Input): User input information and operation behavior

[2210] (Output): Parsed user sentiment

[2211] Step 5:

[2212] Search for similar past cases from the database

[2213] The server searches a database for similar past cases based on the analyzed keywords and queries related cases.

[2214] (Input): Parsed keyword

[2215] (Output): Past similar cases

[2216] Step 6:

[2217] Calculating the probability of failure causes using statistical data

[2218] Based on the search results, the server uses statistical data to calculate the probability of the cause of the failure. For example, it estimates that the smart lock's battery is most likely dead.

[2219] (Input): Past similar cases

[2220] (Output): Probability of each failure cause

[2221] Step 7:

[2222] Estimation of the cause of failure

[2223] The server estimates the most likely cause of the failure based on the calculated occurrence probability.

[2224] (Input): Probability of each failure cause

[2225] (Output): Estimated cause of failure

[2226] Step 8:

[2227] Generating solutions using generative AI models

[2228] The server uses a generative AI model to generate an optimal solution based on the estimated cause of the failure. For example, the server inputs a prompt statement such as "The smart lock is unresponsive. The situation is as follows: the battery is dead. Please suggest the optimal solution." into the generative AI model to obtain a solution.

[2229] (Input): Estimated cause of failure and prompt statement

[2230] (Output): The optimal solution generated

[2231] Step 9:

[2232] Provide the user with a solution

[2233] The server optimizes the generated solution based on the user's emotions and sends it to the user's device. For example, if the user is frustrated, the solution explanation will be presented in a format that is as simple and quick as possible.

[2234] (Input): Generated solutions and analyzed user sentiment

[2235] (Output): Optimized solution

[2236] Step 10:

[2237] Receiving feedback and recording it in a database

[2238] The user tries the proposed solutions and inputs the results into the terminal and sends them to the server. The server receives this feedback and records it in a database. The analyzed emotional data is also recorded at the same time.

[2239] (Input): User feedback

[2240] (Output): Feedback and emotion data recorded in a database

[2241] The above processing steps allow the user to quickly and appropriately resolve the problem, and also realize a response that takes into consideration the user's feelings.

[2242] The specific processing unit 290 transmits the result of t...

Claims

1. means for receiving defect information input by a user; means for analyzing the received defect information by natural language processing; A means for searching for similar past cases from a database based on the analysis results; A means for calculating the probability of occurrence of a failure cause using statistical data from the search results; means for estimating the cause of a failure based on the calculated probability; A means for generating an optimal solution based on the estimated cause of the failure; means for presenting the generated solution to a user; A system including:

2. 2. The system of claim 1, further comprising means for receiving and recording in a database the results of the user's actions based on the proposed solution.

3. The system of claim 1 further comprising means for generating and presenting to the user a list of replacement parts required for the solution.

Images