System

Abstract

A system is provided.SOLUTION: A system for automating maintenance and conservation of a facility at the time of occurrence of a failure, comprising: means for acquiring and learning a past failure case, a handling method, configuration information of the facility, and an instruction manual; means for receiving and analyzing real-time data and an alarm; means for specifying and recording the failure from the received data; means for generating a temporary handling method and a permanent measure based on the failure; and means for creating and distributing an operation guide for on-site handling.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In equipment maintenance and upkeep work, a fast and accurate response is required when a failure occurs. However, the causes of failure are diverse, and identifying and responding to them requires highly specialized skills and time. Furthermore, a delayed response can lead to major problems such as production delays and a loss of corporate credibility. However, with previous systems, it has been difficult to automate and centralize such failure responses. The purpose of this invention is to solve these problems and provide a system that can respond quickly and appropriately to equipment failures. [Means for solving the problem]

[0005] To solve the above problems, the present invention provides a system for automating equipment maintenance and upkeep when a fault occurs. This system includes a means for acquiring and learning from past fault cases, response methods, equipment configuration information, and instruction manuals, a means for receiving and analyzing real-time data and alarms, a means for identifying and recording faults from the received data, a means for generating temporary and permanent countermeasures based on the fault, and a means for creating and distributing an on-site operation guide. This enables a quick and effective response when a fault occurs, improving the reliability and efficiency of the equipment.

[0006] A "failure" is an event that prevents normal operation of equipment, resulting in disruption to business or production.

[0007] "Maintenance" means inspection, repair, and replacement work carried out periodically or as needed to maintain the proper operation of equipment.

[0008] "Maintenance" refers to preventive inspection and management activities carried out to prevent deterioration and breakdown of equipment.

[0009] A "system" is an organized whole in which multiple elements work together to achieve a specific function.

[0010] "Device" means a mechanical or electronic device configured to perform a particular function.

[0011] "Real-time data" refers to data that indicates the current status of equipment and is continuously transmitted from sensors and monitors.

[0012] An "alarm" is a signal or notification that warns of an abnormality or failure when it is detected.

[0013] "Generative AI" is artificial intelligence that learns from past data and examples and makes judgments and suggestions based on new data.

[0014] A "temporary solution" is a quick response measure to temporarily solve a problem when a failure occurs.

[0015] "Permanent measures" are long-term measures to make fundamental repairs and improvements to prevent similar problems from recurring.

[0016] An "operation guide" is a document or instruction manual that shows specific operating procedures on-site. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an operation guide for on-site response.

[0039] Server-side roles

[0040] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It then uses this data to train the generative AI model. When real-time data and alarms are received, the data is analyzed and the failure is identified. At this time, the generative AI model can be used to identify the cause of the failure quickly and with high accuracy. Based on the identified cause, it generates a temporary solution and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device.

[0041] The role of the terminal

[0042] The terminal displays notifications and operation guides received from the server to the user. The user follows the instructions displayed on the terminal to take temporary measures such as restarting equipment or replacing parts. After completing the measures, the results are fed back to the server. This feedback information is also used for further model training.

[0043] User Role

[0044] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. This information is added to the server's fault response database and used for future learning.

[0045] Specific examples

[0046] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[0047] 1. Receiving and analyzing real-time data (server)

[0048] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[0049] 2. Generate a temporary solution (server)

[0050] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[0051] 3. Distribution of operation guide (server and terminal)

[0052] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[0053] 4. On-site response (user)

[0054] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[0055] 5. Feedback Analysis (Server)

[0056] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[0057] In this way, the system of the present invention enables a quick and appropriate response when a fault occurs, thereby improving the reliability and efficiency of the facility.

[0058] The processing flow will be explained below.

[0059] Server-side processing steps

[0060] Step 1:

[0061] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the proposed company, and stores the received data in a local database.

[0062] Step 2:

[0063] The server trains a generative AI model based on the stored data, using machine learning algorithms to detect patterns, causes, and solutions for failures.

[0064] Step 3:

[0065] The server receives real-time data and alarm signals sent from each facility, including the facility's operating status and abnormality detection information.

[0066] Step 4:

[0067] The server analyzes real-time data and uses an anomaly detection algorithm to determine whether there is a problem. If an anomaly is detected, the generating AI identifies the cause.

[0068] Step 5:

[0069] The server generates a temporary solution based on the identified cause of the failure, for example, a specific procedure such as "restart the pump."

[0070] Step 6:

[0071] Along with providing instructions on how to deal with the problem, the server also generates a permanent solution, which is intended to prevent the problem from recurring in the future.

[0072] Step 7:

[0073] If on-site assistance is required, the server will create a detailed operation guide, including reboot procedures and safety precautions.

[0074] Step 8:

[0075] The server distributes the generated temporary solution, permanent solution plan, and operation guide to the terminal.

[0076] Terminal processing steps

[0077] Step 1:

[0078] The terminal displays the notifications and operation guides received from the server to the user.

[0079] Step 2:

[0080] The terminal provides the user with temporary and permanent solutions in real time and displays instructions on how to deal with the problem.

[0081] User processing steps

[0082] Step 1:

[0083] The user follows the temporary solution displayed on the terminal and performs a specific operation, such as restarting the pump.

[0084] Step 2:

[0085] Once the user has completed their on-site visit, they enter the results into the terminal, recording success or failure and any additional observations.

[0086] Feedback processing steps

[0087] Step 1:

[0088] The terminal transmits the response result feedback from the user to the server.

[0089] Step 2:

[0090] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[0091] In this way, the server, terminal, and user can cooperate to proceed with the troubleshooting process, thereby improving the reliability and efficiency of the facility.

[0092] Example 1

[0093] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0094] When a fault occurs during equipment maintenance and upkeep, a quick and accurate response is required, but it was difficult to achieve this efficiently with conventional systems. Furthermore, identifying the cause of the fault and formulating a countermeasure required human resources and specialized knowledge, which resulted in a time-consuming response and adversely affected the equipment's operating rate. This created a need to improve the reliability and efficiency of the equipment.

[0095] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0096] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and warning signals, means for quickly and accurately identifying failures from the received data using a generative AI model, means for generating temporary and permanent solutions based on the failure, means for generating and distributing operating procedures for on-site response to terminals, and means for receiving and analyzing feedback from users. This enables quick and accurate response when a failure occurs, improves the equipment availability rate, and makes it possible to streamline maintenance and repair work.

[0097] "Past failure cases" are records of problems or failures that have previously occurred in the equipment.

[0098] "Response methods" refer to the specific measures and repair procedures taken when equipment failure occurs.

[0099] "Facility configuration information" refers to information such as detailed specifications, layout, and connection status of each part of the facility.

[0100] An "instruction manual" is a document that contains instructions on how to operate and maintain equipment.

[0101] "Real-time data" refers to data collected in real time regarding the operation status and performance of equipment.

[0102] A "warning signal" is a notification issued when an abnormal value or unexpected behavior is detected in equipment.

[0103] A "generative AI model" is an artificial intelligence that learns using large amounts of data and makes highly accurate predictions and judgments for specific tasks.

[0104] "Failure" refers to a problem or malfunction that prevents the equipment from functioning properly.

[0105] A "temporary solution" is a specific procedure for temporarily resolving a problem when a failure occurs.

[0106] A "permanent fix" is a measure or repair that will be effective over the long term to prevent the problem from recurring.

[0107] "Operating procedures" are descriptions of specific steps and precautions that users should follow when operating equipment.

[0108] A "terminal" is a device, such as a tablet or smartphone, through which a user receives operating instructions and notifications.

[0109] "Feedback" refers to the act of sending the results of operations performed by a user or information observed by a user to a server.

[0110] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes a generative AI model to identify the cause of the fault, propose temporary and permanent solutions, and provide an operational guide for on-site response. The program for this system is described in detail below.

[0111] The server collects past failure cases, response methods, equipment configuration information, and instruction manuals from the companies to which the proposals are made. This collected data is used to train a generative AI model (e.g., GPT-4 (registered trademark)) using data analysis tools such as TENSORFLOW (registered trademark) and PyTorch.

[0112] After training, the server receives real-time data and warning signals from the equipment. This real-time data includes information such as the operating status, pressure, and temperature of pumps and motors. The received data is immediately analyzed and input into the generative AI model, allowing the AI ​​model to identify faults quickly and with high accuracy.

[0113] Based on the identified cause of the problem, the server generates temporary and permanent solutions. Temporary solutions are specific steps to quickly resolve the problem, such as "restart the pump." Permanent solutions are long-term solutions to prevent recurrence.

[0114] Next, the server generates an operation guide detailing the on-site operation procedures and distributes it to the user's device. The operation guide includes specific operation procedures, necessary precautions, and safety measures. The device displays the notifications and operation guide received from the server to the user.

[0115] The user performs on-site operations by following the operation guide displayed on the terminal. For example, if a pump malfunction is discovered, the user follows the instructions displayed on the terminal to restart the pump. Once the restart is complete, the results are entered into the terminal and sent to the server.

[0116] The server analyzes the feedback received from users, stores the response results and additional information in a database, and uses them to further train the generative AI model to improve the accuracy of future incident responses.

[0117] Specific examples

[0118] For example, if a pump malfunctions on a factory production line, the system will operate as follows:

[0119] 1. Data collection and learning

[0120] The server collects data on past pump failure cases and how to deal with them, and trains the AI ​​model.

[0121] 2. Receiving and analyzing real-time data

[0122] The server receives real-time operational status data from the production line and analyzes it using a generative AI model.

[0123] 3. Generation of temporary solutions

[0124] Based on the analysis results of the AI ​​model, the server generates a solution such as "Perform the pump restart procedure."

[0125] 4. Distribution of operation guide

[0126] The server distributes an operation guide detailing the reboot procedure to the terminal and instructs the user.

[0127] 5. Implementation of local response

[0128] The user restarts the pump by following the restart procedure displayed on the terminal.

[0129] 6. Feedback Analysis

[0130] The server receives feedback data, including whether the reboot was successful, and feeds this back into the AI ​​model to improve its accuracy in the future.

[0131] Prompt Sentence Examples

[0132] "Pump pressure has exceeded the specified value. Please identify the cause."

[0133] "Please provide pump restart procedure."

[0134] "Please detail the pump restart procedure."

[0135] "The user has performed a pump restart. What are the next steps?"

[0136] "The pump restart was successful. Is there anything else I should do or be careful of?"

[0137] The above is a specific description of the embodiment of the present invention. This method enables a quick and accurate response when a fault occurs, and achieves efficient maintenance and upkeep of the equipment.

[0138] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0139] Step 1:

[0140] The server collects past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals. This data is used to train the generative AI model.

[0141] How it works: The server connects to the company's database and automatically downloads the necessary data. The input data includes a history of incidents and their resolutions, which are organized by category and saved in a folder for model training. The output is an organized dataset.

[0142] Step 2:

[0143] The server trains a generative AI model based on the collected data, for example, by using data analysis tools such as TensorFlow or PyTorch to train the model (e.g., GPT-4).

[0144] How it works: The server inputs historical data into the AI ​​model and runs a training process to learn trends and patterns in the data. Collected failure cases and solutions are provided as input data, and a trained generative AI model is obtained as output.

[0145] Step 3:

[0146] The server receives real-time data and warning signals sent from the equipment, including the equipment's operating status, pressure, temperature, etc.

[0147] How it works: The server continuously collects real-time data from sensors and stores it in a database. During this process, data is updated either hourly or by events. Real-time measurements are provided as input data, and an updated database is obtained as output.

[0148] Step 4:

[0149] The server analyzes the received real-time data and uses generative AI models to quickly identify faults.

[0150] How it works: The server inputs real-time data into the AI ​​model and performs analysis to detect abnormal patterns and signs of failure. Real-time sensor data is provided as input data, and detailed information about identified failures is obtained as output.

[0151] Step 5:

[0152] The server generates a temporary solution based on the identified cause of the problem, which includes specific steps to quickly resolve the problem.

[0153] Specific operation: The server documents temporary solutions based on the fault handling scenarios generated by the AI ​​model. Identified fault information is provided as input data, and the documented temporary solution procedures are obtained as output.

[0154] Step 6:

[0155] The server creates a detailed operation guide based on the temporary solution and distributes it to the user's device. The operation guide includes specific operating procedures and important points to note.

[0156] Specific operation: The server creates an operation guide based on the temporary solution procedure and sends it to the terminal in a displayable format. The temporary solution procedure is provided as input data, and the operation guide delivered to the user is obtained as output.

[0157] Step 7:

[0158] The user follows the operation guide displayed on the terminal to perform on-site operations, such as restarting a pump.

[0159] Specific operation: The user follows the procedures displayed on the terminal to operate the necessary equipment. The operation guide is provided as input data, and the results of the executed operations are obtained as output.

[0160] Step 8:

[0161] The server analyzes the feedback received from users and stores the responses and additional information in a database, which is used to further train the generative AI model.

[0162] Specific operation: The server analyzes the operation results and observation data received from the user and adds them to the database. Feedback information is provided as input data, and an updated learning database is obtained as output.

[0163] The above is an explanation of the specific operation of each processing step and its inputs and outputs. This system enables quick and highly accurate response when a fault occurs, improving the efficiency of equipment maintenance and repair.

[0164] (Application example 1)

[0165] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0166] When equipment failure occurs in a factory, a prompt and appropriate response is required. However, current systems can be slow to identify the cause of the failure and provide a solution, resulting in prolonged equipment downtime. In addition, because operators respond manually, there is a high risk of error. Therefore, there is a need for a system that can automatically identify the cause of the failure, take action, and provide feedback in real time.

[0167] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0168] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary responses and permanent measures based on the failure, means for creating and distributing an operation guide for on-site response, means for the factory robot to monitor the operating status in real time and detect abnormalities, means for identifying the cause of the failure using a generative AI model learned on the server, and means for providing an operation guide to on-site operators so that they can respond as instructed. This enables quick and accurate response when a failure occurs.

[0169] "Failure" refers to a condition in which equipment or systems are not functioning properly.

[0170] "Maintenance and preservation" refers to a set of activities that involve the upkeep and repair of equipment and systems.

[0171] A "system" refers to a single mechanism in which multiple elements function in relation to one another.

[0172] "Past failure cases" refers to specific cases and records of failures that have occurred in the past.

[0173] "Response methods" refers to the specific methods and procedures for dealing with problems when they occur.

[0174] "Configuration information" refers to information about each component of a facility or system.

[0175] An "instruction manual" refers to a document that describes how to use equipment or a system and precautions to take.

[0176] "Learning" refers to the process of using data to train a generative AI model and gain knowledge.

[0177] "Real-time data" refers to operational data that shows the current status of equipment or systems.

[0178] An "alarm" is a warning signal that notifies you that an abnormality or problem has occurred.

[0179] "Analysis" refers to the process of analyzing data and information in detail to clarify meaning and relationships.

[0180] "Identification" refers to clearly indicating the cause of a problem or issue.

[0181] "Recording" refers to storing information for future reference.

[0182] A "temporary solution" refers to a temporary solution to a problem that has occurred.

[0183] "Permanent fix" refers to a permanent solution that takes measures to prevent the same problem from recurring.

[0184] "Operation Guide" means instructions that provide specific operating procedures to be followed by local operators.

[0185] "Distribution" refers to the act of sending information or data to a specific device or user.

[0186] A "factory robot" refers to a robot that performs automated tasks in a factory.

[0187] "Operating status" refers to the current operation or state of a facility or system.

[0188] A "generative AI model" refers to an artificial intelligence model that learns from large amounts of data to perform specific tasks.

[0189] "On-site operator" refers to the person in charge of operating facilities and equipment at a factory or on-site.

[0190] The present invention relates to a system for automating the maintenance and upkeep of equipment in a factory production line and for responding quickly and accurately when a fault occurs. A detailed description of an embodiment of this system will be given below.

[0191] System Configuration

[0192] It consists of a server, factory robots, and terminals used by operators. The server has a generative AI model that has learned past failure cases, response methods, equipment configuration information, and instruction manuals.

[0193] Server Roles

[0194] The server performs the following series of processes.

[0195] 1. Learning from failure cases: The server acquires past failure cases, response methods, equipment configuration information, and instruction manuals, and uses these to train the generative AI model. Specifically, the generative AI model is trained using artificial intelligence libraries such as TensorFlow and PyTorch.

[0196] 2. Real-time data analysis: Receive real-time data and alarms sent from factory robots and analyze this data, including operational data such as temperature and pressure.

[0197] 3. Identifying the cause of the failure: Using a generative AI model, the cause of the failure is identified from the analysis results.

[0198] 4. Generate temporary and permanent solutions: Based on the identified causes, generate temporary and permanent solutions.

[0199] 5. Creation and distribution of operation guides: If on-site support is required, a detailed operation guide will be created and distributed to the device. This guide will include specific operating procedures and important points to note.

[0200] Device Role

[0201] The terminal displays notifications and operation guides received from the server. The user (operator) follows the instructions displayed on the terminal to take temporary measures such as restarting the equipment or replacing parts. For example, if a pump abnormality is detected, a restart instruction will be displayed on the terminal. The operator follows this instruction and restarts the pump. Once the restart is complete, the results are entered into the terminal and sent to the server. This information is added to the fault response database on the server side and will be used for future learning.

[0202] Specific examples

[0203] For example, the system operates as follows when a pump malfunctions on a factory production line.

[0204] 1. Receiving and analyzing real-time data (server): The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc.

[0205] 2. Generation of temporary solution (server): If the generative AI model determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[0206] 3. Distribution of operation guide (server and terminal): The server creates a detailed operation guide based on this temporary solution and distributes it to the terminal.

[0207] 4. On-site response (user): The user follows the instructions displayed on the terminal to restart the pump. The results are entered into the terminal and sent to the server.

[0208] 5. Feedback Analysis (Server): The server receives feedback from the user and analyzes the results. If the restart is successful, the information is added to the database for future learning.

[0209] Prompt example

[0210] For example, by inputting a prompt such as, "The robot's sensor has detected a high temperature (75°C). Please tell me what the possible problems are and how to deal with them," into a generative AI model, it is possible to quickly and accurately derive a specific response method.

[0211] The interaction of the above elements enables rapid and accurate response when a fault occurs, improving the efficiency and reliability of the factory.

[0212] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0213] Step 1:

[0214] The server retrieves past failure cases, response methods, equipment configuration information, and instruction manuals, and uses this data to train a generative AI model. Specifically, this information is read from the database and the AI ​​model is trained using TensorFlow and PyTorch. The model's input is data on failure cases and response methods, and its output is the identification of failures and recommended response methods.

[0215] Step 2:

[0216] Real-time data and alarms from factory robots are sent to a server. Such data includes temperature, pressure, operation status, etc. The server receives this data and parses it in JSON format. The input is the robot's sensor data, and the output is recorded as parsed data.

[0217] Step 3:

[0218] The server analyzes the received real-time data and detects anomalies. It uses a generative AI model to identify the cause of the fault based on the data. The input is sensor data and alarm information, and the output is the identified cause of the fault. The AI ​​model performs data analysis and fault identification with high accuracy.

[0219] Step 4:

[0220] The server generates temporary and permanent solutions based on the identified cause of the problem. First, it generates a temporary solution and issues a specific instruction, such as "restart the pump." The input is the identified cause of the problem, and the output is a proposal for temporary and permanent solutions.

[0221] Step 5:

[0222] The server creates a detailed operation guide based on the generated temporary solution and distributes it to the terminal. The guide includes operating procedures and important points to note. The input is the temporary solution data, and the output is the operation guide.

[0223] Step 6:

[0224] The user (operator) follows the operation guide received from the terminal and takes measures such as restarting equipment or replacing parts. For example, if an instruction to restart a pump is received, the user restarts the pump using the operation panel. The input is the terminal's operation guide, and the output is the result of the action taken.

[0225] Step 7:

[0226] Once the user has completed the action, they enter the results into their device and send them to the server. Specifically, it records whether the reboot was successful or not, and whether any other abnormalities occurred. The input is the data on the action results, and the output is sent to the server as feedback.

[0227] Step 8:

[0228] The server receives and analyzes user feedback. Successful solutions and other information are added to a database for future learning. The input is feedback data, and the output is an updated model and database. This allows the AI ​​model to continually improve, resulting in more accurate fault identification and response.

[0229] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0230] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system uses generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an on-site operation guide. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide response support that takes into account the user's psychological state.

[0231] Server-side roles

[0232] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It stores the received data in a local database. Next, it uses this data to train a generative AI model. It receives real-time data and alarms, analyzes the data, and identifies the failure. It generates a temporary solution based on the identified cause of the failure and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device. By combining it with an emotion engine, it analyzes the user's real-time emotional state and adjusts the response content and notification method.

[0233] The role of the terminal

[0234] The device displays notifications and operation guides received from the server to the user. It also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server, allowing the server to adjust its response methods and notification content according to the user's emotional state.

[0235] User Role

[0236] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. The system also provides psychological support as needed based on the user's emotional state analyzed by the emotion engine.

[0237] Specific examples

[0238] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[0239] 1. Receiving and analyzing real-time data (server)

[0240] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[0241] 2. Generate a temporary solution (server)

[0242] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[0243] 3. Distribution of operation guide (server and terminal)

[0244] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[0245] 4. On-site response (user)

[0246] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[0247] 5. Sentiment analysis and notification adjustment (device and server)

[0248] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[0249] 6. Feedback Analysis (Server)

[0250] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[0251] In this way, the system of the present invention enables a prompt and appropriate response when a failure occurs, improves the reliability and efficiency of the facility, and also reduces the psychological burden on the user.

[0252] The processing flow will be explained below.

[0253] Server-side processing steps

[0254] Step 1:

[0255] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals, and stores this data in a local database.

[0256] Step 2:

[0257] The server trains a generative AI model based on the stored data, using machine learning algorithms to learn patterns, causes, and solutions to problems.

[0258] Step 3:

[0259] The server receives real-time data and alarms sent from each facility, including sensor information and error logs.

[0260] Step 4:

[0261] The server analyzes real-time data and applies anomaly detection algorithms to determine whether a fault exists. If a fault is detected, the generative AI identifies the cause.

[0262] Step 5:

[0263] The server generates a temporary solution based on the identified cause of the failure, for example, a specific instruction to "restart the pump."

[0264] Step 6:

[0265] The server also generates permanent countermeasures to prevent future recurrence of the problem, such as "strengthen regular inspections of pumps."

[0266] Step 7:

[0267] If on-site support is required, the server will create a detailed operation guide, which includes reboot procedures and safety precautions.

[0268] Step 8:

[0269] The server distributes temporary solutions, permanent solutions, and operation guides to the terminal.

[0270] Terminal processing steps

[0271] Step 1:

[0272] The terminal displays the notifications and operation guides received from the server to the user.

[0273] Step 2:

[0274] The terminal analyzes the user's emotional state in real time via an emotion engine and transmits the information to the server.

[0275] User processing steps

[0276] Step 1:

[0277] The user follows the temporary solution displayed on the terminal and performs specific operations, such as "restart the pump."

[0278] Step 2:

[0279] Once the on-site troubleshooting is complete, the user enters the results into the terminal. Specific results such as "reboot successful / failed" are recorded.

[0280] Feedback processing steps

[0281] Step 1:

[0282] The terminal transmits the response result feedback from the user to the server.

[0283] Step 2:

[0284] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[0285] Emotion Engine Processing Steps

[0286] Step 1:

[0287] The device uses an emotion engine to analyze the user's emotional state, including facial expression recognition and voice analysis.

[0288] Step 2:

[0289] The device sends the analysis results to the server and requests that the server flexibly change the notification content and response methods according to the user's emotional state.

[0290] Step 3:

[0291] The server adjusts the content of notifications based on the user's emotional state. For example, if the user is feeling stressed, the server shortens the content of notifications and takes measures to reduce the user's psychological burden.

[0292] As a concrete example, the sequence of events that occurs when a pump malfunctions is shown below:

[0293] 1. Receiving and analyzing real-time data (server)

[0294] The server receives real-time data sent from the production line, analyzes the data, and detects any abnormalities in the pump.

[0295] 2. Generate a temporary solution (server)

[0296] The server generates a workaround: "Perform a pump restart."

[0297] 3. Distribution of operation guide (server and terminal)

[0298] The server creates a detailed operation guide and distributes it to the device, which includes instructions and important points to note.

[0299] 4. On-site response (user)

[0300] The user follows the instructions on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[0301] 5. Sentiment analysis and notification adjustment (device and server)

[0302] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server, which then changes the notification content to be more concise and reassuring if the user is feeling stressed.

[0303] 6. Feedback Analysis (Server)

[0304] The server receives feedback from the user, analyzes the results, and if successful, adds the information to the database for future learning.

[0305] Example 2

[0306] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0307] In the maintenance and upkeep of equipment, a fast and accurate response is required when a failure occurs. However, conventional systems have difficulty in responding in real time, which places a heavy psychological burden on users. Furthermore, it is often difficult for inexperienced users to select the appropriate response method. Furthermore, because the system is unable to respond in a way that takes into account the user's emotions and stress level, it can be difficult for users to understand the operation instructions.

[0308] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, a means for receiving and analyzing real-time data and alarms, and a means for analyzing user emotions and adjusting response methods and notification contents based on the analysis results. This enables quick and accurate failure analysis and flexible response according to the user.

[0309] "Past failure cases" refers to cases of abnormalities or failures in equipment or systems that have occurred in the past.

[0310] "Response methods" refer to the procedures and methods for dealing with failures or abnormalities when they occur.

[0311] "Facility configuration information" refers to detailed information such as the facility's blueprint, construction method, and component configuration.

[0312] An "instruction manual" refers to a document that describes how to operate equipment or systems, how to maintain them, and precautions to take.

[0313] "Real-time data" refers to data that instantly records the operating status of an ongoing process or machine.

[0314] An "alarm" refers to a warning signal or message that notifies personnel when an abnormality occurs in equipment or a system.

[0315] A "temporary solution" refers to a means of quickly and temporarily resolving a problem or abnormality when it occurs.

[0316] "Permanent measures" refer to measures or improvements that are taken on a permanent basis to prevent recurrence.

[0317] An "operation guide" refers to a detailed instruction manual that explains the operating procedures for equipment or systems.

[0318] "Means for analyzing emotions" refers to techniques and methods for analyzing a user's psychological state and emotions in real time.

[0319] "Means for adjusting the response method and notification content based on the analysis results" refers to a method for dynamically changing the optimal response method and notification content based on the results of the user's emotion analysis.

[0320] This invention relates to a system that automates the maintenance and upkeep of equipment and responds quickly and accurately when a fault occurs. This system deals with faults through cooperation between a server, a terminal, and a user.

[0321] Server processing

[0322] The server first obtains past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the client company, and stores them in a local database, allowing for centralized management of the data needed to troubleshoot.

[0323] This data is then used to train a generative AI model using machine learning libraries such as TensorFlow and PyTorch. Once the generative AI model is trained, the server receives real-time data and alarms, and analyzes this data to identify faults.

[0324] When a fault is identified, the server generates temporary and permanent solutions. The temporary solutions include specific operational procedures and deliver operation guides to the user. For example, if a pump abnormality is detected, the server generates a guide instructing the user to "restart the pump" and delivers it to the user's device.

[0325] Furthermore, the server is equipped with an emotion engine that can analyze the user's emotional state. Based on the results of this analysis, the notification content and response method can be flexibly adjusted. For example, if the user is feeling stressed, the tone and content of the notification can be changed to provide psychological support.

[0326] Processing by the terminal

[0327] The device displays notifications and operation guides received from the server to the user. For example, it displays detailed instructions and precautions for restarting the pump. The device also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server. The server's responses and notifications are then adjusted appropriately based on the user's emotional state.

[0328] User Action

[0329] The user performs operations according to the operation guide displayed on the terminal. For example, if an instruction to restart the pump is given, the user restarts the pump according to the instruction. Once the restart is complete, the result is entered into the terminal and sent to the server.

[0330] As a concrete example, consider a case where a pump malfunction occurs on a factory production line. The server receives real-time data sent from the production line, and the generative AI model identifies the malfunction. As a result, a solution to the problem, such as "restart the pump," is generated and sent to the user's device. The user follows the device's instructions to restart the pump. At that time, the device analyzes the user's emotional state and adjusts the notification content if the user is under high stress.

[0331] An example of a prompt sentence could be a specific question such as, "Please tell me what to do if a pump abnormality is detected." In this way, the system enables a quick and accurate response when a fault occurs, improving the reliability and efficiency of the equipment while also reducing the psychological burden on the user.

[0332] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0333] Step 1:

[0334] Receiving and storing data (server)

[0335] The server uses an API to receive information from the client company, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals. The inputs include various data files and data streams. The received data is sequentially saved in a local database. The data consistency is also checked, and any missing information is checked. The output is a structured database entry.

[0336] Step 2:

[0337] Generative AI model training (server)

[0338] The stored data is used to train a generative AI model. The server uses machine learning libraries such as TensorFlow and PyTorch to preprocess the data and train the model. A training dataset is required as input, and a trained AI model is obtained as output. Specific operations include loading data, preprocessing, model definition, compilation, and training.

[0339] Step 3:

[0340] Receiving and analyzing real-time data (server)

[0341] The server receives real-time sensor data sent from a production line or other device via an API. The input is a data stream that fluctuates over time. The received data is analyzed by a generative AI model to determine whether an anomaly exists. The output is a result regarding whether an anomaly exists or not. Specific operations include receiving the data, analyzing it using the model, and detecting an anomaly.

[0342] Step 4:

[0343] Creation and notification of temporary solutions (server)

[0344] When an abnormality is detected, the server identifies the cause and generates a temporary solution. The input is the analysis result, and the output is a notification message that includes a specific solution. Specific operations include generating a solution, determining the content of the notification, and sending it to the user terminal. The solution includes specific procedures such as restarting the system and checking the status.

[0345] Step 5:

[0346] Distribution of operation guide (server and terminal)

[0347] The server creates a detailed operation guide based on the generated temporary solution and delivers it to the user's terminal. The generated solution is required as input, and the output is an operation guide that is displayed to the user. Specific operations include creating the guide and delivering it to the terminal. The terminal receives it and displays it to the user.

[0348] Step 6:

[0349] On-site response (user)

[0350] The user performs operations according to the instructions displayed on the terminal. For example, if instructed to restart a pump, the user operates according to the procedure. The input is the operation guide, and the output is the result of the operation. Specific operations include actual operations on site and feedback of the results.

[0351] Step 7:

[0352] Sentiment analysis and notification adjustment (device and server)

[0353] The device uses an emotion engine to analyze the user's emotional state in real time and transmits that information to the server. The input is the user's biometric information and voice data, and the output is the emotion analysis results. The server adjusts the notification content and response methods based on these results. Specific operations include analyzing the emotion data, transmitting the analysis results to the server, and flexibly changing the notification content.

[0354] Step 8:

[0355] Receiving and analyzing feedback (server)

[0356] The server receives feedback from users and analyzes the corresponding results. The input is the user's feedback information, and the output is the analysis results and additional information added to the database. Specific operations include receiving feedback, analyzing it, saving it to the database, and using it as future model training data.

[0357] (Application example 2)

[0358] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0359] Conventional equipment maintenance and repair systems can sometimes have difficulty responding quickly and appropriately when a fault occurs. In particular, when workers are not familiar with dealing with faults, delays and errors can occur. Furthermore, when workers are in an unstable psychological state, it becomes difficult to respond efficiently, which can lead to reduced reliability and efficiency of the equipment. There is a need for a system that can solve these problems and carry out equipment maintenance and repair work more efficiently and accurately.

[0360] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0361] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary and permanent response methods based on the failure, means for creating and distributing an on-site operation guide, and means for analyzing the emotional state of users in real time and adjusting response methods and notification content based on that information. This enables prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the equipment.

[0362] "Past failure cases" refers to specific records of past failures that occurred in the equipment and the history of how they were dealt with.

[0363] "Response methods" refer to the specific actions and procedures to be taken when a failure occurs, and the related processes.

[0364] "Facility configuration information" refers to data relating to the physical and logical components of the facility in question.

[0365] "Instruction manual" refers to guidelines and manuals on how to use, maintain, and troubleshoot equipment.

[0366] "Real-time data" refers to data obtained from equipment regarding its current operating conditions and parameters.

[0367] An "alarm" refers to a warning signal issued when an abnormality or failure in equipment is detected.

[0368] "Emotional state" refers to a state that indicates a user's psychological state or emotional response.

[0369] "Analysis" refers to analyzing data and extracting meaningful patterns and information from it.

[0370] "Identifying the fault" refers to clarifying the cause and location of an abnormality or problem that has occurred.

[0371] "Temporary measures" refer to temporary measures to improve the situation in the short term when a problem occurs.

[0372] "Permanent measures" refer to long-term measures to eliminate the root cause to prevent recurrence and improve the reliability of the equipment.

[0373] An "operation guide" refers to an instruction manual that shows the steps and precautions a user should take when performing a specific operation.

[0374] "Notification content" refers to messages and instructions sent from the system to the user.

[0375] The present invention is a system for automating equipment maintenance and upkeep in a factory, and in particular, provides real-time troubleshooting using smart glasses and flexible notifications based on the emotional state of the user. An embodiment of this system will be described below.

[0376] Server-side roles

[0377] The server plays a central role in collecting and analyzing data within the factory. Specifically, it processes and calculates the data using the following hardware and software:

[0378] Hardware: High-performance server

[0379] Software: TensorFlow (for training generative AI models), database management system (DBMS)

[0380] The server first obtains past failure cases, response methods, equipment configuration information, and instruction manuals from the equipment and stores this data in a local database. It then uses this data to train a generative AI model. It receives real-time data and alarms from remote sensors and cameras, analyzes them, and identifies failures. Based on the identified failures, it generates temporary and permanent solutions and notifies the user. If on-site operation is required, it also creates a detailed operation guide and distributes it to the smart glasses.

[0381] The server also incorporates an emotion engine to analyze the user's emotional state, analyzing the user's real-time emotional state (camera footage and audio) and adjusting the response method and notification content based on that information.

[0382] The role of the terminal

[0383] Smart glasses are used as the device, and the device has the following functions:

[0384] Hardware: Smart glasses (camera, display, microphone)

[0385] Software: OpenCV (image analysis), Emotion API (emotion analysis)

[0386] The device displays notifications and operation guides from the server to the user. It also analyzes the user's emotional state in real time using data acquired from the camera and microphone and sends that information to the server, allowing the server to provide the optimal response based on the user's emotional state.

[0387] User Role

[0388] The user operates the device (smart glasses) by following the notifications and operation guide displayed on the device. Specific operation examples are shown below.

[0389] For example, if a pump malfunction occurs on a production line, the following measures will be taken.

[0390] 1. Receiving and analyzing real-time data (server)

[0391] The server receives real-time data showing the pump's operating status and analyzes it using a generative AI model to identify the cause of the fault.

[0392] 2. Generate a temporary solution (server)

[0393] If the server determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[0394] 3. Distribution of operation guide (server and terminal)

[0395] Based on this temporary solution, the server creates an operation guide including reboot procedures and precautions, and distributes it to the smart glasses.

[0396] 4. On-site response (user)

[0397] The user follows the instructions displayed on the smart glasses to restart the pump, and the results are entered into the smart glasses and sent to the server.

[0398] 5. Sentiment analysis and notification adjustment (device and server)

[0399] The smart glasses use a camera and microphone to analyze the user's emotional state (e.g., stress level) and send that information to a server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[0400] Here is an example of a prompt from a generative AI model:

[0401] "While the user is wearing the smart glasses, the AI ​​model should analyze images of machinery with bearing abnormalities and detect the bearing abnormality. Also, if the user's stress level is high, the smart glasses should display a message instructing the user to 'take a deep breath and calm down' and restart the device."

[0402] As a result, the system of the present invention enables a prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the facility.

[0403] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0404] Step 1:

[0405] The server collects data from within the factory, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals, and stores this information in a local database. This creates a database for subsequent failure analysis and countermeasure development.

[0406] Input: Past failure cases and instruction manuals obtained from the equipment

[0407] Output: Data stored in a local database

[0408] Step 2:

[0409] The server uses the collected data to train the generative AI model, which uses TensorFlow to analyze equipment configuration information and past failure cases to learn patterns when failures occur.

[0410] Input: Data stored in a local database

[0411] Output: Trained generative AI model

[0412] Step 3:

[0413] The server receives real-time data and alarms from remote sensors and cameras in the factory, including equipment operating status, pressure, temperature, and more.

[0414] Input: Real-time data from remote sensors and cameras

[0415] Output: Real-time monitoring data

[0416] Step 4:

[0417] The server analyzes the received real-time data and identifies faults using a generative AI model, which uses the received data as input to determine whether a fault has occurred and its cause.

[0418] Input: Real-time monitoring data

[0419] Output: Whether there is a fault and its cause

[0420] Step 5:

[0421] The server generates temporary solutions based on the identified faults (for example, if a pump needs to be restarted, it generates a restart procedure), and also suggests the necessary repairs or part replacements for permanent solutions.

[0422] Input: Identified fault information

[0423] Output: Temporary and permanent solutions

[0424] Step 6:

[0425] The server creates a detailed operation guide based on the generated temporary and permanent solutions and distributes it to the device (smart glasses).

[0426] Input: Temporary and permanent solutions

[0427] Output: Operation guide to terminal

[0428] Step 7:

[0429] The device (smart glasses) displays the operation guide received from the server to the user, who then follows the guide to perform the operation.

[0430] Input: Operation guide from the server

[0431] Output: Guide display for users

[0432] Step 8:

[0433] The device uses a camera and microphone to analyze the user's emotional state (e.g., stress level) in real time and transmits the information to a server, where the analysis is performed using an emotion engine.

[0434] Input: Data from the camera and microphone

[0435] Output: User's emotional state data

[0436] Step 9:

[0437] The server flexibly adjusts the notification content and response methods based on the user's emotional state data. For example, if the user is feeling stressed, the server may adjust the notification content to be gentler or provide instructions for additional breaks.

[0438] Input: User's emotional state data

[0439] Output: Adjusted notification content and action to be taken

[0440] Step 10:

[0441] The user follows the instructions displayed on the device (smart glasses) to take local action. For example, they restart the pump, enter the results into the device, and send them to the server.

[0442] Input: the instructions displayed on the terminal

[0443] Output: Send the restart result to the server

[0444] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0445] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0446] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0447] [Second embodiment]

[0448] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0449] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0450] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0451] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0452] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0453] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0454] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0455] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0456] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0457] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0458] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0459] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0460] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an operation guide for on-site response.

[0461] Server-side roles

[0462] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It then uses this data to train the generative AI model. When real-time data and alarms are received, the data is analyzed and the failure is identified. At this time, the generative AI model can be used to identify the cause of the failure quickly and with high accuracy. Based on the identified cause, it generates a temporary solution and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device.

[0463] The role of the terminal

[0464] The terminal displays notifications and operation guides received from the server to the user. The user follows the instructions displayed on the terminal to take temporary measures such as restarting equipment or replacing parts. After completing the measures, the results are fed back to the server. This feedback information is also used for further model training.

[0465] User Role

[0466] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. This information is added to the server's fault response database and used for future learning.

[0467] Specific examples

[0468] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[0469] 1. Receiving and analyzing real-time data (server)

[0470] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[0471] 2. Generate a temporary solution (server)

[0472] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[0473] 3. Distribution of operation guide (server and terminal)

[0474] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[0475] 4. On-site response (user)

[0476] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[0477] 5. Feedback Analysis (Server)

[0478] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[0479] In this way, the system of the present invention enables a quick and appropriate response when a fault occurs, thereby improving the reliability and efficiency of the facility.

[0480] The processing flow will be explained below.

[0481] Server-side processing steps

[0482] Step 1:

[0483] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the proposed company, and stores the received data in a local database.

[0484] Step 2:

[0485] The server trains a generative AI model based on the stored data, using machine learning algorithms to detect patterns, causes, and solutions for failures.

[0486] Step 3:

[0487] The server receives real-time data and alarm signals sent from each facility, including the facility's operating status and abnormality detection information.

[0488] Step 4:

[0489] The server analyzes real-time data and uses an anomaly detection algorithm to determine whether there is a problem. If an anomaly is detected, the generating AI identifies the cause.

[0490] Step 5:

[0491] The server generates a temporary solution based on the identified cause of the failure, for example, a specific procedure such as "restart the pump."

[0492] Step 6:

[0493] Along with providing instructions on how to deal with the problem, the server also generates a permanent solution, which is intended to prevent the problem from recurring in the future.

[0494] Step 7:

[0495] If on-site assistance is required, the server will create a detailed operation guide, including reboot procedures and safety precautions.

[0496] Step 8:

[0497] The server distributes the generated temporary solution, permanent solution plan, and operation guide to the terminal.

[0498] Terminal processing steps

[0499] Step 1:

[0500] The terminal displays the notifications and operation guides received from the server to the user.

[0501] Step 2:

[0502] The terminal provides the user with temporary and permanent solutions in real time and displays instructions on how to deal with the problem.

[0503] User processing steps

[0504] Step 1:

[0505] The user follows the temporary solution displayed on the terminal and performs a specific operation, such as restarting the pump.

[0506] Step 2:

[0507] Once the user has completed their on-site visit, they enter the results into the terminal, recording success or failure and any additional observations.

[0508] Feedback processing steps

[0509] Step 1:

[0510] The terminal transmits the response result feedback from the user to the server.

[0511] Step 2:

[0512] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[0513] In this way, the server, terminal, and user can cooperate to proceed with the troubleshooting process, thereby improving the reliability and efficiency of the facility.

[0514] Example 1

[0515] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0516] When a fault occurs during equipment maintenance and upkeep, a quick and accurate response is required, but it was difficult to achieve this efficiently with conventional systems. Furthermore, identifying the cause of the fault and formulating a countermeasure required human resources and specialized knowledge, which resulted in a time-consuming response and adversely affected the equipment's operating rate. This created a need to improve the reliability and efficiency of the equipment.

[0517] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0518] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and warning signals, means for quickly and accurately identifying failures from the received data using a generative AI model, means for generating temporary and permanent solutions based on the failure, means for generating and distributing operating procedures for on-site response to terminals, and means for receiving and analyzing feedback from users. This enables quick and accurate response when a failure occurs, improves the equipment availability rate, and makes it possible to streamline maintenance and repair work.

[0519] "Past failure cases" are records of problems or failures that have previously occurred in the equipment.

[0520] "Response methods" refer to the specific measures and repair procedures taken when equipment failure occurs.

[0521] "Facility configuration information" refers to information such as detailed specifications, layout, and connection status of each part of the facility.

[0522] An "instruction manual" is a document that contains instructions on how to operate and maintain equipment.

[0523] "Real-time data" refers to data collected in real time regarding the operation status and performance of equipment.

[0524] A "warning signal" is a notification issued when an abnormal value or unexpected behavior is detected in equipment.

[0525] A "generative AI model" is an artificial intelligence that learns using large amounts of data and makes highly accurate predictions and judgments for specific tasks.

[0526] "Failure" refers to a problem or malfunction that prevents the equipment from functioning properly.

[0527] A "temporary solution" is a specific procedure for temporarily resolving a problem when a failure occurs.

[0528] A "permanent fix" is a measure or repair that will be effective over the long term to prevent the problem from recurring.

[0529] "Operating procedures" are descriptions of specific steps and precautions that users should follow when operating equipment.

[0530] A "terminal" is a device, such as a tablet or smartphone, through which a user receives operating instructions and notifications.

[0531] "Feedback" refers to the act of sending the results of operations performed by a user or information observed by a user to a server.

[0532] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes a generative AI model to identify the cause of the fault, propose temporary and permanent solutions, and provide an operational guide for on-site response. The program for this system is described in detail below.

[0533] The server collects past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it proposes solutions. This collected data is used to train a generative AI model (e.g., GPT-4) using data analysis tools such as TensorFlow and PyTorch.

[0534] After training, the server receives real-time data and warning signals from the equipment. This real-time data includes information such as the operating status, pressure, and temperature of pumps and motors. The received data is immediately analyzed and input into the generative AI model, allowing the AI ​​model to identify faults quickly and with high accuracy.

[0535] Based on the identified cause of the problem, the server generates temporary and permanent solutions. Temporary solutions are specific steps to quickly resolve the problem, such as "restart the pump." Permanent solutions are long-term solutions to prevent recurrence.

[0536] Next, the server generates an operation guide detailing the on-site operation procedures and distributes it to the user's device. The operation guide includes specific operation procedures, necessary precautions, and safety measures. The device displays the notifications and operation guide received from the server to the user.

[0537] The user performs on-site operations by following the operation guide displayed on the terminal. For example, if a pump malfunction is discovered, the user follows the instructions displayed on the terminal to restart the pump. Once the restart is complete, the results are entered into the terminal and sent to the server.

[0538] The server analyzes the feedback received from users, stores the response results and additional information in a database, and uses them to further train the generative AI model to improve the accuracy of future incident responses.

[0539] Specific examples

[0540] For example, if a pump malfunctions on a factory production line, the system will operate as follows:

[0541] 1. Data collection and learning

[0542] The server collects data on past pump failure cases and how to deal with them, and trains the AI ​​model.

[0543] 2. Receiving and analyzing real-time data

[0544] The server receives real-time operational status data from the production line and analyzes it using a generative AI model.

[0545] 3. Generation of temporary solutions

[0546] Based on the analysis results of the AI ​​model, the server generates a solution such as "Perform the pump restart procedure."

[0547] 4. Distribution of operation guide

[0548] The server distributes an operation guide detailing the reboot procedure to the terminal and instructs the user.

[0549] 5. Implementation of local response

[0550] The user restarts the pump by following the restart procedure displayed on the terminal.

[0551] 6. Feedback Analysis

[0552] The server receives feedback data, including whether the reboot was successful, and feeds this back into the AI ​​model to improve its accuracy in the future.

[0553] Prompt Sentence Examples

[0554] "Pump pressure has exceeded the specified value. Please identify the cause."

[0555] "Please provide pump restart procedure."

[0556] "Please detail the pump restart procedure."

[0557] "The user has performed a pump restart. What are the next steps?"

[0558] "The pump restart was successful. Is there anything else I should do or be careful of?"

[0559] The above is a specific description of the embodiment of the present invention. This method enables a quick and accurate response when a fault occurs, and achieves efficient maintenance and upkeep of the equipment.

[0560] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0561] Step 1:

[0562] The server collects past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals. This data is used to train the generative AI model.

[0563] How it works: The server connects to the company's database and automatically downloads the necessary data. The input data includes a history of incidents and their resolutions, which are organized by category and saved in a folder for model training. The output is an organized dataset.

[0564] Step 2:

[0565] The server trains a generative AI model based on the collected data, for example, by using data analysis tools such as TensorFlow or PyTorch to train the model (e.g., GPT-4).

[0566] How it works: The server inputs historical data into the AI ​​model and runs a training process to learn trends and patterns in the data. Collected failure cases and solutions are provided as input data, and a trained generative AI model is obtained as output.

[0567] Step 3:

[0568] The server receives real-time data and warning signals sent from the equipment, including the equipment's operating status, pressure, temperature, etc.

[0569] How it works: The server continuously collects real-time data from sensors and stores it in a database. During this process, data is updated either hourly or by events. Real-time measurements are provided as input data, and an updated database is obtained as output.

[0570] Step 4:

[0571] The server analyzes the received real-time data and uses generative AI models to quickly identify faults.

[0572] How it works: The server inputs real-time data into the AI ​​model and performs analysis to detect abnormal patterns and signs of failure. Real-time sensor data is provided as input data, and detailed information about identified failures is obtained as output.

[0573] Step 5:

[0574] The server generates a temporary solution based on the identified cause of the problem, which includes specific steps to quickly resolve the problem.

[0575] Specific operation: The server documents temporary solutions based on the fault handling scenarios generated by the AI ​​model. Identified fault information is provided as input data, and the documented temporary solution procedures are obtained as output.

[0576] Step 6:

[0577] The server creates a detailed operation guide based on the temporary solution and distributes it to the user's device. The operation guide includes specific operating procedures and important points to note.

[0578] Specific operation: The server creates an operation guide based on the temporary solution procedure and sends it to the terminal in a displayable format. The temporary solution procedure is provided as input data, and the operation guide delivered to the user is obtained as output.

[0579] Step 7:

[0580] The user follows the operation guide displayed on the terminal to perform on-site operations, such as restarting a pump.

[0581] Specific operation: The user follows the procedures displayed on the terminal to operate the necessary equipment. The operation guide is provided as input data, and the results of the executed operations are obtained as output.

[0582] Step 8:

[0583] The server analyzes the feedback received from users and stores the responses and additional information in a database, which is used to further train the generative AI model.

[0584] Specific operation: The server analyzes the operation results and observation data received from the user and adds them to the database. Feedback information is provided as input data, and an updated learning database is obtained as output.

[0585] The above is an explanation of the specific operation of each processing step and its inputs and outputs. This system enables quick and highly accurate response when a fault occurs, improving the efficiency of equipment maintenance and repair.

[0586] (Application example 1)

[0587] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0588] When equipment failure occurs in a factory, a prompt and appropriate response is required. However, current systems can be slow to identify the cause of the failure and provide a solution, resulting in prolonged equipment downtime. In addition, because operators respond manually, there is a high risk of error. Therefore, there is a need for a system that can automatically identify the cause of the failure, take action, and provide feedback in real time.

[0589] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0590] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary responses and permanent measures based on the failure, means for creating and distributing an operation guide for on-site response, means for the factory robot to monitor the operating status in real time and detect abnormalities, means for identifying the cause of the failure using a generative AI model learned on the server, and means for providing an operation guide to on-site operators so that they can respond as instructed. This enables quick and accurate response when a failure occurs.

[0591] "Failure" refers to a condition in which equipment or systems are not functioning properly.

[0592] "Maintenance and preservation" refers to a set of activities that involve the upkeep and repair of equipment and systems.

[0593] A "system" refers to a single mechanism in which multiple elements function in relation to one another.

[0594] "Past failure cases" refers to specific cases and records of failures that have occurred in the past.

[0595] "Response methods" refers to the specific methods and procedures for dealing with problems when they occur.

[0596] "Configuration information" refers to information about each component of a facility or system.

[0597] An "instruction manual" refers to a document that describes how to use equipment or a system and precautions to take.

[0598] "Learning" refers to the process of using data to train a generative AI model and gain knowledge.

[0599] "Real-time data" refers to operational data that shows the current status of equipment or systems.

[0600] An "alarm" is a warning signal that notifies you that an abnormality or problem has occurred.

[0601] "Analysis" refers to the process of analyzing data and information in detail to clarify meaning and relationships.

[0602] "Identification" refers to clearly indicating the cause of a problem or issue.

[0603] "Recording" refers to storing information for future reference.

[0604] A "temporary solution" refers to a temporary solution to a problem that has occurred.

[0605] "Permanent fix" refers to a permanent solution that takes measures to prevent the same problem from recurring.

[0606] "Operation Guide" means instructions that provide specific operating procedures to be followed by local operators.

[0607] "Distribution" refers to the act of sending information or data to a specific device or user.

[0608] A "factory robot" refers to a robot that performs automated tasks in a factory.

[0609] "Operating status" refers to the current operation or state of a facility or system.

[0610] A "generative AI model" refers to an artificial intelligence model that learns from large amounts of data to perform specific tasks.

[0611] "On-site operator" refers to the person in charge of operating facilities and equipment at a factory or on-site.

[0612] The present invention relates to a system for automating the maintenance and upkeep of equipment in a factory production line and for responding quickly and accurately when a fault occurs. A detailed description of an embodiment of this system will be given below.

[0613] System Configuration

[0614] It consists of a server, factory robots, and terminals used by operators. The server has a generative AI model that has learned past failure cases, response methods, equipment configuration information, and instruction manuals.

[0615] Server Roles

[0616] The server performs the following series of processes.

[0617] 1. Learning from failure cases: The server acquires past failure cases, response methods, equipment configuration information, and instruction manuals, and uses these to train the generative AI model. Specifically, the generative AI model is trained using artificial intelligence libraries such as TensorFlow and PyTorch.

[0618] 2. Real-time data analysis: Receive real-time data and alarms sent from factory robots and analyze this data, including operational data such as temperature and pressure.

[0619] 3. Identifying the cause of the failure: Using a generative AI model, the cause of the failure is identified from the analysis results.

[0620] 4. Generate temporary and permanent solutions: Based on the identified causes, generate temporary and permanent solutions.

[0621] 5. Creation and distribution of operation guides: If on-site support is required, a detailed operation guide will be created and distributed to the device. This guide will include specific operating procedures and important points to note.

[0622] Device Role

[0623] The terminal displays notifications and operation guides received from the server. The user (operator) follows the instructions displayed on the terminal to take temporary measures such as restarting the equipment or replacing parts. For example, if a pump abnormality is detected, a restart instruction will be displayed on the terminal. The operator follows this instruction and restarts the pump. Once the restart is complete, the results are entered into the terminal and sent to the server. This information is added to the fault response database on the server side and will be used for future learning.

[0624] Specific examples

[0625] For example, the system operates as follows when a pump malfunctions on a factory production line.

[0626] 1. Receiving and analyzing real-time data (server): The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc.

[0627] 2. Generation of temporary solution (server): If the generative AI model determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[0628] 3. Distribution of operation guide (server and terminal): The server creates a detailed operation guide based on this temporary solution and distributes it to the terminal.

[0629] 4. On-site response (user): The user follows the instructions displayed on the terminal to restart the pump. The results are entered into the terminal and sent to the server.

[0630] 5. Feedback Analysis (Server): The server receives feedback from the user and analyzes the results. If the restart is successful, the information is added to the database for future learning.

[0631] Prompt example

[0632] For example, by inputting a prompt such as, "The robot's sensor has detected a high temperature (75°C). Please tell me what the possible problems are and how to deal with them," into a generative AI model, it is possible to quickly and accurately derive a specific response method.

[0633] The interaction of the above elements enables rapid and accurate response when a fault occurs, improving the efficiency and reliability of the factory.

[0634] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0635] Step 1:

[0636] The server retrieves past failure cases, response methods, equipment configuration information, and instruction manuals, and uses this data to train a generative AI model. Specifically, this information is read from the database and the AI ​​model is trained using TensorFlow and PyTorch. The model's input is data on failure cases and response methods, and its output is the identification of failures and recommended response methods.

[0637] Step 2:

[0638] Real-time data and alarms from factory robots are sent to a server. Such data includes temperature, pressure, operation status, etc. The server receives this data and parses it in JSON format. The input is the robot's sensor data, and the output is recorded as parsed data.

[0639] Step 3:

[0640] The server analyzes the received real-time data and detects anomalies. It uses a generative AI model to identify the cause of the fault based on the data. The input is sensor data and alarm information, and the output is the identified cause of the fault. The AI ​​model performs data analysis and fault identification with high accuracy.

[0641] Step 4:

[0642] The server generates temporary and permanent solutions based on the identified cause of the problem. First, it generates a temporary solution and issues a specific instruction, such as "restart the pump." The input is the identified cause of the problem, and the output is a proposal for temporary and permanent solutions.

[0643] Step 5:

[0644] The server creates a detailed operation guide based on the generated temporary solution and distributes it to the terminal. The guide includes operating procedures and important points to note. The input is the temporary solution data, and the output is the operation guide.

[0645] Step 6:

[0646] The user (operator) follows the operation guide received from the terminal and takes measures such as restarting equipment or replacing parts. For example, if an instruction to restart a pump is received, the user restarts the pump using the operation panel. The input is the terminal's operation guide, and the output is the result of the action taken.

[0647] Step 7:

[0648] Once the user has completed the action, they enter the results into their device and send them to the server. Specifically, it records whether the reboot was successful or if any other abnormalities occurred. The input is the data on the action results, and the output is sent to the server as feedback.

[0649] Step 8:

[0650] The server receives and analyzes user feedback. Successful solutions and other information are added to a database for future learning. The input is feedback data, and the output is an updated model and database. This allows the AI ​​model to continually improve, resulting in more accurate fault identification and response.

[0651] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0652] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system uses generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an on-site operation guide. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide response support that takes into account the user's psychological state.

[0653] Server-side roles

[0654] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It stores the received data in a local database. Next, it uses this data to train a generative AI model. It receives real-time data and alarms, analyzes the data, and identifies the failure. It generates a temporary solution based on the identified cause of the failure and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device. By combining it with an emotion engine, it analyzes the user's real-time emotional state and adjusts the response content and notification method.

[0655] The role of the terminal

[0656] The device displays notifications and operation guides received from the server to the user. It also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server, allowing the server to adjust its response methods and notification content according to the user's emotional state.

[0657] User Role

[0658] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. The system also provides psychological support as needed based on the user's emotional state analyzed by the emotion engine.

[0659] Specific examples

[0660] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[0661] 1. Receiving and analyzing real-time data (server)

[0662] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[0663] 2. Generate a temporary solution (server)

[0664] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[0665] 3. Distribution of operation guide (server and terminal)

[0666] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[0667] 4. On-site response (user)

[0668] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[0669] 5. Sentiment analysis and notification adjustment (device and server)

[0670] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[0671] 6. Feedback Analysis (Server)

[0672] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[0673] In this way, the system of the present invention enables a prompt and appropriate response when a failure occurs, improves the reliability and efficiency of the facility, and also reduces the psychological burden on the user.

[0674] The processing flow will be explained below.

[0675] Server-side processing steps

[0676] Step 1:

[0677] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals, and stores this data in a local database.

[0678] Step 2:

[0679] The server trains a generative AI model based on the stored data, using machine learning algorithms to learn patterns, causes, and solutions to problems.

[0680] Step 3:

[0681] The server receives real-time data and alarms sent from each facility, including sensor information and error logs.

[0682] Step 4:

[0683] The server analyzes real-time data and applies anomaly detection algorithms to determine whether a fault exists. If a fault is detected, the generative AI identifies the cause.

[0684] Step 5:

[0685] The server generates a temporary solution based on the identified cause of the failure, for example, a specific instruction to "restart the pump."

[0686] Step 6:

[0687] The server also generates permanent countermeasures to prevent future recurrence of the problem, such as "strengthen regular inspections of pumps."

[0688] Step 7:

[0689] If on-site support is required, the server will create a detailed operation guide, which includes reboot procedures and safety precautions.

[0690] Step 8:

[0691] The server distributes temporary solutions, permanent solutions, and operation guides to the terminal.

[0692] Terminal processing steps

[0693] Step 1:

[0694] The terminal displays the notifications and operation guides received from the server to the user.

[0695] Step 2:

[0696] The terminal analyzes the user's emotional state in real time via an emotion engine and transmits the information to the server.

[0697] User processing steps

[0698] Step 1:

[0699] The user follows the temporary solution displayed on the terminal and performs specific operations, such as "restart the pump."

[0700] Step 2:

[0701] Once the on-site troubleshooting is complete, the user enters the results into the terminal. Specific results such as "reboot successful / failed" are recorded.

[0702] Feedback processing steps

[0703] Step 1:

[0704] The terminal transmits the response result feedback from the user to the server.

[0705] Step 2:

[0706] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[0707] Emotion Engine Processing Steps

[0708] Step 1:

[0709] The device uses an emotion engine to analyze the user's emotional state, including facial expression recognition and voice analysis.

[0710] Step 2:

[0711] The device sends the analysis results to the server and requests that the server flexibly change the notification content and response methods according to the user's emotional state.

[0712] Step 3:

[0713] The server adjusts the content of notifications based on the user's emotional state. For example, if the user is feeling stressed, the server shortens the content of notifications and takes measures to reduce the user's psychological burden.

[0714] As a concrete example, the sequence of events that occurs when a pump malfunctions is shown below:

[0715] 1. Receiving and analyzing real-time data (server)

[0716] The server receives real-time data sent from the production line, analyzes the data, and detects any abnormalities in the pump.

[0717] 2. Generate a temporary solution (server)

[0718] The server generates a workaround: "Perform a pump restart."

[0719] 3. Distribution of operation guide (server and terminal)

[0720] The server creates a detailed operation guide and distributes it to the device, which includes instructions and important points to note.

[0721] 4. On-site response (user)

[0722] The user follows the instructions on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[0723] 5. Sentiment analysis and notification adjustment (device and server)

[0724] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server, which then changes the notification content to be more concise and reassuring if the user is feeling stressed.

[0725] 6. Feedback Analysis (Server)

[0726] The server receives feedback from the user, analyzes the results, and if successful, adds the information to the database for future learning.

[0727] Example 2

[0728] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0729] In the maintenance and upkeep of equipment, a fast and accurate response is required when a failure occurs. However, conventional systems have difficulty in responding in real time, which places a heavy psychological burden on users. Furthermore, it is often difficult for inexperienced users to select the appropriate response method. Furthermore, because the system is unable to respond in a way that takes into account the user's emotions and stress level, it can be difficult for users to understand the operation instructions.

[0730] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, a means for receiving and analyzing real-time data and alarms, and a means for analyzing user emotions and adjusting response methods and notification contents based on the analysis results. This enables quick and accurate failure analysis and flexible response according to the user.

[0731] "Past failure cases" refers to cases of abnormalities or failures in equipment or systems that have occurred in the past.

[0732] "Response methods" refer to the procedures and methods for dealing with failures or abnormalities when they occur.

[0733] "Facility configuration information" refers to detailed information such as the facility's blueprint, construction method, and component configuration.

[0734] An "instruction manual" refers to a document that describes how to operate equipment or systems, how to maintain them, and precautions to take.

[0735] "Real-time data" refers to data that instantly records the operating status of an ongoing process or machine.

[0736] An "alarm" refers to a warning signal or message that notifies personnel when an abnormality occurs in equipment or a system.

[0737] A "temporary solution" refers to a means of quickly and temporarily resolving a problem or abnormality when it occurs.

[0738] "Permanent measures" refer to measures or improvements that are taken on a permanent basis to prevent recurrence.

[0739] An "operation guide" refers to a detailed instruction manual that explains the operating procedures for equipment or systems.

[0740] "Means for analyzing emotions" refers to techniques and methods for analyzing a user's psychological state and emotions in real time.

[0741] "Means for adjusting the response method and notification content based on the analysis results" refers to a method for dynamically changing the optimal response method and notification content based on the results of the user's emotion analysis.

[0742] This invention relates to a system that automates the maintenance and upkeep of equipment and responds quickly and accurately when a fault occurs. This system deals with faults through cooperation between a server, a terminal, and a user.

[0743] Server processing

[0744] The server first obtains past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the client company, and stores them in a local database, allowing for centralized management of the data needed to troubleshoot.

[0745] This data is then used to train a generative AI model using machine learning libraries such as TensorFlow and PyTorch. Once the generative AI model is trained, the server receives real-time data and alarms, and analyzes this data to identify faults.

[0746] When a fault is identified, the server generates temporary and permanent solutions. The temporary solutions include specific operational procedures and deliver operation guides to the user. For example, if a pump abnormality is detected, the server generates a guide instructing the user to "restart the pump" and delivers it to the user's device.

[0747] Furthermore, the server is equipped with an emotion engine that can analyze the user's emotional state. Based on the results of this analysis, the notification content and response method can be flexibly adjusted. For example, if the user is feeling stressed, the tone and content of the notification can be changed to provide psychological support.

[0748] Processing by the terminal

[0749] The device displays notifications and operation guides received from the server to the user. For example, it displays detailed instructions and precautions for restarting the pump. The device also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server. The server's responses and notifications are then adjusted appropriately based on the user's emotional state.

[0750] User Action

[0751] The user performs operations according to the operation guide displayed on the terminal. For example, if an instruction to restart the pump is given, the user restarts the pump according to the instruction. Once the restart is complete, the result is entered into the terminal and sent to the server.

[0752] As a concrete example, consider a case where a pump malfunction occurs on a factory production line. The server receives real-time data sent from the production line, and the generative AI model identifies the malfunction. As a result, a solution to the problem, such as "restart the pump," is generated and sent to the user's device. The user follows the device's instructions to restart the pump. At that time, the device analyzes the user's emotional state and adjusts the notification content if the user is under high stress.

[0753] An example of a prompt sentence could be a specific question such as, "Please tell me what to do if a pump abnormality is detected." In this way, the system enables a quick and accurate response when a fault occurs, improving the reliability and efficiency of the equipment while also reducing the psychological burden on the user.

[0754] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0755] Step 1:

[0756] Receiving and storing data (server)

[0757] The server uses an API to receive information from the client company, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals. The inputs include various data files and data streams. The received data is sequentially saved in a local database. The data consistency is also checked, and any missing information is checked. The output is a structured database entry.

[0758] Step 2:

[0759] Generative AI model training (server)

[0760] The stored data is used to train a generative AI model. The server uses machine learning libraries such as TensorFlow and PyTorch to preprocess the data and train the model. A training dataset is required as input, and a trained AI model is obtained as output. Specific operations include loading data, preprocessing, model definition, compilation, and training.

[0761] Step 3:

[0762] Receiving and analyzing real-time data (server)

[0763] The server receives real-time sensor data sent from a production line or other device via an API. The input is a data stream that fluctuates over time. The received data is analyzed by a generative AI model to determine whether an anomaly exists. The output is a result regarding whether an anomaly exists or not. Specific operations include receiving the data, analyzing it using the model, and detecting an anomaly.

[0764] Step 4:

[0765] Creation and notification of temporary solutions (server)

[0766] When an abnormality is detected, the server identifies the cause and generates a temporary solution. The input is the analysis result, and the output is a notification message that includes a specific solution. Specific operations include generating a solution, determining the content of the notification, and sending it to the user terminal. The solution includes specific procedures such as restarting the system and checking the status.

[0767] Step 5:

[0768] Distribution of operation guide (server and terminal)

[0769] The server creates a detailed operation guide based on the generated temporary solution and delivers it to the user's terminal. The generated solution is required as input, and the output is an operation guide that is displayed to the user. Specific operations include creating the guide and delivering it to the terminal. The terminal receives it and displays it to the user.

[0770] Step 6:

[0771] On-site response (user)

[0772] The user performs operations according to the instructions displayed on the terminal. For example, if instructed to restart a pump, the user operates according to the procedure. The input is the operation guide, and the output is the result of the operation. Specific operations include actual operations on site and feedback of the results.

[0773] Step 7:

[0774] Sentiment analysis and notification adjustment (device and server)

[0775] The device uses an emotion engine to analyze the user's emotional state in real time and transmits that information to the server. The input is the user's biometric information and voice data, and the output is the emotion analysis results. The server adjusts the notification content and response methods based on these results. Specific operations include analyzing the emotion data, transmitting the analysis results to the server, and flexibly changing the notification content.

[0776] Step 8:

[0777] Receiving and analyzing feedback (server)

[0778] The server receives feedback from users and analyzes the corresponding results. The input is the user's feedback information, and the output is the analysis results and additional information added to the database. Specific operations include receiving feedback, analyzing it, saving it to the database, and using it as future model training data.

[0779] (Application example 2)

[0780] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0781] Conventional equipment maintenance and repair systems can sometimes have difficulty responding quickly and appropriately when a fault occurs. In particular, when workers are not familiar with dealing with faults, delays and errors can occur. Furthermore, when workers are in an unstable psychological state, it becomes difficult to respond efficiently, which can lead to reduced reliability and efficiency of the equipment. There is a need for a system that can solve these problems and carry out equipment maintenance and repair work more efficiently and accurately.

[0782] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0783] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary and permanent response methods based on the failure, means for creating and distributing an on-site operation guide, and means for analyzing the emotional state of users in real time and adjusting response methods and notification content based on that information. This enables prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the equipment.

[0784] "Past failure cases" refers to specific records of past failures that occurred in the equipment and the history of how they were dealt with.

[0785] "Response methods" refer to the specific actions and procedures to be taken when a failure occurs, and the related processes.

[0786] "Facility configuration information" refers to data relating to the physical and logical components of the facility in question.

[0787] "Instruction manual" refers to guidelines and manuals on how to use, maintain, and troubleshoot equipment.

[0788] "Real-time data" refers to data obtained from equipment regarding its current operating conditions and parameters.

[0789] An "alarm" refers to a warning signal issued when an abnormality or failure in equipment is detected.

[0790] "Emotional state" refers to a state that indicates a user's psychological state or emotional response.

[0791] "Analysis" refers to analyzing data and extracting meaningful patterns and information from it.

[0792] "Identifying the fault" refers to clarifying the cause and location of an abnormality or problem that has occurred.

[0793] "Temporary measures" refer to temporary measures to improve the situation in the short term when a problem occurs.

[0794] "Permanent measures" refer to long-term measures to eliminate the root cause to prevent recurrence and improve the reliability of the equipment.

[0795] An "operation guide" refers to an instruction manual that shows the steps and precautions a user should take when performing a specific operation.

[0796] "Notification content" refers to messages and instructions sent from the system to the user.

[0797] The present invention is a system for automating equipment maintenance and upkeep in a factory, and in particular, provides real-time troubleshooting using smart glasses and flexible notifications based on the emotional state of the user. An embodiment of this system will be described below.

[0798] Server-side roles

[0799] The server plays a central role in collecting and analyzing data within the factory. Specifically, it processes and calculates the data using the following hardware and software:

[0800] Hardware: High-performance server

[0801] Software: TensorFlow (for training generative AI models), database management system (DBMS)

[0802] The server first obtains past failure cases, response methods, equipment configuration information, and instruction manuals from the equipment and stores this data in a local database. It then uses this data to train a generative AI model. It receives real-time data and alarms from remote sensors and cameras, analyzes them, and identifies failures. Based on the identified failures, it generates temporary and permanent solutions and notifies the user. If on-site operation is required, it also creates a detailed operation guide and distributes it to the smart glasses.

[0803] The server also incorporates an emotion engine to analyze the user's emotional state, analyzing the user's real-time emotional state (camera footage and audio) and adjusting the response method and notification content based on that information.

[0804] The role of the terminal

[0805] Smart glasses are used as the device, and the device has the following functions:

[0806] Hardware: Smart glasses (camera, display, microphone)

[0807] Software: OpenCV (image analysis), Emotion API (emotion analysis)

[0808] The device displays notifications and operation guides from the server to the user. It also analyzes the user's emotional state in real time using data acquired from the camera and microphone and sends that information to the server, allowing the server to provide the optimal response based on the user's emotional state.

[0809] User Role

[0810] The user operates the device (smart glasses) by following the notifications and operation guide displayed on the device. Specific operation examples are shown below.

[0811] For example, if a pump malfunction occurs on a production line, the following measures will be taken.

[0812] 1. Receiving and analyzing real-time data (server)

[0813] The server receives real-time data showing the pump's operating status and analyzes it using a generative AI model to identify the cause of the fault.

[0814] 2. Generate a temporary solution (server)

[0815] If the server determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[0816] 3. Distribution of operation guide (server and terminal)

[0817] Based on this temporary solution, the server creates an operation guide including reboot procedures and precautions, and distributes it to the smart glasses.

[0818] 4. On-site response (user)

[0819] The user follows the instructions displayed on the smart glasses to restart the pump, and the results are entered into the smart glasses and sent to the server.

[0820] 5. Sentiment analysis and notification adjustment (device and server)

[0821] The smart glasses use a camera and microphone to analyze the user's emotional state (e.g., stress level) and send that information to a server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[0822] Here is an example of a prompt from a generative AI model:

[0823] "While the user is wearing the smart glasses, the AI ​​model should analyze images of machinery with bearing abnormalities and detect the bearing abnormality. Also, if the user's stress level is high, the smart glasses should display a message instructing the user to 'take a deep breath and calm down' and restart the device."

[0824] As a result, the system of the present invention enables a prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the facility.

[0825] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0826] Step 1:

[0827] The server collects data from within the factory, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals, and stores this information in a local database. This creates a database for subsequent failure analysis and countermeasure development.

[0828] Input: Past failure cases and instruction manuals obtained from the equipment

[0829] Output: Data stored in a local database

[0830] Step 2:

[0831] The server uses the collected data to train the generative AI model, which uses TensorFlow to analyze equipment configuration information and past failure cases to learn patterns when failures occur.

[0832] Input: Data stored in a local database

[0833] Output: Trained generative AI model

[0834] Step 3:

[0835] The server receives real-time data and alarms from remote sensors and cameras in the factory, including equipment operating status, pressure, temperature, and more.

[0836] Input: Real-time data from remote sensors and cameras

[0837] Output: Real-time monitoring data

[0838] Step 4:

[0839] The server analyzes the received real-time data and identifies faults using a generative AI model, which uses the received data as input to determine whether a fault has occurred and its cause.

[0840] Input: Real-time monitoring data

[0841] Output: Whether there is a fault and its cause

[0842] Step 5:

[0843] The server generates temporary solutions based on the identified faults (for example, if a pump needs to be restarted, it generates a restart procedure), and also suggests the necessary repairs or part replacements for permanent solutions.

[0844] Input: Identified fault information

[0845] Output: Temporary and permanent solutions

[0846] Step 6:

[0847] The server creates a detailed operation guide based on the generated temporary and permanent solutions and distributes it to the device (smart glasses).

[0848] Input: Temporary and permanent solutions

[0849] Output: Operation guide to terminal

[0850] Step 7:

[0851] The device (smart glasses) displays the operation guide received from the server to the user, who then follows the guide to perform the operation.

[0852] Input: Operation guide from the server

[0853] Output: Guide display for users

[0854] Step 8:

[0855] The device uses a camera and microphone to analyze the user's emotional state (e.g., stress level) in real time and transmits the information to a server, where the analysis is performed using an emotion engine.

[0856] Input: Data from the camera and microphone

[0857] Output: User's emotional state data

[0858] Step 9:

[0859] The server flexibly adjusts the notification content and response methods based on the user's emotional state data. For example, if the user is feeling stressed, the server may adjust the notification content to be gentler or provide instructions for additional breaks.

[0860] Input: User's emotional state data

[0861] Output: Adjusted notification content and action to be taken

[0862] Step 10:

[0863] The user follows the instructions displayed on the device (smart glasses) to take local action. For example, they restart the pump, enter the results into the device, and send them to the server.

[0864] Input: the instructions displayed on the terminal

[0865] Output: Send the restart result to the server

[0866] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0867] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0868] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0869] [Third embodiment]

[0870] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0871] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0872] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0873] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0874] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0875] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0876] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0877] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0878] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0879] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0880] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0881] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0882] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an operation guide for on-site response.

[0883] Server-side roles

[0884] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It then uses this data to train the generative AI model. When real-time data and alarms are received, the data is analyzed and the failure is identified. At this time, the generative AI model can be used to identify the cause of the failure quickly and with high accuracy. Based on the identified cause, it generates a temporary solution and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device.

[0885] The role of the terminal

[0886] The terminal displays notifications and operation guides received from the server to the user. The user follows the instructions displayed on the terminal to take temporary measures such as restarting equipment or replacing parts. After completing the measures, the results are fed back to the server. This feedback information is also used for further model training.

[0887] User Role

[0888] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. This information is added to the server's fault response database and used for future learning.

[0889] Specific examples

[0890] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[0891] 1. Receiving and analyzing real-time data (server)

[0892] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[0893] 2. Generate a temporary solution (server)

[0894] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[0895] 3. Distribution of operation guide (server and terminal)

[0896] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[0897] 4. On-site response (user)

[0898] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[0899] 5. Feedback Analysis (Server)

[0900] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[0901] In this way, the system of the present invention enables a quick and appropriate response when a fault occurs, thereby improving the reliability and efficiency of the facility.

[0902] The processing flow will be explained below.

[0903] Server-side processing steps

[0904] Step 1:

[0905] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the proposed company, and stores the received data in a local database.

[0906] Step 2:

[0907] The server trains a generative AI model based on the stored data, using machine learning algorithms to detect patterns, causes, and solutions for failures.

[0908] Step 3:

[0909] The server receives real-time data and alarm signals sent from each facility, including the facility's operating status and abnormality detection information.

[0910] Step 4:

[0911] The server analyzes real-time data and uses an anomaly detection algorithm to determine whether there is a problem. If an anomaly is detected, the generating AI identifies the cause.

[0912] Step 5:

[0913] The server generates a temporary solution based on the identified cause of the failure, for example, a specific procedure such as "restart the pump."

[0914] Step 6:

[0915] Along with providing instructions on how to deal with the problem, the server also generates a permanent solution, which is intended to prevent the problem from recurring in the future.

[0916] Step 7:

[0917] If on-site assistance is required, the server will create a detailed operation guide, including reboot procedures and safety precautions.

[0918] Step 8:

[0919] The server distributes the generated temporary solution, permanent solution plan, and operation guide to the terminal.

[0920] Terminal processing steps

[0921] Step 1:

[0922] The terminal displays the notifications and operation guides received from the server to the user.

[0923] Step 2:

[0924] The terminal provides the user with temporary and permanent solutions in real time and displays instructions on how to deal with the problem.

[0925] User processing steps

[0926] Step 1:

[0927] The user follows the temporary solution displayed on the terminal and performs a specific operation, such as restarting the pump.

[0928] Step 2:

[0929] Once the user has completed their on-site visit, they enter the results into the terminal, recording success or failure and any additional observations.

[0930] Feedback processing steps

[0931] Step 1:

[0932] The terminal transmits the response result feedback from the user to the server.

[0933] Step 2:

[0934] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[0935] In this way, the server, terminal, and user can cooperate to proceed with the troubleshooting process, thereby improving the reliability and efficiency of the facility.

[0936] Example 1

[0937] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0938] When a fault occurs during equipment maintenance and upkeep, a quick and accurate response is required, but it was difficult to achieve this efficiently with conventional systems. Furthermore, identifying the cause of the fault and formulating a countermeasure required human resources and specialized knowledge, which resulted in a time-consuming response and adversely affected the equipment's operating rate. This created a need to improve the reliability and efficiency of the equipment.

[0939] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0940] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and warning signals, means for quickly and accurately identifying failures from the received data using a generative AI model, means for generating temporary and permanent solutions based on the failure, means for generating and distributing operating procedures for on-site response to terminals, and means for receiving and analyzing feedback from users. This enables quick and accurate response when a failure occurs, improves the equipment availability rate, and makes it possible to streamline maintenance and repair work.

[0941] "Past failure cases" are records of problems or failures that have previously occurred in the equipment.

[0942] "Response methods" refer to the specific measures and repair procedures taken when equipment failure occurs.

[0943] "Facility configuration information" refers to information such as detailed specifications, layout, and connection status of each part of the facility.

[0944] An "instruction manual" is a document that contains instructions on how to operate and maintain equipment.

[0945] "Real-time data" refers to data collected in real time regarding the operation status and performance of equipment.

[0946] A "warning signal" is a notification issued when an abnormal value or unexpected behavior is detected in equipment.

[0947] A "generative AI model" is an artificial intelligence that learns using large amounts of data and makes highly accurate predictions and judgments for specific tasks.

[0948] "Failure" refers to a problem or malfunction that prevents the equipment from functioning properly.

[0949] A "temporary solution" is a specific procedure for temporarily resolving a problem when a failure occurs.

[0950] A "permanent fix" is a measure or repair that will be effective over the long term to prevent the problem from recurring.

[0951] "Operating procedures" are descriptions of specific steps and precautions that users should follow when operating equipment.

[0952] A "terminal" is a device, such as a tablet or smartphone, through which a user receives operating instructions and notifications.

[0953] "Feedback" refers to the act of sending the results of operations performed by a user or information observed by a user to a server.

[0954] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes a generative AI model to identify the cause of the fault, propose temporary and permanent solutions, and provide an operational guide for on-site response. The program for this system is described in detail below.

[0955] The server collects past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it proposes solutions. This collected data is used to train a generative AI model (e.g., GPT-4) using data analysis tools such as TensorFlow and PyTorch.

[0956] After training, the server receives real-time data and warning signals from the equipment. This real-time data includes information such as the operating status, pressure, and temperature of pumps and motors. The received data is immediately analyzed and input into the generative AI model, allowing the AI ​​model to identify faults quickly and with high accuracy.

[0957] Based on the identified cause of the problem, the server generates temporary and permanent solutions. Temporary solutions are specific steps to quickly resolve the problem, such as "restart the pump." Permanent solutions are long-term solutions to prevent recurrence.

[0958] Next, the server generates an operation guide detailing the on-site operation procedures and distributes it to the user's device. The operation guide includes specific operation procedures, necessary precautions, and safety measures. The device displays the notifications and operation guide received from the server to the user.

[0959] The user performs on-site operations by following the operation guide displayed on the terminal. For example, if a pump malfunction is discovered, the user follows the instructions displayed on the terminal to restart the pump. Once the restart is complete, the results are entered into the terminal and sent to the server.

[0960] The server analyzes the feedback received from users, stores the response results and additional information in a database, and uses them to further train the generative AI model to improve the accuracy of future incident responses.

[0961] Specific examples

[0962] For example, if a pump malfunctions on a factory production line, the system will operate as follows:

[0963] 1. Data collection and learning

[0964] The server collects data on past pump failure cases and how to deal with them, and trains the AI ​​model.

[0965] 2. Receiving and analyzing real-time data

[0966] The server receives real-time operational status data from the production line and analyzes it using a generative AI model.

[0967] 3. Generation of temporary solutions

[0968] Based on the analysis results of the AI ​​model, the server generates a solution such as "Perform the pump restart procedure."

[0969] 4. Distribution of operation guide

[0970] The server distributes an operation guide detailing the reboot procedure to the terminal and instructs the user.

[0971] 5. Implementation of local response

[0972] The user restarts the pump by following the restart procedure displayed on the terminal.

[0973] 6. Feedback Analysis

[0974] The server receives feedback data, including whether the reboot was successful, and feeds this back into the AI ​​model to improve its accuracy in the future.

[0975] Prompt Sentence Examples

[0976] "Pump pressure has exceeded the specified value. Please identify the cause."

[0977] "Please provide pump restart procedure."

[0978] "Please detail the pump restart procedure."

[0979] "The user has performed a pump restart. What are the next steps?"

[0980] "The pump restart was successful. Is there anything else I should do or be careful of?"

[0981] The above is a specific description of the embodiment of the present invention. This method enables a quick and accurate response when a fault occurs, and achieves efficient maintenance and upkeep of the equipment.

[0982] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0983] Step 1:

[0984] The server collects past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals. This data is used to train the generative AI model.

[0985] How it works: The server connects to the company's database and automatically downloads the necessary data. The input data includes a history of incidents and their resolutions, which are organized by category and saved in a folder for model training. The output is an organized dataset.

[0986] Step 2:

[0987] The server trains a generative AI model based on the collected data, for example, by using data analysis tools such as TensorFlow or PyTorch to train the model (e.g., GPT-4).

[0988] How it works: The server inputs historical data into the AI ​​model and runs a training process to learn trends and patterns in the data. Collected failure cases and solutions are provided as input data, and a trained generative AI model is obtained as output.

[0989] Step 3:

[0990] The server receives real-time data and warning signals sent from the equipment, including the equipment's operating status, pressure, temperature, etc.

[0991] How it works: The server continuously collects real-time data from sensors and stores it in a database. During this process, data is updated either hourly or by events. Real-time measurements are provided as input data, and an updated database is obtained as output.

[0992] Step 4:

[0993] The server analyzes the received real-time data and uses generative AI models to quickly identify faults.

[0994] How it works: The server inputs real-time data into the AI ​​model and performs analysis to detect abnormal patterns and signs of failure. Real-time sensor data is provided as input data, and detailed information about identified failures is obtained as output.

[0995] Step 5:

[0996] The server generates a temporary solution based on the identified cause of the problem, which includes specific steps to quickly resolve the problem.

[0997] Specific operation: The server documents temporary solutions based on the fault handling scenarios generated by the AI ​​model. Identified fault information is provided as input data, and the documented temporary solution procedures are obtained as output.

[0998] Step 6:

[0999] The server creates a detailed operation guide based on the temporary solution and distributes it to the user's device. The operation guide includes specific operating procedures and important points to note.

[1000] Specific operation: The server creates an operation guide based on the temporary solution procedure and sends it to the terminal in a displayable format. The temporary solution procedure is provided as input data, and the operation guide delivered to the user is obtained as output.

[1001] Step 7:

[1002] The user follows the operation guide displayed on the terminal to perform on-site operations, such as restarting a pump.

[1003] Specific operation: The user follows the procedures displayed on the terminal to operate the necessary equipment. The operation guide is provided as input data, and the results of the executed operations are obtained as output.

[1004] Step 8:

[1005] The server analyzes the feedback received from users and stores the responses and additional information in a database, which is used to further train the generative AI model.

[1006] Specific operation: The server analyzes the operation results and observation data received from the user and adds them to the database. Feedback information is provided as input data, and an updated learning database is obtained as output.

[1007] The above is an explanation of the specific operation of each processing step and its inputs and outputs. This system enables quick and highly accurate response when a fault occurs, improving the efficiency of equipment maintenance and repair.

[1008] (Application example 1)

[1009] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] When equipment failure occurs in a factory, a prompt and appropriate response is required. However, current systems can be slow to identify the cause of the failure and provide a solution, resulting in prolonged equipment downtime. In addition, because operators respond manually, there is a high risk of error. Therefore, there is a need for a system that can automatically identify the cause of the failure, take action, and provide feedback in real time.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1012] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary responses and permanent measures based on the failure, means for creating and distributing an operation guide for on-site response, means for the factory robot to monitor the operating status in real time and detect abnormalities, means for identifying the cause of the failure using a generative AI model learned on the server, and means for providing an operation guide to on-site operators so that they can respond as instructed. This enables quick and accurate response when a failure occurs.

[1013] "Failure" refers to a condition in which equipment or systems are not functioning properly.

[1014] "Maintenance and preservation" refers to a set of activities that involve the upkeep and repair of equipment and systems.

[1015] A "system" refers to a single mechanism in which multiple elements function in relation to one another.

[1016] "Past failure cases" refers to specific cases and records of failures that have occurred in the past.

[1017] "Response methods" refers to the specific methods and procedures for dealing with problems when they occur.

[1018] "Configuration information" refers to information about each component of a facility or system.

[1019] An "instruction manual" refers to a document that describes how to use equipment or a system and precautions to take.

[1020] "Learning" refers to the process of using data to train a generative AI model and gain knowledge.

[1021] "Real-time data" refers to operational data that shows the current status of equipment or systems.

[1022] An "alarm" is a warning signal that notifies you that an abnormality or problem has occurred.

[1023] "Analysis" refers to the process of analyzing data and information in detail to clarify meaning and relationships.

[1024] "Identification" refers to clearly indicating the cause of a problem or issue.

[1025] "Recording" refers to storing information for future reference.

[1026] A "temporary solution" refers to a temporary solution to a problem that has occurred.

[1027] "Permanent fix" refers to a permanent solution that takes measures to prevent the same problem from recurring.

[1028] "Operation Guide" means instructions that provide specific operating procedures to be followed by local operators.

[1029] "Distribution" refers to the act of sending information or data to a specific device or user.

[1030] A "factory robot" refers to a robot that performs automated tasks in a factory.

[1031] "Operating status" refers to the current operation or state of a facility or system.

[1032] A "generative AI model" refers to an artificial intelligence model that learns from large amounts of data to perform specific tasks.

[1033] "On-site operator" refers to the person in charge of operating facilities and equipment at a factory or on-site.

[1034] The present invention relates to a system for automating the maintenance and upkeep of equipment in a factory production line and for responding quickly and accurately when a fault occurs. A detailed description of an embodiment of this system will be given below.

[1035] System Configuration

[1036] It consists of a server, factory robots, and terminals used by operators. The server has a generative AI model that has learned past failure cases, response methods, equipment configuration information, and instruction manuals.

[1037] Server Roles

[1038] The server performs the following series of processes.

[1039] 1. Learning from failure cases: The server acquires past failure cases, response methods, equipment configuration information, and instruction manuals, and uses these to train the generative AI model. Specifically, the generative AI model is trained using artificial intelligence libraries such as TensorFlow and PyTorch.

[1040] 2. Real-time data analysis: Receive real-time data and alarms sent from factory robots and analyze this data, including operational data such as temperature and pressure.

[1041] 3. Identifying the cause of the failure: Using a generative AI model, the cause of the failure is identified from the analysis results.

[1042] 4. Generate temporary and permanent solutions: Based on the identified causes, generate temporary and permanent solutions.

[1043] 5. Creation and distribution of operation guides: If on-site support is required, a detailed operation guide will be created and distributed to the device. This guide will include specific operating procedures and important points to note.

[1044] Device Role

[1045] The terminal displays notifications and operation guides received from the server. The user (operator) follows the instructions displayed on the terminal to take temporary measures such as restarting the equipment or replacing parts. For example, if a pump abnormality is detected, a restart instruction will be displayed on the terminal. The operator follows this instruction and restarts the pump. Once the restart is complete, the results are entered into the terminal and sent to the server. This information is added to the fault response database on the server side and will be used for future learning.

[1046] Specific examples

[1047] For example, the system operates as follows when a pump malfunctions on a factory production line.

[1048] 1. Receiving and analyzing real-time data (server): The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc.

[1049] 2. Generation of temporary solution (server): If the generative AI model determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[1050] 3. Distribution of operation guide (server and terminal): The server creates a detailed operation guide based on this temporary solution and distributes it to the terminal.

[1051] 4. On-site response (user): The user follows the instructions displayed on the terminal to restart the pump. The results are entered into the terminal and sent to the server.

[1052] 5. Feedback Analysis (Server): The server receives feedback from the user and analyzes the results. If the restart is successful, the information is added to the database for future learning.

[1053] Prompt example

[1054] For example, by inputting a prompt such as, "The robot's sensor has detected a high temperature (75°C). Please tell me what the possible problems are and how to deal with them," into a generative AI model, it is possible to quickly and accurately derive a specific response method.

[1055] The interaction of the above elements enables rapid and accurate response when a fault occurs, improving the efficiency and reliability of the factory.

[1056] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1057] Step 1:

[1058] The server retrieves past failure cases, response methods, equipment configuration information, and instruction manuals, and uses this data to train a generative AI model. Specifically, this information is read from the database and the AI ​​model is trained using TensorFlow and PyTorch. The model's input is data on failure cases and response methods, and its output is the identification of failures and recommended response methods.

[1059] Step 2:

[1060] Real-time data and alarms from factory robots are sent to a server. Such data includes temperature, pressure, operation status, etc. The server receives this data and parses it in JSON format. The input is the robot's sensor data, and the output is recorded as parsed data.

[1061] Step 3:

[1062] The server analyzes the received real-time data and detects anomalies. It uses a generative AI model to identify the cause of the fault based on the data. The input is sensor data and alarm information, and the output is the identified cause of the fault. The AI ​​model performs data analysis and fault identification with high accuracy.

[1063] Step 4:

[1064] The server generates temporary and permanent solutions based on the identified cause of the problem. First, it generates a temporary solution and issues a specific instruction, such as "restart the pump." The input is the identified cause of the problem, and the output is a proposal for temporary and permanent solutions.

[1065] Step 5:

[1066] The server creates a detailed operation guide based on the generated temporary solution and distributes it to the terminal. The guide includes operating procedures and important points to note. The input is the temporary solution data, and the output is the operation guide.

[1067] Step 6:

[1068] The user (operator) follows the operation guide received from the terminal and takes measures such as restarting equipment or replacing parts. For example, if an instruction to restart a pump is received, the user restarts the pump using the operation panel. The input is the terminal's operation guide, and the output is the result of the action taken.

[1069] Step 7:

[1070] Once the user has completed the action, they enter the results into their device and send them to the server. Specifically, it records whether the reboot was successful or if any other abnormalities occurred. The input is the data on the action results, and the output is sent to the server as feedback.

[1071] Step 8:

[1072] The server receives and analyzes user feedback. Successful solutions and other information are added to a database for future learning. The input is feedback data, and the output is an updated model and database. This allows the AI ​​model to continually improve, resulting in more accurate fault identification and response.

[1073] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1074] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system uses generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an on-site operation guide. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide response support that takes into account the user's psychological state.

[1075] Server-side roles

[1076] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It stores the received data in a local database. Next, it uses this data to train a generative AI model. It receives real-time data and alarms, analyzes the data, and identifies the failure. It generates a temporary solution based on the identified cause of the failure and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device. By combining it with an emotion engine, it analyzes the user's real-time emotional state and adjusts the response content and notification method.

[1077] The role of the terminal

[1078] The device displays notifications and operation guides received from the server to the user. It also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server, allowing the server to adjust its response methods and notification content according to the user's emotional state.

[1079] User Role

[1080] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. The system also provides psychological support as needed based on the user's emotional state analyzed by the emotion engine.

[1081] Specific examples

[1082] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[1083] 1. Receiving and analyzing real-time data (server)

[1084] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[1085] 2. Generate a temporary solution (server)

[1086] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[1087] 3. Distribution of operation guide (server and terminal)

[1088] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[1089] 4. On-site response (user)

[1090] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[1091] 5. Sentiment analysis and notification adjustment (device and server)

[1092] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[1093] 6. Feedback Analysis (Server)

[1094] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[1095] In this way, the system of the present invention enables a prompt and appropriate response when a failure occurs, improves the reliability and efficiency of the facility, and also reduces the psychological burden on the user.

[1096] The processing flow will be explained below.

[1097] Server-side processing steps

[1098] Step 1:

[1099] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals, and stores this data in a local database.

[1100] Step 2:

[1101] The server trains a generative AI model based on the stored data, using machine learning algorithms to learn patterns, causes, and solutions to problems.

[1102] Step 3:

[1103] The server receives real-time data and alarms sent from each facility, including sensor information and error logs.

[1104] Step 4:

[1105] The server analyzes real-time data and applies anomaly detection algorithms to determine whether a fault exists. If a fault is detected, the generative AI identifies the cause.

[1106] Step 5:

[1107] The server generates a temporary solution based on the identified cause of the failure, for example, a specific instruction to "restart the pump."

[1108] Step 6:

[1109] The server also generates permanent countermeasures to prevent future recurrence of the problem, such as "strengthen regular inspections of pumps."

[1110] Step 7:

[1111] If on-site support is required, the server will create a detailed operation guide, which includes reboot procedures and safety precautions.

[1112] Step 8:

[1113] The server distributes temporary solutions, permanent solutions, and operation guides to the terminal.

[1114] Terminal processing steps

[1115] Step 1:

[1116] The terminal displays the notifications and operation guides received from the server to the user.

[1117] Step 2:

[1118] The terminal analyzes the user's emotional state in real time via an emotion engine and transmits the information to the server.

[1119] User processing steps

[1120] Step 1:

[1121] The user follows the temporary solution displayed on the terminal and performs specific operations, such as "restart the pump."

[1122] Step 2:

[1123] Once the on-site troubleshooting is complete, the user enters the results into the terminal. Specific results such as "reboot successful / failed" are recorded.

[1124] Feedback processing steps

[1125] Step 1:

[1126] The terminal transmits the response result feedback from the user to the server.

[1127] Step 2:

[1128] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[1129] Emotion Engine Processing Steps

[1130] Step 1:

[1131] The device uses an emotion engine to analyze the user's emotional state, including facial expression recognition and voice analysis.

[1132] Step 2:

[1133] The device sends the analysis results to the server and requests that the server flexibly change the notification content and response methods according to the user's emotional state.

[1134] Step 3:

[1135] The server adjusts the content of notifications based on the user's emotional state. For example, if the user is feeling stressed, the server shortens the content of notifications and takes measures to reduce the user's psychological burden.

[1136] As a concrete example, the sequence of events that occurs when a pump malfunctions is shown below:

[1137] 1. Receiving and analyzing real-time data (server)

[1138] The server receives real-time data sent from the production line, analyzes the data, and detects any abnormalities in the pump.

[1139] 2. Generate a temporary solution (server)

[1140] The server generates a workaround: "Perform a pump restart."

[1141] 3. Distribution of operation guide (server and terminal)

[1142] The server creates a detailed operation guide and distributes it to the device, which includes instructions and important points to note.

[1143] 4. On-site response (user)

[1144] The user follows the instructions on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[1145] 5. Sentiment analysis and notification adjustment (device and server)

[1146] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server, which then changes the notification content to be more concise and reassuring if the user is feeling stressed.

[1147] 6. Feedback Analysis (Server)

[1148] The server receives feedback from the user, analyzes the results, and if successful, adds the information to the database for future learning.

[1149] Example 2

[1150] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1151] In the maintenance and upkeep of equipment, a fast and accurate response is required when a failure occurs. However, conventional systems have difficulty in responding in real time, which places a heavy psychological burden on users. Furthermore, it is often difficult for inexperienced users to select the appropriate response method. Furthermore, because the system is unable to respond in a way that takes into account the user's emotions and stress level, it can be difficult for users to understand the operation instructions.

[1152] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, a means for receiving and analyzing real-time data and alarms, and a means for analyzing user emotions and adjusting response methods and notification contents based on the analysis results. This enables quick and accurate failure analysis and flexible response according to the user.

[1153] "Past failure cases" refers to cases of abnormalities or failures in equipment or systems that have occurred in the past.

[1154] "Response methods" refer to the procedures and methods for dealing with failures or abnormalities when they occur.

[1155] "Facility configuration information" refers to detailed information such as the facility's blueprint, construction method, and component configuration.

[1156] An "instruction manual" refers to a document that describes how to operate equipment or systems, how to maintain them, and precautions to take.

[1157] "Real-time data" refers to data that instantly records the operating status of an ongoing process or machine.

[1158] An "alarm" refers to a warning signal or message that notifies personnel when an abnormality occurs in equipment or a system.

[1159] A "temporary solution" refers to a means of quickly and temporarily resolving a problem or abnormality when it occurs.

[1160] "Permanent measures" refer to measures or improvements that are taken on a permanent basis to prevent recurrence.

[1161] An "operation guide" refers to a detailed instruction manual that explains the operating procedures for equipment or systems.

[1162] "Means for analyzing emotions" refers to techniques and methods for analyzing a user's psychological state and emotions in real time.

[1163] "Means for adjusting the response method and notification content based on the analysis results" refers to a method for dynamically changing the optimal response method and notification content based on the results of the user's emotion analysis.

[1164] This invention relates to a system that automates the maintenance and upkeep of equipment and responds quickly and accurately when a fault occurs. This system deals with faults through cooperation between a server, a terminal, and a user.

[1165] Server processing

[1166] The server first obtains past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the client company, and stores them in a local database, allowing for centralized management of the data needed to troubleshoot.

[1167] This data is then used to train a generative AI model using machine learning libraries such as TensorFlow and PyTorch. Once the generative AI model is trained, the server receives real-time data and alarms, and analyzes this data to identify faults.

[1168] When a fault is identified, the server generates temporary and permanent solutions. The temporary solutions include specific operational procedures and deliver operation guides to the user. For example, if a pump abnormality is detected, the server generates a guide instructing the user to "restart the pump" and delivers it to the user's device.

[1169] Furthermore, the server is equipped with an emotion engine that can analyze the user's emotional state. Based on the results of this analysis, the notification content and response method can be flexibly adjusted. For example, if the user is feeling stressed, the tone and content of the notification can be changed to provide psychological support.

[1170] Processing by the terminal

[1171] The device displays notifications and operation guides received from the server to the user. For example, it displays detailed instructions and precautions for restarting the pump. The device also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server. The server's responses and notifications are then adjusted appropriately based on the user's emotional state.

[1172] User Action

[1173] The user performs operations according to the operation guide displayed on the terminal. For example, if an instruction to restart the pump is given, the user restarts the pump according to the instruction. Once the restart is complete, the result is entered into the terminal and sent to the server.

[1174] As a concrete example, consider a case where a pump malfunction occurs on a factory production line. The server receives real-time data sent from the production line, and the generative AI model identifies the malfunction. As a result, a solution to the problem, such as "restart the pump," is generated and sent to the user's device. The user follows the device's instructions to restart the pump. At that time, the device analyzes the user's emotional state and adjusts the notification content if the user is under high stress.

[1175] An example of a prompt sentence could be a specific question such as, "Please tell me what to do if a pump abnormality is detected." In this way, the system enables a quick and accurate response when a fault occurs, improving the reliability and efficiency of the equipment while also reducing the psychological burden on the user.

[1176] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1177] Step 1:

[1178] Receiving and storing data (server)

[1179] The server uses an API to receive information from the client company, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals. The inputs include various data files and data streams. The received data is sequentially saved in a local database. The data consistency is also checked, and any missing information is checked. The output is a structured database entry.

[1180] Step 2:

[1181] Generative AI model training (server)

[1182] The stored data is used to train a generative AI model. The server uses machine learning libraries such as TensorFlow and PyTorch to preprocess the data and train the model. A training dataset is required as input, and a trained AI model is obtained as output. Specific operations include loading data, preprocessing, model definition, compilation, and training.

[1183] Step 3:

[1184] Receiving and analyzing real-time data (server)

[1185] The server receives real-time sensor data sent from a production line or other device via an API. The input is a data stream that fluctuates over time. The received data is analyzed by a generative AI model to determine whether an anomaly exists. The output is a result regarding whether an anomaly exists or not. Specific operations include receiving the data, analyzing it using the model, and detecting an anomaly.

[1186] Step 4:

[1187] Creation and notification of temporary solutions (server)

[1188] When an abnormality is detected, the server identifies the cause and generates a temporary solution. The input is the analysis result, and the output is a notification message that includes a specific solution. Specific operations include generating a solution, determining the content of the notification, and sending it to the user terminal. The solution includes specific procedures such as restarting the system and checking the status.

[1189] Step 5:

[1190] Distribution of operation guide (server and terminal)

[1191] The server creates a detailed operation guide based on the generated temporary solution and delivers it to the user's terminal. The generated solution is required as input, and the output is an operation guide that is displayed to the user. Specific operations include creating the guide and delivering it to the terminal. The terminal receives it and displays it to the user.

[1192] Step 6:

[1193] On-site response (user)

[1194] The user performs operations according to the instructions displayed on the terminal. For example, if instructed to restart a pump, the user operates according to the procedure. The input is the operation guide, and the output is the result of the operation. Specific operations include actual operations on site and feedback of the results.

[1195] Step 7:

[1196] Sentiment analysis and notification adjustment (device and server)

[1197] The device uses an emotion engine to analyze the user's emotional state in real time and transmits that information to the server. The input is the user's biometric information and voice data, and the output is the emotion analysis results. The server adjusts the notification content and response methods based on these results. Specific operations include analyzing the emotion data, transmitting the analysis results to the server, and flexibly changing the notification content.

[1198] Step 8:

[1199] Receiving and analyzing feedback (server)

[1200] The server receives feedback from users and analyzes the corresponding results. The input is the user's feedback information, and the output is the analysis results and additional information added to the database. Specific operations include receiving feedback, analyzing it, saving it to the database, and using it as future model training data.

[1201] (Application example 2)

[1202] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1203] Conventional equipment maintenance and repair systems can sometimes have difficulty responding quickly and appropriately when a fault occurs. In particular, when workers are not familiar with dealing with faults, delays and errors can occur. Furthermore, when workers are in an unstable psychological state, it becomes difficult to respond efficiently, which can lead to reduced reliability and efficiency of the equipment. There is a need for a system that can solve these problems and carry out equipment maintenance and repair work more efficiently and accurately.

[1204] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1205] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary and permanent response methods based on the failure, means for creating and distributing an on-site operation guide, and means for analyzing the emotional state of users in real time and adjusting response methods and notification content based on that information. This enables prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the equipment.

[1206] "Past failure cases" refers to specific records of past failures that occurred in the equipment and the history of how they were dealt with.

[1207] "Response methods" refer to the specific actions and procedures to be taken when a failure occurs, and the related processes.

[1208] "Facility configuration information" refers to data relating to the physical and logical components of the facility in question.

[1209] "Instruction manual" refers to guidelines and manuals on how to use, maintain, and troubleshoot equipment.

[1210] "Real-time data" refers to data obtained from equipment regarding its current operating conditions and parameters.

[1211] An "alarm" refers to a warning signal issued when an abnormality or failure in equipment is detected.

[1212] "Emotional state" refers to a state that indicates a user's psychological state or emotional response.

[1213] "Analysis" refers to analyzing data and extracting meaningful patterns and information from it.

[1214] "Identifying the fault" refers to clarifying the cause and location of an abnormality or problem that has occurred.

[1215] "Temporary measures" refer to temporary measures to improve the situation in the short term when a problem occurs.

[1216] "Permanent measures" refer to long-term measures to eliminate the root cause to prevent recurrence and improve the reliability of the equipment.

[1217] An "operation guide" refers to an instruction manual that shows the steps and precautions a user should take when performing a specific operation.

[1218] "Notification content" refers to messages and instructions sent from the system to the user.

[1219] The present invention is a system for automating equipment maintenance and upkeep in a factory, and in particular, provides real-time troubleshooting using smart glasses and flexible notifications based on the emotional state of the user. An embodiment of this system will be described below.

[1220] Server-side roles

[1221] The server plays a central role in collecting and analyzing data within the factory. Specifically, it processes and calculates the data using the following hardware and software:

[1222] Hardware: High-performance server

[1223] Software: TensorFlow (for training generative AI models), database management system (DBMS)

[1224] The server first obtains past failure cases, response methods, equipment configuration information, and instruction manuals from the equipment and stores this data in a local database. It then uses this data to train a generative AI model. It receives real-time data and alarms from remote sensors and cameras, analyzes them, and identifies failures. Based on the identified failures, it generates temporary and permanent solutions and notifies the user. If on-site operation is required, it also creates a detailed operation guide and distributes it to the smart glasses.

[1225] The server also incorporates an emotion engine to analyze the user's emotional state, analyzing the user's real-time emotional state (camera footage and audio) and adjusting the response method and notification content based on that information.

[1226] The role of the terminal

[1227] Smart glasses are used as the device, and the device has the following functions:

[1228] Hardware: Smart glasses (camera, display, microphone)

[1229] Software: OpenCV (image analysis), Emotion API (emotion analysis)

[1230] The device displays notifications and operation guides from the server to the user. It also analyzes the user's emotional state in real time using data acquired from the camera and microphone and sends that information to the server, allowing the server to provide the optimal response based on the user's emotional state.

[1231] User Role

[1232] The user operates the device (smart glasses) by following the notifications and operation guide displayed on the device. Specific operation examples are shown below.

[1233] For example, if a pump malfunction occurs on a production line, the following measures will be taken.

[1234] 1. Receiving and analyzing real-time data (server)

[1235] The server receives real-time data showing the pump's operating status and analyzes it using a generative AI model to identify the cause of the fault.

[1236] 2. Generate a temporary solution (server)

[1237] If the server determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[1238] 3. Distribution of operation guide (server and terminal)

[1239] Based on this temporary solution, the server creates an operation guide including reboot procedures and precautions, and distributes it to the smart glasses.

[1240] 4. On-site response (user)

[1241] The user follows the instructions displayed on the smart glasses to restart the pump, and the results are entered into the smart glasses and sent to the server.

[1242] 5. Sentiment analysis and notification adjustment (device and server)

[1243] The smart glasses use a camera and microphone to analyze the user's emotional state (e.g., stress level) and send that information to a server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[1244] Here is an example of a prompt from a generative AI model:

[1245] "While the user is wearing the smart glasses, the AI ​​model should analyze images of machinery with bearing abnormalities and detect the bearing abnormality. Also, if the user's stress level is high, the smart glasses should display a message instructing the user to 'take a deep breath and calm down' and restart the device."

[1246] As a result, the system of the present invention enables a prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the facility.

[1247] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1248] Step 1:

[1249] The server collects data from within the factory, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals, and stores this information in a local database. This creates a database for subsequent failure analysis and countermeasure development.

[1250] Input: Past failure cases and instruction manuals obtained from the equipment

[1251] Output: Data stored in a local database

[1252] Step 2:

[1253] The server uses the collected data to train the generative AI model, which uses TensorFlow to analyze equipment configuration information and past failure cases to learn patterns when failures occur.

[1254] Input: Data stored in a local database

[1255] Output: Trained generative AI model

[1256] Step 3:

[1257] The server receives real-time data and alarms from remote sensors and cameras in the factory, including equipment operating status, pressure, temperature, and more.

[1258] Input: Real-time data from remote sensors and cameras

[1259] Output: Real-time monitoring data

[1260] Step 4:

[1261] The server analyzes the received real-time data and identifies faults using a generative AI model, which uses the received data as input to determine whether a fault has occurred and its cause.

[1262] Input: Real-time monitoring data

[1263] Output: Whether there is a fault and its cause

[1264] Step 5:

[1265] The server generates temporary solutions based on the identified faults (for example, if a pump needs to be restarted, it generates a restart procedure), and also suggests the necessary repairs or part replacements for permanent solutions.

[1266] Input: Identified fault information

[1267] Output: Temporary and permanent solutions

[1268] Step 6:

[1269] The server creates a detailed operation guide based on the generated temporary and permanent solutions and distributes it to the device (smart glasses).

[1270] Input: Temporary and permanent solutions

[1271] Output: Operation guide to terminal

[1272] Step 7:

[1273] The device (smart glasses) displays the operation guide received from the server to the user, who then follows the guide to perform the operation.

[1274] Input: Operation guide from the server

[1275] Output: Guide display for users

[1276] Step 8:

[1277] The device uses a camera and microphone to analyze the user's emotional state (e.g., stress level) in real time and transmits the information to a server, where the analysis is performed using an emotion engine.

[1278] Input: Data from the camera and microphone

[1279] Output: User's emotional state data

[1280] Step 9:

[1281] The server flexibly adjusts the notification content and response methods based on the user's emotional state data. For example, if the user is feeling stressed, the server may adjust the notification content to be gentler or provide instructions for additional breaks.

[1282] Input: User's emotional state data

[1283] Output: Adjusted notification content and action to be taken

[1284] Step 10:

[1285] The user follows the instructions displayed on the device (smart glasses) to take local action. For example, they restart the pump, enter the results into the device, and send them to the server.

[1286] Input: the instructions displayed on the terminal

[1287] Output: Send the restart result to the server

[1288] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1289] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1290] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1291] [Fourth embodiment]

[1292] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1293] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1294] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1295] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1296] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1297] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1298] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1299] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1300] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1301] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1302] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1303] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1304] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1305] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an operation guide for on-site response.

[1306] Server-side roles

[1307] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It then uses this data to train the generative AI model. When real-time data and alarms are received, the data is analyzed and the failure is identified. At this time, the generative AI model can be used to identify the cause of the failure quickly and with high accuracy. Based on the identified cause, it generates a temporary solution and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device.

[1308] The role of the terminal

[1309] The terminal displays notifications and operation guides received from the server to the user. The user follows the instructions displayed on the terminal to take temporary measures such as restarting equipment or replacing parts. After completing the measures, the results are fed back to the server. This feedback information is also used for further model training.

[1310] User Role

[1311] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. This information is added to the server's fault response database and used for future learning.

[1312] Specific examples

[1313] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[1314] 1. Receiving and analyzing real-time data (server)

[1315] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[1316] 2. Generate a temporary solution (server)

[1317] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[1318] 3. Distribution of operation guide (server and terminal)

[1319] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[1320] 4. On-site response (user)

[1321] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[1322] 5. Feedback Analysis (Server)

[1323] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[1324] In this way, the system of the present invention enables a quick and appropriate response when a fault occurs, thereby improving the reliability and efficiency of the facility.

[1325] The processing flow will be explained below.

[1326] Server-side processing steps

[1327] Step 1:

[1328] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the proposed company, and stores the received data in a local database.

[1329] Step 2:

[1330] The server trains a generative AI model based on the stored data, using machine learning algorithms to detect patterns, causes, and solutions for failures.

[1331] Step 3:

[1332] The server receives real-time data and alarm signals sent from each facility, including the facility's operating status and abnormality detection information.

[1333] Step 4:

[1334] The server analyzes real-time data and uses an anomaly detection algorithm to determine whether there is a problem. If an anomaly is detected, the generating AI identifies the cause.

[1335] Step 5:

[1336] The server generates a temporary solution based on the identified cause of the failure, for example, a specific procedure such as "restart the pump."

[1337] Step 6:

[1338] Along with providing instructions on how to deal with the problem, the server also generates a permanent solution, which is intended to prevent the problem from recurring in the future.

[1339] Step 7:

[1340] If on-site assistance is required, the server will create a detailed operation guide, including reboot procedures and safety precautions.

[1341] Step 8:

[1342] The server distributes the generated temporary solution, permanent solution plan, and operation guide to the terminal.

[1343] Terminal processing steps

[1344] Step 1:

[1345] The terminal displays the notifications and operation guides received from the server to the user.

[1346] Step 2:

[1347] The terminal provides the user with temporary and permanent solutions in real time and displays instructions on how to deal with the problem.

[1348] User processing steps

[1349] Step 1:

[1350] The user follows the temporary solution displayed on the terminal and performs a specific operation, such as restarting the pump.

[1351] Step 2:

[1352] Once the user has completed their on-site visit, they enter the results into the terminal, recording success or failure and any additional observations.

[1353] Feedback processing steps

[1354] Step 1:

[1355] The terminal transmits the response result feedback from the user to the server.

[1356] Step 2:

[1357] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[1358] In this way, the server, terminal, and user can cooperate to proceed with the troubleshooting process, thereby improving the reliability and efficiency of the facility.

[1359] Example 1

[1360] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1361] When a fault occurs during equipment maintenance and upkeep, a quick and accurate response is required, but it was difficult to achieve this efficiently with conventional systems. Furthermore, identifying the cause of the fault and formulating a countermeasure required human resources and specialized knowledge, which resulted in a time-consuming response and adversely affected the equipment's operating rate. This created a need to improve the reliability and efficiency of the equipment.

[1362] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1363] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and warning signals, means for quickly and accurately identifying failures from the received data using a generative AI model, means for generating temporary and permanent solutions based on the failure, means for generating and distributing operating procedures for on-site response to terminals, and means for receiving and analyzing feedback from users. This enables quick and accurate response when a failure occurs, improves the equipment availability rate, and makes it possible to streamline maintenance and repair work.

[1364] "Past failure cases" are records of problems or failures that have previously occurred in the equipment.

[1365] "Response methods" refer to the specific measures and repair procedures taken when equipment failure occurs.

[1366] "Facility configuration information" refers to information such as detailed specifications, layout, and connection status of each part of the facility.

[1367] An "instruction manual" is a document that contains instructions on how to operate and maintain equipment.

[1368] "Real-time data" refers to data collected in real time regarding the operation status and performance of equipment.

[1369] A "warning signal" is a notification issued when an abnormal value or unexpected behavior is detected in equipment.

[1370] A "generative AI model" is an artificial intelligence that learns using large amounts of data and makes highly accurate predictions and judgments for specific tasks.

[1371] "Failure" refers to a problem or malfunction that prevents the equipment from functioning properly.

[1372] A "temporary solution" is a specific procedure for temporarily resolving a problem when a failure occurs.

[1373] A "permanent fix" is a measure or repair that will be effective over the long term to prevent the problem from recurring.

[1374] "Operating procedures" are descriptions of specific steps and precautions that users should follow when operating equipment.

[1375] A "terminal" is a device, such as a tablet or smartphone, through which a user receives operating instructions and notifications.

[1376] "Feedback" refers to the act of sending the results of operations performed by a user or information observed by a user to a server.

[1377] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system utilizes a generative AI model to identify the cause of the fault, propose temporary and permanent solutions, and provide an operational guide for on-site response. The program for this system is described in detail below.

[1378] The server collects past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it proposes solutions. This collected data is used to train a generative AI model (e.g., GPT-4) using data analysis tools such as TensorFlow and PyTorch.

[1379] After training, the server receives real-time data and warning signals from the equipment. This real-time data includes information such as the operating status, pressure, and temperature of pumps and motors. The received data is immediately analyzed and input into the generative AI model, allowing the AI ​​model to identify faults quickly and with high accuracy.

[1380] Based on the identified cause of the problem, the server generates temporary and permanent solutions. Temporary solutions are specific steps to quickly resolve the problem, such as "restart the pump." Permanent solutions are long-term solutions to prevent recurrence.

[1381] Next, the server generates an operation guide detailing the on-site operation procedures and distributes it to the user's device. The operation guide includes specific operation procedures, necessary precautions, and safety measures. The device displays the notifications and operation guide received from the server to the user.

[1382] The user performs on-site operations by following the operation guide displayed on the terminal. For example, if a pump malfunction is discovered, the user follows the instructions displayed on the terminal to restart the pump. Once the restart is complete, the results are entered into the terminal and sent to the server.

[1383] The server analyzes the feedback received from users, stores the response results and additional information in a database, and uses them to further train the generative AI model to improve the accuracy of future incident responses.

[1384] Specific examples

[1385] For example, if a pump malfunctions on a factory production line, the system will operate as follows:

[1386] 1. Data collection and learning

[1387] The server collects data on past pump failure cases and how to deal with them, and trains the AI ​​model.

[1388] 2. Receiving and analyzing real-time data

[1389] The server receives real-time operational status data from the production line and analyzes it using a generative AI model.

[1390] 3. Generation of temporary solutions

[1391] Based on the analysis results of the AI ​​model, the server generates a solution such as "Perform the pump restart procedure."

[1392] 4. Distribution of operation guide

[1393] The server distributes an operation guide detailing the reboot procedure to the terminal and instructs the user.

[1394] 5. Implementation of local response

[1395] The user restarts the pump by following the restart procedure displayed on the terminal.

[1396] 6. Feedback Analysis

[1397] The server receives feedback data, including whether the reboot was successful, and feeds this back into the AI ​​model to improve its accuracy in the future.

[1398] Prompt Sentence Examples

[1399] "Pump pressure has exceeded the specified value. Please identify the cause."

[1400] "Please provide pump restart procedure."

[1401] "Please detail the pump restart procedure."

[1402] "The user has performed a pump restart. What are the next steps?"

[1403] "The pump restart was successful. Is there anything else I should do or be careful of?"

[1404] The above is a specific description of the embodiment of the present invention. This method enables a quick and accurate response when a fault occurs, and achieves efficient maintenance and upkeep of the equipment.

[1405] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1406] Step 1:

[1407] The server collects past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals. This data is used to train the generative AI model.

[1408] How it works: The server connects to the company's database and automatically downloads the necessary data. The input data includes a history of incidents and their resolutions, which are organized by category and saved in a folder for model training. The output is an organized dataset.

[1409] Step 2:

[1410] The server trains a generative AI model based on the collected data, for example, by using data analysis tools such as TensorFlow or PyTorch to train the model (e.g., GPT-4).

[1411] How it works: The server inputs historical data into the AI ​​model and runs a training process to learn trends and patterns in the data. Collected failure cases and solutions are provided as input data, and a trained generative AI model is obtained as output.

[1412] Step 3:

[1413] The server receives real-time data and warning signals sent from the equipment, including the equipment's operating status, pressure, temperature, etc.

[1414] How it works: The server continuously collects real-time data from sensors and stores it in a database. During this process, data is updated either hourly or by events. Real-time measurements are provided as input data, and an updated database is obtained as output.

[1415] Step 4:

[1416] The server analyzes the received real-time data and uses generative AI models to quickly identify faults.

[1417] How it works: The server inputs real-time data into the AI ​​model and performs analysis to detect abnormal patterns and signs of failure. Real-time sensor data is provided as input data, and detailed information about identified failures is obtained as output.

[1418] Step 5:

[1419] The server generates a temporary solution based on the identified cause of the problem, which includes specific steps to quickly resolve the problem.

[1420] Specific operation: The server documents temporary solutions based on the fault handling scenarios generated by the AI ​​model. Identified fault information is provided as input data, and the documented temporary solution procedures are obtained as output.

[1421] Step 6:

[1422] The server creates a detailed operation guide based on the temporary solution and distributes it to the user's device. The operation guide includes specific operating procedures and important points to note.

[1423] Specific operation: The server creates an operation guide based on the temporary solution procedure and sends it to the terminal in a displayable format. The temporary solution procedure is provided as input data, and the operation guide delivered to the user is obtained as output.

[1424] Step 7:

[1425] The user follows the operation guide displayed on the terminal to perform on-site operations, such as restarting a pump.

[1426] Specific operation: The user follows the procedures displayed on the terminal to operate the necessary equipment. The operation guide is provided as input data, and the results of the executed operations are obtained as output.

[1427] Step 8:

[1428] The server analyzes the feedback received from users and stores the responses and additional information in a database, which is used to further train the generative AI model.

[1429] Specific operation: The server analyzes the operation results and observation data received from the user and adds them to the database. Feedback information is provided as input data, and an updated learning database is obtained as output.

[1430] The above is an explanation of the specific operation of each processing step and its inputs and outputs. This system enables quick and highly accurate response when a fault occurs, improving the efficiency of equipment maintenance and repair.

[1431] (Application example 1)

[1432] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1433] When equipment failure occurs in a factory, a prompt and appropriate response is required. However, current systems can be slow to identify the cause of the failure and provide a solution, resulting in prolonged equipment downtime. In addition, because operators respond manually, there is a high risk of error. Therefore, there is a need for a system that can automatically identify the cause of the failure, take action, and provide feedback in real time.

[1434] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1435] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary responses and permanent measures based on the failure, means for creating and distributing an operation guide for on-site response, means for the factory robot to monitor the operating status in real time and detect abnormalities, means for identifying the cause of the failure using a generative AI model learned on the server, and means for providing an operation guide to on-site operators so that they can respond as instructed. This enables quick and accurate response when a failure occurs.

[1436] "Failure" refers to a condition in which equipment or systems are not functioning properly.

[1437] "Maintenance and preservation" refers to a set of activities that involve the upkeep and repair of equipment and systems.

[1438] A "system" refers to a single mechanism in which multiple elements function in relation to one another.

[1439] "Past failure cases" refers to specific cases and records of failures that have occurred in the past.

[1440] "Response methods" refers to the specific methods and procedures for dealing with problems when they occur.

[1441] "Configuration information" refers to information about each component of a facility or system.

[1442] An "instruction manual" refers to a document that describes how to use equipment or a system and precautions to take.

[1443] "Learning" refers to the process of using data to train a generative AI model and gain knowledge.

[1444] "Real-time data" refers to operational data that shows the current status of equipment or systems.

[1445] An "alarm" is a warning signal that notifies you that an abnormality or problem has occurred.

[1446] "Analysis" refers to the process of analyzing data and information in detail to clarify meaning and relationships.

[1447] "Identification" refers to clearly indicating the cause of a problem or issue.

[1448] "Recording" refers to storing information for future reference.

[1449] A "temporary solution" refers to a temporary solution to a problem that has occurred.

[1450] "Permanent fix" refers to a permanent solution that takes measures to prevent the same problem from recurring.

[1451] "Operation Guide" means instructions that provide specific operating procedures to be followed by local operators.

[1452] "Distribution" refers to the act of sending information or data to a specific device or user.

[1453] A "factory robot" refers to a robot that performs automated tasks in a factory.

[1454] "Operating status" refers to the current operation or state of a facility or system.

[1455] A "generative AI model" refers to an artificial intelligence model that learns from large amounts of data to perform specific tasks.

[1456] "On-site operator" refers to the person in charge of operating facilities and equipment at a factory or on-site.

[1457] The present invention relates to a system for automating the maintenance and upkeep of equipment in a factory production line and for responding quickly and accurately when a fault occurs. A detailed description of an embodiment of this system will be given below.

[1458] System Configuration

[1459] It consists of a server, factory robots, and terminals used by operators. The server has a generative AI model that has learned past failure cases, response methods, equipment configuration information, and instruction manuals.

[1460] Server Roles

[1461] The server performs the following series of processes.

[1462] 1. Learning from failure cases: The server acquires past failure cases, response methods, equipment configuration information, and instruction manuals, and uses these to train the generative AI model. Specifically, the generative AI model is trained using artificial intelligence libraries such as TensorFlow and PyTorch.

[1463] 2. Real-time data analysis: Receive real-time data and alarms sent from factory robots and analyze this data, including operational data such as temperature and pressure.

[1464] 3. Identifying the cause of the failure: Using a generative AI model, the cause of the failure is identified from the analysis results.

[1465] 4. Generate temporary and permanent solutions: Based on the identified causes, generate temporary and permanent solutions.

[1466] 5. Creation and distribution of operation guides: If on-site support is required, a detailed operation guide will be created and distributed to the device. This guide will include specific operating procedures and important points to note.

[1467] Device Role

[1468] The terminal displays notifications and operation guides received from the server. The user (operator) follows the instructions displayed on the terminal to take temporary measures such as restarting the equipment or replacing parts. For example, if a pump abnormality is detected, a restart instruction will be displayed on the terminal. The operator follows this instruction and restarts the pump. Once the restart is complete, the results are entered into the terminal and sent to the server. This information is added to the fault response database on the server side and will be used for future learning.

[1469] Specific examples

[1470] For example, the system operates as follows when a pump malfunctions on a factory production line.

[1471] 1. Receiving and analyzing real-time data (server): The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc.

[1472] 2. Generation of temporary solution (server): If the generative AI model determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[1473] 3. Distribution of operation guide (server and terminal): The server creates a detailed operation guide based on this temporary solution and distributes it to the terminal.

[1474] 4. On-site response (user): The user follows the instructions displayed on the terminal to restart the pump. The results are entered into the terminal and sent to the server.

[1475] 5. Feedback Analysis (Server): The server receives feedback from the user and analyzes the results. If the restart is successful, the information is added to the database for future learning.

[1476] Prompt example

[1477] For example, by inputting a prompt such as, "The robot's sensor has detected a high temperature (75°C). Please tell me what the possible problems are and how to deal with them," into a generative AI model, it is possible to quickly and accurately derive a specific response method.

[1478] The interaction of the above elements enables rapid and accurate response when a fault occurs, improving the efficiency and reliability of the factory.

[1479] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1480] Step 1:

[1481] The server retrieves past failure cases, response methods, equipment configuration information, and instruction manuals, and uses this data to train a generative AI model. Specifically, this information is read from the database and the AI ​​model is trained using TensorFlow and PyTorch. The model's input is data on failure cases and response methods, and its output is the identification of failures and recommended response methods.

[1482] Step 2:

[1483] Real-time data and alarms from factory robots are sent to a server. Such data includes temperature, pressure, operation status, etc. The server receives this data and parses it in JSON format. The input is the robot's sensor data, and the output is recorded as parsed data.

[1484] Step 3:

[1485] The server analyzes the received real-time data and detects anomalies. It uses a generative AI model to identify the cause of the fault based on the data. The input is sensor data and alarm information, and the output is the identified cause of the fault. The AI ​​model performs data analysis and fault identification with high accuracy.

[1486] Step 4:

[1487] The server generates temporary and permanent solutions based on the identified cause of the problem. First, it generates a temporary solution and issues a specific instruction, such as "restart the pump." The input is the identified cause of the problem, and the output is a proposal for temporary and permanent solutions.

[1488] Step 5:

[1489] The server creates a detailed operation guide based on the generated temporary solution and distributes it to the terminal. The guide includes operating procedures and important points to note. The input is the temporary solution data, and the output is the operation guide.

[1490] Step 6:

[1491] The user (operator) follows the operation guide received from the terminal and takes measures such as restarting equipment or replacing parts. For example, if an instruction to restart a pump is received, the user restarts the pump using the operation panel. The input is the terminal's operation guide, and the output is the result of the action taken.

[1492] Step 7:

[1493] Once the user has completed the action, they enter the results into their device and send them to the server. Specifically, it records whether the reboot was successful or if any other abnormalities occurred. The input is the data on the action results, and the output is sent to the server as feedback.

[1494] Step 8:

[1495] The server receives and analyzes user feedback. Successful solutions and other information are added to a database for future learning. The input is feedback data, and the output is an updated model and database. This allows the AI ​​model to continually improve, resulting in more accurate fault identification and response.

[1496] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1497] This invention relates to a system that automates equipment maintenance and upkeep, and responds quickly and accurately when a fault occurs. This system uses generative AI to identify the cause of the fault, propose temporary and permanent solutions, and provide an on-site operation guide. Furthermore, by combining it with an emotion engine that recognizes the user's emotions, it is possible to provide response support that takes into account the user's psychological state.

[1498] Server-side roles

[1499] The server performs the following series of processes. First, it receives past failure cases, response methods, equipment configuration information, and instruction manuals from the proposed company. It stores the received data in a local database. Next, it uses this data to train a generative AI model. It receives real-time data and alarms, analyzes the data, and identifies the failure. It generates a temporary solution based on the identified cause of the failure and notifies the user. It also generates and recommends a permanent solution. If on-site operation is required, it creates an operation guide and distributes it to the user's device. By combining it with an emotion engine, it analyzes the user's real-time emotional state and adjusts the response content and notification method.

[1500] The role of the terminal

[1501] The device displays notifications and operation guides received from the server to the user. It also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server, allowing the server to adjust its response methods and notification content according to the user's emotional state.

[1502] User Role

[1503] The user operates the device by following the notifications and operation guides displayed on the device. For example, if a pump malfunction is detected, the device will display instructions to restart the pump. The user follows these instructions and restarts the pump. Once the restart is complete, the results are entered into the device and sent to the server. The system also provides psychological support as needed based on the user's emotional state analyzed by the emotion engine.

[1504] Specific examples

[1505] For example, if a pump malfunction occurs on a factory production line, the response system will operate as follows:

[1506] 1. Receiving and analyzing real-time data (server)

[1507] The server receives real-time data sent from the production line, including pump operation status, pressure, temperature, etc. When an abnormality is detected, the generative AI model analyzes it and identifies the cause of the failure.

[1508] 2. Generate a temporary solution (server)

[1509] If the server determines that the pump abnormality can be resolved by restarting the pump, it generates a temporary solution of "restart the pump."

[1510] 3. Distribution of operation guide (server and terminal)

[1511] The server creates a detailed operation guide based on this temporary solution and distributes it to the user's device. The guide includes restart procedures and important points to note.

[1512] 4. On-site response (user)

[1513] The user follows the instructions displayed on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[1514] 5. Sentiment analysis and notification adjustment (device and server)

[1515] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[1516] 6. Feedback Analysis (Server)

[1517] The server receives feedback from the user, analyzes the results, and if the restart is successful, adds the information to a database for future learning.

[1518] In this way, the system of the present invention enables a prompt and appropriate response when a failure occurs, improves the reliability and efficiency of the facility, and also reduces the psychological burden on the user.

[1519] The processing flow will be explained below.

[1520] Server-side processing steps

[1521] Step 1:

[1522] The server receives past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the companies to which it makes proposals, and stores this data in a local database.

[1523] Step 2:

[1524] The server trains a generative AI model based on the stored data, using machine learning algorithms to learn patterns, causes, and solutions to problems.

[1525] Step 3:

[1526] The server receives real-time data and alarms sent from each facility, including sensor information and error logs.

[1527] Step 4:

[1528] The server analyzes real-time data and applies anomaly detection algorithms to determine whether a fault exists. If a fault is detected, the generative AI identifies the cause.

[1529] Step 5:

[1530] The server generates a temporary solution based on the identified cause of the failure, for example, a specific instruction to "restart the pump."

[1531] Step 6:

[1532] The server also generates permanent countermeasures to prevent future recurrence of the problem, such as "strengthen regular inspections of pumps."

[1533] Step 7:

[1534] If on-site support is required, the server will create a detailed operation guide, which includes reboot procedures and safety precautions.

[1535] Step 8:

[1536] The server distributes temporary solutions, permanent solutions, and operation guides to the terminal.

[1537] Terminal processing steps

[1538] Step 1:

[1539] The terminal displays the notifications and operation guides received from the server to the user.

[1540] Step 2:

[1541] The terminal analyzes the user's emotional state in real time via an emotion engine and transmits the information to the server.

[1542] User processing steps

[1543] Step 1:

[1544] The user follows the temporary solution displayed on the terminal and performs specific operations, such as "restart the pump."

[1545] Step 2:

[1546] Once the on-site troubleshooting is complete, the user enters the results into the terminal. Specific results such as "reboot successful / failed" are recorded.

[1547] Feedback processing steps

[1548] Step 1:

[1549] The terminal transmits the response result feedback from the user to the server.

[1550] Step 2:

[1551] The server analyzes the received feedback and updates the fault response database, adding new data points to be used for the next learning.

[1552] Emotion Engine Processing Steps

[1553] Step 1:

[1554] The device uses an emotion engine to analyze the user's emotional state, including facial expression recognition and voice analysis.

[1555] Step 2:

[1556] The device sends the analysis results to the server and requests that the server flexibly change the notification content and response methods according to the user's emotional state.

[1557] Step 3:

[1558] The server adjusts the content of notifications based on the user's emotional state. For example, if the user is feeling stressed, the server shortens the content of notifications and takes measures to reduce the user's psychological burden.

[1559] As a concrete example, the sequence of events that occurs when a pump malfunctions is shown below:

[1560] 1. Receiving and analyzing real-time data (server)

[1561] The server receives real-time data sent from the production line, analyzes the data, and detects any abnormalities in the pump.

[1562] 2. Generate a temporary solution (server)

[1563] The server generates a workaround: "Perform a pump restart."

[1564] 3. Distribution of operation guide (server and terminal)

[1565] The server creates a detailed operation guide and distributes it to the device, which includes instructions and important points to note.

[1566] 4. On-site response (user)

[1567] The user follows the instructions on the terminal to restart the pump, and the results are entered into the terminal and sent to the server.

[1568] 5. Sentiment analysis and notification adjustment (device and server)

[1569] The device uses an emotion engine to analyze the user's emotional state and sends that information to the server, which then changes the notification content to be more concise and reassuring if the user is feeling stressed.

[1570] 6. Feedback Analysis (Server)

[1571] The server receives feedback from the user, analyzes the results, and if successful, adds the information to the database for future learning.

[1572] Example 2

[1573] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1574] In the maintenance and upkeep of equipment, a fast and accurate response is required when a failure occurs. However, conventional systems have difficulty in responding in real time, which places a heavy psychological burden on users. Furthermore, it is often difficult for inexperienced users to select the appropriate response method. Furthermore, because the system is unable to respond in a way that takes into account the user's emotions and stress level, it can be difficult for users to understand the operation instructions.

[1575] The identification process by the identification processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means. In this invention, the server includes a means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, a means for receiving and analyzing real-time data and alarms, and a means for analyzing user emotions and adjusting response methods and notification contents based on the analysis results. This enables quick and accurate failure analysis and flexible response according to the user.

[1576] "Past failure cases" refers to cases of abnormalities or failures in equipment or systems that have occurred in the past.

[1577] "Response methods" refer to the procedures and methods for dealing with failures or abnormalities when they occur.

[1578] "Facility configuration information" refers to detailed information such as the facility's blueprint, construction method, and component configuration.

[1579] An "instruction manual" refers to a document that describes how to operate equipment or systems, how to maintain them, and precautions to take.

[1580] "Real-time data" refers to data that instantly records the operating status of an ongoing process or machine.

[1581] An "alarm" refers to a warning signal or message that notifies personnel when an abnormality occurs in equipment or a system.

[1582] A "temporary solution" refers to a means of quickly and temporarily resolving a problem or abnormality when it occurs.

[1583] "Permanent measures" refer to measures or improvements that are taken on a permanent basis to prevent recurrence.

[1584] An "operation guide" refers to a detailed instruction manual that explains the operating procedures for equipment or systems.

[1585] "Means for analyzing emotions" refers to techniques and methods for analyzing a user's psychological state and emotions in real time.

[1586] "Means for adjusting the response method and notification content based on the analysis results" refers to a method for dynamically changing the optimal response method and notification content based on the results of the user's emotion analysis.

[1587] This invention relates to a system that automates the maintenance and upkeep of equipment and responds quickly and accurately when a fault occurs. This system deals with faults through cooperation between a server, a terminal, and a user.

[1588] Server processing

[1589] The server first obtains past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals from the client company, and stores them in a local database, allowing for centralized management of the data needed to troubleshoot.

[1590] This data is then used to train a generative AI model using machine learning libraries such as TensorFlow and PyTorch. Once the generative AI model is trained, the server receives real-time data and alarms, and analyzes this data to identify faults.

[1591] When a fault is identified, the server generates temporary and permanent solutions. The temporary solutions include specific operational procedures and deliver operation guides to the user. For example, if a pump abnormality is detected, the server generates a guide instructing the user to "restart the pump" and delivers it to the user's device.

[1592] Furthermore, the server is equipped with an emotion engine that can analyze the user's emotional state. Based on the results of this analysis, the notification content and response method can be flexibly adjusted. For example, if the user is feeling stressed, the tone and content of the notification can be changed to provide psychological support.

[1593] Processing by the terminal

[1594] The device displays notifications and operation guides received from the server to the user. For example, it displays detailed instructions and precautions for restarting the pump. The device also analyzes the user's emotional state in real time via an emotion engine and sends that information to the server. The server's responses and notifications are then adjusted appropriately based on the user's emotional state.

[1595] User Action

[1596] The user performs operations according to the operation guide displayed on the terminal. For example, if an instruction to restart the pump is given, the user restarts the pump according to the instruction. Once the restart is complete, the result is entered into the terminal and sent to the server.

[1597] As a concrete example, consider a case where a pump malfunction occurs on a factory production line. The server receives real-time data sent from the production line, and the generative AI model identifies the malfunction. As a result, a solution to the problem, such as "restart the pump," is generated and sent to the user's device. The user follows the device's instructions to restart the pump. At that time, the device analyzes the user's emotional state and adjusts the notification content if the user is under high stress.

[1598] An example of a prompt sentence could be a specific question such as, "Please tell me what to do if a pump abnormality is detected." In this way, the system enables a quick and accurate response when a fault occurs, improving the reliability and efficiency of the equipment while also reducing the psychological burden on the user.

[1599] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1600] Step 1:

[1601] Receiving and storing data (server)

[1602] The server uses an API to receive information from the client company, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals. The inputs include various data files and data streams. The received data is sequentially saved in a local database. The data consistency is also checked, and any missing information is checked. The output is a structured database entry.

[1603] Step 2:

[1604] Generative AI model training (server)

[1605] The stored data is used to train a generative AI model. The server uses machine learning libraries such as TensorFlow and PyTorch to preprocess the data and train the model. A training dataset is required as input, and a trained AI model is obtained as output. Specific operations include loading data, preprocessing, model definition, compilation, and training.

[1606] Step 3:

[1607] Receiving and analyzing real-time data (server)

[1608] The server receives real-time sensor data sent from a production line or other device via an API. The input is a data stream that fluctuates over time. The received data is analyzed by a generative AI model to determine whether an anomaly exists. The output is a result regarding whether an anomaly exists or not. Specific operations include receiving the data, analyzing it using the model, and detecting an anomaly.

[1609] Step 4:

[1610] Creation and notification of temporary solutions (server)

[1611] When an abnormality is detected, the server identifies the cause and generates a temporary solution. The input is the analysis result, and the output is a notification message that includes a specific solution. Specific operations include generating a solution, determining the content of the notification, and sending it to the user terminal. The solution includes specific procedures such as restarting the system and checking the status.

[1612] Step 5:

[1613] Distribution of operation guide (server and terminal)

[1614] The server creates a detailed operation guide based on the generated temporary solution and delivers it to the user's terminal. The generated solution is required as input, and the output is an operation guide that is displayed to the user. Specific operations include creating the guide and delivering it to the terminal. The terminal receives it and displays it to the user.

[1615] Step 6:

[1616] On-site response (user)

[1617] The user performs operations according to the instructions displayed on the terminal. For example, if instructed to restart a pump, the user operates according to the procedure. The input is the operation guide, and the output is the result of the operation. Specific operations include actual operations on site and feedback of the results.

[1618] Step 7:

[1619] Sentiment analysis and notification adjustment (device and server)

[1620] The device uses an emotion engine to analyze the user's emotional state in real time and transmits that information to the server. The input is the user's biometric information and voice data, and the output is the emotion analysis results. The server adjusts the notification content and response methods based on these results. Specific operations include analyzing the emotion data, transmitting the analysis results to the server, and flexibly changing the notification content.

[1621] Step 8:

[1622] Receiving and analyzing feedback (server)

[1623] The server receives feedback from users and analyzes the corresponding results. The input is the user's feedback information, and the output is the analysis results and additional information added to the database. Specific operations include receiving feedback, analyzing it, saving it to the database, and using it as future model training data.

[1624] (Application example 2)

[1625] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1626] Conventional equipment maintenance and repair systems can sometimes have difficulty responding quickly and appropriately when a fault occurs. In particular, when workers are not familiar with dealing with faults, delays and errors can occur. Furthermore, when workers are in an unstable psychological state, it becomes difficult to respond efficiently, which can lead to reduced reliability and efficiency of the equipment. There is a need for a system that can solve these problems and carry out equipment maintenance and repair work more efficiently and accurately.

[1627] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1628] In this invention, the server includes means for acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals, means for receiving and analyzing real-time data and alarms, means for identifying and recording failures from the received data, means for generating temporary and permanent response methods based on the failure, means for creating and distributing an on-site operation guide, and means for analyzing the emotional state of users in real time and adjusting response methods and notification content based on that information. This enables prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the equipment.

[1629] "Past failure cases" refers to specific records of past failures that occurred in the equipment and the history of how they were dealt with.

[1630] "Response methods" refer to the specific actions and procedures to be taken when a failure occurs, and the related processes.

[1631] "Facility configuration information" refers to data relating to the physical and logical components of the facility in question.

[1632] "Instruction manual" refers to guidelines and manuals on how to use, maintain, and troubleshoot equipment.

[1633] "Real-time data" refers to data obtained from equipment regarding its current operating conditions and parameters.

[1634] An "alarm" refers to a warning signal issued when an abnormality or failure in equipment is detected.

[1635] "Emotional state" refers to a state that indicates a user's psychological state or emotional response.

[1636] "Analysis" refers to analyzing data and extracting meaningful patterns and information from it.

[1637] "Identifying the fault" refers to clarifying the cause and location of an abnormality or problem that has occurred.

[1638] "Temporary measures" refer to temporary measures to improve the situation in the short term when a problem occurs.

[1639] "Permanent measures" refer to long-term measures to eliminate the root cause to prevent recurrence and improve the reliability of the equipment.

[1640] An "operation guide" refers to an instruction manual that shows the steps and precautions a user should take when performing a specific operation.

[1641] "Notification content" refers to messages and instructions sent from the system to the user.

[1642] The present invention is a system for automating equipment maintenance and upkeep in a factory, and in particular, provides real-time troubleshooting using smart glasses and flexible notifications based on the emotional state of the user. An embodiment of this system will be described below.

[1643] Server-side roles

[1644] The server plays a central role in collecting and analyzing data within the factory. Specifically, it processes and calculates the data using the following hardware and software:

[1645] Hardware: High-performance server

[1646] Software: TensorFlow (for training generative AI models), database management system (DBMS)

[1647] The server first obtains past failure cases, response methods, equipment configuration information, and instruction manuals from the equipment and stores this data in a local database. It then uses this data to train a generative AI model. It receives real-time data and alarms from remote sensors and cameras, analyzes them, and identifies failures. Based on the identified failures, it generates temporary and permanent solutions and notifies the user. If on-site operation is required, it also creates a detailed operation guide and distributes it to the smart glasses.

[1648] The server also incorporates an emotion engine to analyze the user's emotional state, analyzing the user's real-time emotional state (camera footage and audio) and adjusting the response method and notification content based on that information.

[1649] The role of the terminal

[1650] Smart glasses are used as the device, and the device has the following functions:

[1651] Hardware: Smart glasses (camera, display, microphone)

[1652] Software: OpenCV (image analysis), Emotion API (emotion analysis)

[1653] The device displays notifications and operation guides from the server to the user. It also analyzes the user's emotional state in real time using data acquired from the camera and microphone and sends that information to the server, allowing the server to provide the optimal response based on the user's emotional state.

[1654] User Role

[1655] The user operates the device (smart glasses) by following the notifications and operation guide displayed on the device. Specific operation examples are shown below.

[1656] For example, if a pump malfunction occurs on a production line, the following measures will be taken.

[1657] 1. Receiving and analyzing real-time data (server)

[1658] The server receives real-time data showing the pump's operating status and analyzes it using a generative AI model to identify the cause of the fault.

[1659] 2. Generate a temporary solution (server)

[1660] If the server determines that the abnormality can be resolved by restarting the pump, it generates a temporary solution such as "restart the pump."

[1661] 3. Distribution of operation guide (server and terminal)

[1662] Based on this temporary solution, the server creates an operation guide including reboot procedures and precautions, and distributes it to the smart glasses.

[1663] 4. On-site response (user)

[1664] The user follows the instructions displayed on the smart glasses to restart the pump, and the results are entered into the smart glasses and sent to the server.

[1665] 5. Sentiment analysis and notification adjustment (device and server)

[1666] The smart glasses use a camera and microphone to analyze the user's emotional state (e.g., stress level) and send that information to a server. If the user is feeling stressed, the server can flexibly change the notification content and response methods to provide a more appropriate response.

[1667] Here is an example of a prompt from a generative AI model:

[1668] "While the user is wearing the smart glasses, the AI ​​model should analyze images of machinery with bearing abnormalities and detect the bearing abnormality. Also, if the user's stress level is high, the smart glasses should display a message instructing the user to 'take a deep breath and calm down' and restart the device."

[1669] As a result, the system of the present invention enables a prompt and appropriate response when a failure occurs, reduces the psychological burden on users, and improves the reliability and efficiency of the facility.

[1670] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1671] Step 1:

[1672] The server collects data from within the factory, including past failure cases, troubleshooting methods, equipment configuration information, and instruction manuals, and stores this information in a local database. This creates a database for subsequent failure analysis and countermeasure development.

[1673] Input: Past failure cases and instruction manuals obtained from the equipment

[1674] Output: Data stored in a local database

[1675] Step 2:

[1676] The server uses the collected data to train the generative AI model, which uses TensorFlow to analyze equipment configuration information and past failure cases to learn patterns when failures occur.

[1677] Input: Data stored in a local database

[1678] Output: Trained generative AI model

[1679] Step 3:

[1680] The server receives real-time data and alarms from remote sensors and cameras in the factory, including equipment operating status, pressure, temperature, and more.

[1681] Input: Real-time data from remote sensors and cameras

[1682] Output: Real-time monitoring data

[1683] Step 4:

[1684] The server analyzes the received real-time data and identifies faults using a generative AI model, which uses the received data as input to determine whether a fault has occurred and its cause.

[1685] Input: Real-time monitoring data

[1686] Output: Whether there is a fault and its cause

[1687] Step 5:

[1688] The server generates temporary solutions based on the identified faults (for example, if a pump needs to be restarted, it generates a restart procedure), and also suggests the necessary repairs or part replacements for permanent solutions.

[1689] Input: Identified fault information

[1690] Output: Temporary and permanent solutions

[1691] Step 6:

[1692] The server creates a detailed operation guide based on the generated temporary and permanent solutions and distributes it to the device (smart glasses).

[1693] Input: Temporary and permanent solutions

[1694] Output: Operation guide to terminal

[1695] Step 7:

[1696] The device (smart glasses) displays the operation guide received from the server to the user, who then follows the guide to perform the operation.

[1697] Input: Operation guide from the server

[1698] Output: Guide display for users

[1699] Step 8:

[1700] The device uses a camera and microphone to analyze the user's emotional state (e.g., stress level) in real time and transmits the information to a server, where the analysis is performed using an emotion engine.

[1701] Input: Data from the camera and microphone

[1702] Output: User's emotional state data

[1703] Step 9:

[1704] The server flexibly adjusts the notification content and response methods based on the user's emotional state data. For example, if the user is feeling stressed, the server may adjust the notification content to be gentler or provide instructions for additional breaks.

[1705] Input: User's emotional state data

[1706] Output: Adjusted notification content and action to be taken

[1707] Step 10:

[1708] The user follows the instructions displayed on the device (smart glasses) to take local action. For example, they restart the pump, enter the results into the device, and send them to the server.

[1709] Input: the instructions displayed on the terminal

[1710] Output: Send the restart result to the server

[1711] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1712] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1713] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1714] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1715] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1716] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1717] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1718] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1719] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1720] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1721] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1722] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1723] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1724] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1725] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1726] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1727] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1728] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1729] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1730] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1731] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1732] The following is further disclosed regarding the above embodiment.

[1733] (Claim 1)

[1734] A system for automating maintenance and upkeep of equipment when a failure occurs, comprising:

[1735] A means of acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals;

[1736] means for receiving and analyzing real-time data and alarms;

[1737] means for identifying and recording faults from the received data;

[1738] means for generating temporary and permanent workarounds based on the fault;

[1739] A system that includes a means to create and distribute locally adapted operation guides.

[1740] (Claim 2)

[1741] 10. The system of claim 1, wherein the system analyzes real-time data to detect faults.

[1742] (Claim 3)

[1743] 2. The system according to claim 1, wherein a temporary solution to the detected failure is generated and notified.

[1744] "Example 1"

[1745] (Claim 1)

[1746] A means of acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals;

[1747] means for receiving and analyzing real-time data and warning signals;

[1748] A means of quickly and accurately identifying faults from incoming data using generative AI models; and

[1749] means for generating temporary and permanent workarounds based on the fault;

[1750] A means for generating and distributing operational procedures for on-site response to the terminal;

[1751] means for receiving and analyzing feedback from users;

[1752] A system including:

[1753] (Claim 2)

[1754] 10. The system of claim 1, wherein the system uses a generative AI model that analyzes real-time data to detect faults.

[1755] (Claim 3)

[1756] 2. The system according to claim 1, wherein a temporary solution to the detected failure is generated and notified to the terminal.

[1757] "Application Example 1"

[1758] (Claim 1)

[1759] A system for automating maintenance and upkeep of equipment when a failure occurs, comprising:

[1760] A means of acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals;

[1761] means for receiving and analyzing real-time data and alarms;

[1762] means for identifying and recording faults from the received data;

[1763] means for generating temporary and permanent workarounds based on the fault;

[1764] A means of creating and distributing local operation guides,

[1765] A means for factory robots to monitor their operating status in real time and detect abnormalities,

[1766] A means for identifying the cause of a failure using a generative AI model trained on a server;

[1767] A means of providing operational guides to local operators so that they can follow instructions

[1768] A system including:

[1769] (Claim 2)

[1770] 10. The system of claim 1, wherein the system analyzes real-time data to detect faults.

[1771] (Claim 3)

[1772] 2. The system according to claim 1, wherein a temporary solution to the detected failure is generated and notified.

[1773] "Example 2: Combining Emotion Engines"

[1774] (Claim 1)

[1775] A means of acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals;

[1776] means for receiving and analyzing real-time data and alarms;

[1777] means for identifying and recording faults from the received data;

[1778] means for generating temporary and permanent workarounds based on the fault;

[1779] A means of creating and distributing local operation guides,

[1780] A system that includes a means for analyzing a user's emotions and adjusting the response method and notification content based on the analysis results.

[1781] (Claim 2)

[1782] 10. The system of claim 1, wherein the system analyzes real-time data to detect faults.

[1783] (Claim 3)

[1784] 2. The system according to claim 1, wherein a temporary solution to the detected failure is generated and notified.

[1785] "Application example 2 when combining emotion engines"

[1786] (Claim 1)

[1787] A system for automating maintenance and upkeep of equipment when a failure occurs, comprising:

[1788] A means of acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals;

[1789] means for receiving and analyzing real-time data and alarms;

[1790] means for identifying and recording faults from the received data;

[1791] means for generating temporary and permanent workarounds based on the fault;

[1792] A means of creating and distributing local operation guides,

[1793] A system that includes a means to analyze the user's emotional state in real time and adjust response methods and notification content based on that information.

[1794] (Claim 2)

[1795] 10. The system of claim 1, wherein the system analyzes real-time data to detect faults.

[1796] (Claim 3)

[1797] 2. The system according to claim 1, wherein a temporary solution to the detected failure is generated and notified. [Explanation of symbols]

[1798] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A system for automating maintenance and upkeep of equipment when a failure occurs, A means of acquiring and learning from past failure cases, response methods, equipment configuration information, and instruction manuals; means for receiving and analyzing real-time data and alarms; means for identifying and recording faults from the received data; means for generating temporary and permanent workarounds based on the fault; A system that includes a means to create and distribute locally adapted operation guides.

2. 10. The system of claim 1, wherein the system analyzes real-time data to detect faults.

3. 2. The system according to claim 1, wherein a temporary solution to the detected failure is generated and notified.

Images