system

Abstract

We provide the system. [Solution] A means of receiving account registration information from users and storing it in a database, A means of sending a verification link to an email address based on the account registration information, A means of receiving a confirmation link click from the user and updating the authentication status, A means of receiving the username and password entered by the user on the login screen, and authenticating them by comparing them with the information in the database, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Companies aim to automate their operations efficiently and operate with minimal employee resources. However, existing systems require a lot of manual settings and monitoring, which is a heavy burden. In addition, it is difficult to deploy the acquired know-how and information to other companies, the metaverse, and digital twins, and the scope of their utilization is limited. There is a need for a method to improve such problems and achieve smooth and efficient business automation and accumulation and sharing of know-how.

Means for Solving the Problems

[0005] The present invention enables efficient account management through a system that includes means for receiving account registration information from users and storing it in a database, means for sending a confirmation link to an email address based on the account registration information, means for receiving a click on the confirmation link from the user and updating the authentication status, and means for receiving the username and password entered by the user from the login screen and performing authentication by comparing them with the information in the database.

[0006] Furthermore, by including means for receiving business task configuration information and generating automated processes using an AI model, means for monitoring the progress of the generated automated processes in real time and detecting anomalies, and means for providing users with a dashboard that notifies them of the process progress and anomaly information, the system achieves increased efficiency in business automation and optimized monitoring.

[0007] In addition, by including means for collecting result data from automated business processes and user feedback and storing it in a database, means for analyzing the stored data using AI / machine learning models to extract areas for improvement and patterns, and means for providing the analysis results to users and reflecting them in the design of new automated processes, it is possible to accumulate and share know-how and improve business processes.

[0008] A "user" refers to an individual or group that accesses the system and automates or manages tasks.

[0009] "Account registration information" refers to the basic information (such as username, password, and email address) that a user provides when they first access the system.

[0010] A "database" refers to a collection of data used to efficiently store, manage, and retrieve necessary information within a system.

[0011] A "verification link" refers to a hyperlink included in an email that users click to verify or authenticate their account.

[0012] "Authentication status" refers to status information that indicates whether a user's account is active or not.

[0013] A "task" refers to a specific job or activity, and is the target of a system's automation process.

[0014] An "AI model" refers to an algorithm trained to perform a specific task using artificial intelligence.

[0015] An "automation process" refers to a series of business procedures that are automatically executed by an AI model.

[0016] A "dashboard" refers to an interface that visually displays information about the system's status and ongoing processes.

[0017] "Feedback" refers to the opinions and suggestions for improvement that users provide regarding the system's operation and results.

[0018] "Anomaly detection" refers to a process that automatically identifies abnormal conditions or problems that deviate from normal business processes.

[0019] "Know-how" refers to the knowledge and skills needed to efficiently perform specific tasks or duties.

[0020] "Analysis" refers to the process of using collected data to uncover patterns and trends and gain insights.

[0021] The term "metaverse" refers to a virtual shared space that utilizes virtual reality and augmented reality.

[0022] A "digital twin" refers to a technology that digitally reproduces a physical object or system in real time. [Brief explanation of the drawing]

[0023] [Figure 1] It is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] It is a conceptual diagram showing an example of the main functions of a data processing device and a smart device according to the first embodiment. [Figure 3] It is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] It is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] It is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

MODE FOR CARRYING OUT THE INVENTION

[0024] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0025] First, let's explain the terminology used in the following explanation.

[0026] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0027] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0028] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0029] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0030] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0031] [First Embodiment]

[0032] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0033] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0034] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0035] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0036] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0037] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0038] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0039] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0040] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0041] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0042] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0043] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0044] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, and feedback collection and analysis. The specific processing of each module and its implementation examples are described below.

[0045] Account Management Module

[0046] User registration and authentication

[0047] The user uses a device to access the system and enter their account registration information. The device sends the entered information, such as username, password, and email address, to the server. The server stores the received information in a database and sends a confirmation link to the registered email address. When the user clicks the confirmation link in the email, the device sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server, which then authenticates them by comparing it with the information in the database.

[0048] Business task automation module

[0049] Setting up and automating business tasks

[0050] The user enters the business tasks they want to automate (e.g., report creation or data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model. The server then trains the AI ​​model based on the task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[0051] Process monitoring and management module

[0052] Progress monitoring and anomaly detection

[0053] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. The server detects data that deviates from configured parameters and baseline values, and generates an alert when an anomaly is found. The terminal notifies the user of this alert information, allowing them to check progress and problems on a real-time dashboard.

[0054] Feedback collection and analysis module

[0055] Data collection and analysis

[0056] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server then analyzes this data using AI / machine learning models to extract areas for improvement and patterns. The analysis results are then provided to users via their terminals, allowing them to incorporate them into the design of new automated processes.

[0057] External sales compatible module

[0058] Service customization and delivery

[0059] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. The customized service package is released, and support is provided to users during its use. The terminal collects feedback from users and sends it to the server.

[0060] Metaverse / Digital Twin Integration Module

[0061] Data transformation and model generation

[0062] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. Based on the converted data, the server generates a metaverse or digital twin model, and the terminal provides the user with a preview of the model. The user then starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[0063] As described above, each module works in conjunction with others to enable efficient automation, management, analysis, and feedback collection and sharing of operations, providing an environment in which companies can operate with minimal resources.

[0064] The following describes the processing flow.

[0065] Account Management Module

[0066] User registration and authentication

[0067] Step 1:

[0068] The user enters their account registration information (username, password, email address).

[0069] Step 2:

[0070] The terminal sends the entered information to the server.

[0071] Step 3:

[0072] The server saves the received information to a database, generates a confirmation link, and sends it to the registered email address.

[0073] Step 4:

[0074] The user clicks the confirmation link sent to their email address.

[0075] Step 5:

[0076] The device sends a request to the server based on clicking the verification link.

[0077] Step 6:

[0078] The server receives the request and updates the authentication status. It then sends a notification to the user that they can log in.

[0079] Step 7:

[0080] The user enters their username and password on the login screen.

[0081] Step 8:

[0082] The terminal sends the entered information to the server.

[0083] Step 9:

[0084] The server verifies the information against the database and performs authentication. If authentication is successful, the user is provided with a dashboard screen.

[0085] Business task automation module

[0086] Setting up and automating business tasks

[0087] Step 1:

[0088] The user enters the business tasks they want to automate on the settings screen.

[0089] Step 2:

[0090] The terminal sends the configured work task information to the server.

[0091] Step 3:

[0092] The server analyzes the task information it receives and selects the appropriate AI model.

[0093] Step 4:

[0094] The server trains an AI model based on task information and generates an automated process.

[0095] Step 5:

[0096] The server sends the progress and results of the automated process it generates to the terminal.

[0097] Step 6:

[0098] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[0099] Process monitoring and management module

[0100] Progress monitoring and anomaly detection

[0101] Step 1:

[0102] The server monitors the progress of automated business processes in real time and records logs.

[0103] Step 2:

[0104] The database stores monitoring data and works in conjunction with anomaly detection algorithms.

[0105] Step 3:

[0106] The server detects data that deviates from the configured parameters and baseline values, and generates an alert when an anomaly is detected.

[0107] Step 4:

[0108] The terminal provides alert information to the user through an interface.

[0109] Step 5:

[0110] Users can check progress and issues in real time on the dashboard screen.

[0111] Feedback collection and analysis module

[0112] Data collection and analysis

[0113] Step 1:

[0114] The server continuously collects result data from automated processes and feedback from users.

[0115] Step 2:

[0116] The database systematically stores all collected data.

[0117] Step 3:

[0118] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns.

[0119] Step 4:

[0120] The terminal visualizes the analysis results through an interface that provides them to the user.

[0121] Step 5:

[0122] Users review the analysis results and incorporate them into the design of new automated processes.

[0123] External sales compatible module

[0124] Service customization and delivery

[0125] Step 1:

[0126] The server generates standard service packages for external sales based on accumulated know-how.

[0127] Step 2:

[0128] Users can customize the details of external sales services on the settings screen.

[0129] Step 3:

[0130] The device sends customized configuration information to the server.

[0131] Step 4:

[0132] The server reflects the configuration and generates a new service package.

[0133] Step 5:

[0134] The server releases customized service packages and supports users in use.

[0135] Step 6:

[0136] The device collects feedback from users while they are using it and sends it to the server.

[0137] Metaverse / Digital Twin Integration Module

[0138] Data transformation and model generation

[0139] Step 1:

[0140] The server converts business data into formats for the metaverse or digital twin.

[0141] Step 2:

[0142] The database stores the converted data.

[0143] Step 3:

[0144] The server generates metaverse and digital twin models based on the converted data.

[0145] Step 4:

[0146] The device provides the user with a preview of the model.

[0147] Step 5:

[0148] Users initiate simulations within the metaverse or digital twin to verify business processes.

[0149] Step 6:

[0150] The terminal displays and provides the user with simulation results in real time.

[0151] Step 7:

[0152] The server analyzes the simulation results and accumulates feedback.

[0153] (Example 1)

[0154] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0155] Traditional systems relied on manual processes for automating, monitoring, and analyzing business processes, leading to decreased operational efficiency. Furthermore, effectively collecting and analyzing user feedback and incorporating it into new process design proved difficult. Additionally, the limited use of virtual spaces and digital replication hindered the easy simulation and verification of business processes.

[0156] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0157] In this invention, the server includes means for receiving account registration information from a user and storing it in a database; means for sending a confirmation link to an email address based on the account registration information; means for receiving a click of the confirmation link from the user and updating the authentication status; means for converting business data into a virtual space or digital replica format and storing it on an analysis platform; means for generating a virtual space or digital replica model based on the converted data; and means for providing the user with a preview of the generated model and initiating a simulation. This enables automation and efficiency improvements of business operations, real-time monitoring and anomaly detection, and effective collection and analysis of user feedback.

[0158] A "user" refers to anyone who uses this system to perform operations such as registering an account, setting up work tasks, logging in, and providing feedback.

[0159] A "server" refers to a computer system that receives requests from users, stores information in a database, and performs overall system processing, including training and running AI models, generating automated processes, monitoring progress, detecting anomalies, and generating models for virtual spaces and digital replicas.

[0160] "Database" refers to a digital storage system for permanently storing various types of data required by this system, such as account registration information, work task information, progress data, and feedback data.

[0161] A "verification link" refers to a temporary URL sent to the user's email address, which the user clicks to authenticate their account.

[0162] "Authentication status" refers to status information indicating whether the user has clicked the verification link and completed account authentication.

[0163] "Business tasks" refer to business processes, such as report creation and data entry, that users set up in this system.

[0164] A "generative AI model" refers to an artificial intelligence model trained to generate appropriate automation processes based on the user's business task information.

[0165] A "prompt" refers to the text information or question sentences that are input into a generative AI model.

[0166] "Anomaly detection" refers to the process of monitoring the progress of automated business processes and identifying data that deviates from set parameters or baseline values.

[0167] A "virtual space" refers to a digital environment that digitally reproduces the actual physical space for conducting simulations and model verification.

[0168] "Digital replication" refers to a virtual model that reproduces actual business processes and environments as digital data and is used for simulation and verification.

[0169] "Visualized display methods" refer to display methods that use dashboards, graphs, and charts to allow users to intuitively understand the progress of a process and any anomaly information.

[0170] "Analysis techniques" refer to algorithms and methods for analyzing large amounts of data stored in a database and extracting patterns and areas for improvement.

[0171] The above are the key terms and their definitions included in the claims of this system.

[0172] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, feedback collection and analysis, external sales support, and metaverse / digital twin integration. The specific operation of each module is described in detail below.

[0173] Account Management Module

[0174] User registration and authentication

[0175] The user accesses the system using a terminal and enters account registration information such as username, password, and email address. The terminal sends this information to the server, which stores the received information in a database (e.g., MySQL®). The server then sends a verification link to the registered email address (e.g., using an email service). When the user clicks the verification link, the terminal sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the terminal sends this information to the server. The server authenticates the user by comparing it with the information in the database.

[0176] Business task automation module

[0177] Setting up and automating business tasks

[0178] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model (e.g., TENSORFLOW®). The server then trains the AI ​​model based on this task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[0179] Process monitoring and management module

[0180] Progress monitoring and anomaly detection

[0181] The server monitors the progress of automated business processes in real time and logs the data (e.g., using a monitoring tool). The database stores the monitoring data and works with anomaly detection algorithms (e.g., Scikit-learn). The server detects data that deviates from configured parameters or baseline values ​​and generates an alert when an anomaly is found. The terminal notifies the user of this alert information and allows them to check progress and problems on a real-time dashboard.

[0182] Feedback collection and analysis module

[0183] Data collection and analysis

[0184] The server continuously collects result data from automated processes and user feedback, storing it in a database (e.g., PostgreSQL). The server then analyzes the stored data using an AI / machine learning model (e.g., PyTorch) to extract areas for improvement and patterns. The analysis results are provided to the user via a terminal, allowing the user to incorporate them into the design of new automated processes.

[0185] External sales compatible module

[0186] Service customization and delivery

[0187] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. It releases the customized service package and supports users who are using it. The terminal collects feedback from users and sends it to the server.

[0188] Metaverse / Digital Twin Integration Module

[0189] Data transformation and model generation

[0190] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. The server generates a model of the metaverse or digital twin based on the converted data (e.g., using Unity). The terminal provides the user with a preview of the model. The user starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[0191] Examples of specific cases and prompt statements

[0192] For example, a "mail service" can be used to send a confirmation link to a registered email address, and "MySQL" can be used as the database to store user information. Furthermore, "TensorFlow" or "PyTorch" can be used to train AI models. "Visualization tools" can be used to monitor the situation on the dashboard, and "Scikit-learn" can be utilized for anomaly detection. "3D modeling tools" can be used to generate models in virtual spaces or digital replicas.

[0193] Prompt example 1: "Send a confirmation email for new user registration and update the authentication status."

[0194] Prompt example 2: "Use a generative AI model to automate report generation tasks and visualize progress."

[0195] Prompt example 3: "Apply the anomaly detection algorithm and display an alert if an anomaly occurs."

[0196] Prompt example 4: "We will extract patterns through data analysis and identify areas for improvement."

[0197] As described above, each module works together through detailed and specific procedures, providing an environment where companies can efficiently automate their operations and operate with minimal resources.

[0198] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0199] Account Management Module

[0200] User registration and authentication

[0201] Step 1:

[0202] The user accesses the system using a terminal and enters account registration information such as username, password, and email address.

[0203] Input: Username, Password, Email Address

[0204] Output: The request the terminal sends to the server with this information.

[0205] Step 2:

[0206] The device sends the entered account registration information to the server.

[0207] Input: Username, password, and email address entered by the user.

[0208] Output: Data packets to be sent to the server

[0209] Step 3:

[0210] The server parses the information it receives and saves it to a database (for example, MySQL).

[0211] Input: Account registration information sent from the device

[0212] Output: Saving information to the database using SQL INSERT statements

[0213] Specific operation: Execute SQL queries using the Python sqlite3 module or MySQL client.

[0214] Step 4:

[0215] The server will send a confirmation link to the registered email address.

[0216] Input: User's email address

[0217] Output: Email containing a confirmation link

[0218] Specific operation: Use an email service (e.g., SendGrid) to send emails via API.

[0219] Step 5:

[0220] The user clicks the verification link, and the device sends an authentication request to the server.

[0221] Input: Link access via user click

[0222] Output: Request to the server

[0223] Specific action: The device sends an HTTP GET request to the server.

[0224] Step 6:

[0225] The server receives the request and updates the authentication status.

[0226] Input: Authentication request to the server

[0227] Output: Authentication status update

[0228] Specific action: Update the status field of the corresponding user in the database.

[0229] Step 7:

[0230] The user enters their username and password on the login screen, and the device sends that information to the server.

[0231] Input: Username, Password

[0232] Output: Login request to the server

[0233] Specific action: Send data using the terminal's form submission function.

[0234] Step 8:

[0235] The server compares the information with that in the database and performs authentication.

[0236] Input: Username, Password

[0237] Output: Authentication success return code or error message

[0238] Specific operation: Database matching is performed using an SQL SELECT statement.

[0239] ---

[0240] Business task automation module

[0241] Setting up and automating business tasks

[0242] Step 1:

[0243] The user enters the business tasks they want to automate (e.g., report creation, data entry) on the settings screen.

[0244] Input: Detailed information about the task

[0245] Output: The request the terminal sends to the server with this information.

[0246] Step 2:

[0247] The device sends configuration information to the server.

[0248] Input: Detailed information about the task

[0249] Output: Request for configuration information from the server

[0250] Step 3:

[0251] The server analyzes the task information and selects an appropriate AI model (e.g., TensorFlow).

[0252] Input: Detailed information about the task

[0253] Output: Selected AI models

[0254] Specific action: Execute the AI ​​model selection algorithm.

[0255] Step 4:

[0256] The server trains the AI ​​model and generates automated processes.

[0257] Input: Detailed information on the business task, selected AI model

[0258] Output: Trained model, generated automation process

[0259] Specific operation: Train the model using the TensorFlow library.

[0260] Step 5:

[0261] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[0262] Input: Progress data, result data

[0263] Output: Visualized dashboard

[0264] Specific actions: Use visualization tools (e.g., Grafana) to display data in graphs and charts.

[0265] ---

[0266] Process monitoring and management module

[0267] Progress monitoring and anomaly detection

[0268] Step 1:

[0269] The server monitors the progress of automated business processes in real time and records logs.

[0270] Input: Business process progress data

[0271] Output: Real-time monitoring data, recorded logs

[0272] Specific operation: Collect real-time data using a monitoring tool (e.g., Prometheus) and save it as a log.

[0273] Step 2:

[0274] The database stores the monitoring data.

[0275] Input: Real-time monitoring data

[0276] Output: Saved monitoring data

[0277] Specific operation: Execute an SQL INSERT statement into the database and save the data.

[0278] Step 3:

[0279] The server applies an anomaly detection algorithm.

[0280] Input: Stored monitoring data

[0281] Output: Anomaly detection result

[0282] Specific operation: Execute an anomaly detection algorithm using Scikit - learn.

[0283] Step 4:

[0284] The server detects an anomaly and generates an alert.

[0285] Input: Anomaly detection result

[0286] Output: Generated alert

[0287] Specific operation: Generate an alert message and output it in the specified format.

[0288] Step 5:

[0289] The terminal notifies the user of the alert information.

[0290] Input: Generated alert

[0291] Output: Notification to the user

[0292] Specific operation: Use the GUI to perform pop - up notifications and email notifications.

[0293] Step 6:

[0294] Enable the terminal to view the progress and problems in the real - time dashboard.

[0295] Input: Alert information, progress data

[0296] Output: Visualized dashboard<\

[0297] Specific operation: Use a visualization tool (e.g., Grafana) to display the data.

[0298] ---

[0299] Feedback Collection and Analysis Module

[0300] Data Collection and Analysis

[0301] Step 1:

[0302] The server continuously collects the result data of the automation process and feedback from users, and accumulates it in the database.

[0303] Input: Result data, feedback data

[0304] Output: Accumulated data

[0305] Specific operation: Use the API to collect data and save it in the database.

[0306] Step 2:

[0307] The server analyzes the accumulated data using an AI / machine learning model.

[0308] Input: Accumulated data

[0309] Output: Analysis results (improvement points and patterns)

[0310] Specific operation: Use PyTorch to stream the dataset into the model for analysis.

[0311] Step 3:

[0312] The terminal provides the analysis results to the user.

[0313] Input: Analysis results

[0314] Output: Display of analysis results to the user

[0315] Specific operation: Visualize and display the results on the dashboard.

[0316] Step 4:

[0317] Users will incorporate their feedback into the design of new automation processes.

[0318] Input: Analysis results, feedback

[0319] Output: Newly designed automation process

[0320] Specific actions: Use operational design tools to design a new process.

[0321] ---

[0322] External sales compatible module

[0323] Service customization and delivery

[0324] Step 1:

[0325] The server generates the standard service package.

[0326] Input: Accumulated know-how

[0327] Output: Standard service package

[0328] Specific actions: Reading data from the database and executing a template generation script.

[0329] Step 2:

[0330] The user can customize the settings.

[0331] Input: Customization details

[0332] Output: Sending of configuration information by the terminal

[0333] Specific operation: Form input and submission via GUI.

[0334] Step 3:

[0335] The device sends customization information to the server.

[0336] Input: Customization details

[0337] Output: Request to the server

[0338] Specific operation: Send JSON data via an HTTP POST request.

[0339] Step 4:

[0340] The server will reflect the customizations.

[0341] Input: Customization Information

[0342] Output: Customized service package

[0343] Specific action: Generate a new package using a template and save it.

[0344] Step 5:

[0345] The server releases customized service packages and supports users in use.

[0346] Input: Customized package

[0347] Output: Released service packages, support information

[0348] Specific operations: Deploy and support using cloud services such as AWS (registered trademark).

[0349] Step 6:

[0350] The device collects feedback and sends it to the server.

[0351] Input: User feedback

[0352] Output: Sending feedback data to the server

[0353] Specific action: Submit data through the feedback form.

[0354] ---

[0355] Metaverse / Digital Twin Integration Module

[0356] Data transformation and model generation

[0357] Step 1:

[0358] The server converts business data into formats for the metaverse and digital twin.

[0359] Input: Business data

[0360] Output: Converted data

[0361] Specific operation: Executes a data conversion script (e.g., Python or C++).

[0362] Step 2:

[0363] The server saves the converted data to the database.

[0364] Input: Converted data

[0365] Output: Saved data

[0366] Specific action: Execute an SQL INSERT statement into the database.

[0367] Step 3:

[0368] The server generates metaverse and digital twin models based on the converted data.

[0369] Input: Converted data

[0370] Output: Generated model

[0371] Specific actions: Build the model using the Unity API.

[0372] Step 4:

[0373] The device provides the user with a preview of the model.

[0374] Input: Generated model

[0375] Output: Model preview screen

[0376] Specific action: Display the model using the 3D viewer.

[0377] Step 5:

[0378] The user starts a simulation within the metaverse or digital twin.

[0379] Input: Instruction to start simulation

[0380] Output: Run the simulation

[0381] Specific action: Click the Start button to begin the simulation.

[0382] Step 6:

[0383] The device displays the simulation results in real time.

[0384] Input: Simulation result data

[0385] Output: Simulation results displayed in real time

[0386] Specific actions: Display data in dashboards and graphs.

[0387] Step 7:

[0388] The server analyzes the simulation results and accumulates feedback.

[0389] Input: Simulation result data

[0390] Output: Analysis results, feedback

[0391] Specific operation: Analyze the result data and save it as Insights.

[0392] Through the above processing steps, each module is efficiently linked, enabling automation of operations, monitoring of progress, anomaly detection, feedback collection, customized services, and integration with the metaverse and digital twin.

[0393] (Application Example 1)

[0394] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0395] In modern factories, while efficiency and automation are essential, managing robots and setting and monitoring tasks are becoming increasingly complex. The lack of integrated real-time monitoring of work progress, anomaly detection, feedback collection and analysis, and account management for technicians and managers often hinders operational efficiency. Furthermore, the lack of established methods for designing and improving automation processes using generative AI models is slowing down the optimization of operations.

[0396] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0397] In this invention, the server includes means for receiving authentication information from a user and storing it in a database; means for sending a confirmation link via communication means based on the authentication information; means for receiving a click of the confirmation link from a user and updating the authentication status; means for receiving authentication information entered by a user from an authentication screen and performing authentication by comparing it with the information in the database; means for managing accounts of robot operators and administrators and performing access control; means for receiving setting information of business tasks from authenticated users and generating an automated process using a generation AI model; means for monitoring the progress of the generated automated process in real time and detecting anomalies; means for providing a dashboard that notifies the user of the progress of the process and anomaly information; means for collecting result data of the automated business process and feedback from users and storing it in a database; means for analyzing the stored data using a generation AI model and a machine learning model and extracting areas for improvement and patterns; and means for providing the user with the analysis results and reflecting them in the design of new automated processes. As a result, robot management and automation of business tasks in the factory are integrated, enabling improved work efficiency and faster anomaly detection.

[0398] "Authentication information" refers to information used to identify a user and verify their access rights.

[0399] A "database" is an electronic recording device used to efficiently store, search, and manage large amounts of data.

[0400] A "verification link" is a URL that users receive via email or other means and click to authenticate their account.

[0401] "Authentication status" refers to the state of information indicating whether a user has legitimate authority.

[0402] A "robot operator or manager" is a person responsible for operating and managing robots within a factory.

[0403] "Access control" is the process of restricting and managing access rights to information systems.

[0404] "Business tasks" refer to specific tasks or operations that are performed by robots within a factory.

[0405] A "generative AI model" is an artificial intelligence model trained using machine learning algorithms to generate automated processes for business tasks.

[0406] An "automation process" is a set of steps or processes that automatically execute specific tasks or operations.

[0407] "Real-time" refers to the process and display of data and information immediately as soon as it is generated.

[0408] "Anomaly detection" is the process of discovering behavior or errors that deviate from normal operation.

[0409] A "dashboard" is a user interface that visually displays multiple pieces of information, allowing users to check progress and any anomalies at a glance.

[0410] "Feedback" refers to information about opinions and results collected from users and systems.

[0411] "Analysis" is the process of processing and evaluating collected data to extract useful information.

[0412] "Areas for improvement" are parts of a business process that require change to optimize it.

[0413] A "pattern" refers to the regularity or characteristics found within data.

[0414] The system that realizes this application consists of multiple modules and aims to efficiently automate robot management and tasks within a factory. The hardware and software used, as well as specific implementation examples, are described below.

[0415] Hardware and software used

[0416] 1. Server

[0417] Database management systems: MySQL and PostgreSQL

[0418] AI and machine learning frameworks: TensorFlow and PyTorch

[0419] Web frameworks: Flask and Django

[0420] 2. Terminal

[0421] Personal computers, smartphones, tablets, and other devices that users use for input and display.

[0422] 3. User

[0423] Robot operators and managers

[0424] System configuration and details

[0425] 1. Account Management Module

[0426] The server first receives authentication information from the user and stores it in the database. Next, it sends a confirmation link via communication means based on that authentication information. When the user clicks the confirmation link, the server receives the request and updates the authentication status. It receives the authentication information entered by the user on the authentication screen, verifies it against the information in the database, and performs authentication. It also manages accounts for robot operators and administrators and performs access control.

[0427] 2. Business Task Automation Module

[0428] The system receives task configuration information from authenticated users and generates automated processes using a generated AI model. The server monitors this process in real time and detects anomalies. Users are provided with a dashboard that notifies them of process progress and anomaly information. For example, it can easily configure tasks for robots performing welding operations.

[0429] 3. Process Monitoring and Management Module

[0430] The server monitors the progress of automated business processes in real time and immediately generates an alert if an anomaly is detected. Users can monitor the process progress and anomaly information in real time.

[0431] 4. Feedback Collection and Analysis Module

[0432] The server collects result data from automated business processes and user feedback, and stores it in a database. The stored data is then analyzed using generative AI models and machine learning models to extract areas for improvement and patterns. The analysis results are provided to the user and incorporated into the design of new automated processes.

[0433] Specific examples and prompt statements

[0434] For robots performing welding tasks within a factory, the user inputs data to optimize the welding pattern. Data such as weld thickness, defect rate, and speed are collected, and if an AI model predicts an anomaly, it immediately alerts the administrator. The following is an example of a user prompt:

[0435] Example of a prompt:

[0436] "Username: user1, Password: pass123, Email: user1@example.com"

[0437] "Task name: Welding, Target: Part A, Schedule time: 1627884000, Parameters: {'Thickness': 0.5, 'Speed': 8}"

[0438] "Thickness: 0.7, Speed: 10"

[0439] By using this system, robot management and automation of work tasks within the factory can be carried out smoothly.

[0440] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0441] Step 1:

[0442] The user enters authentication information using a terminal. The terminal sends this information (including username, password, and email address) to the server. The server stores the received authentication information in its database and sends a confirmation link to the email address. In this step, the input is the user's authentication information, and the output is an email containing the confirmation link.

[0443] Step 2:

[0444] The user clicks a verification link in an email they received. The device sends a request to the server based on that click. The server receives the request and updates the user's authentication status in the database. The input for this step is the click of the verification link, and the output is the updated authentication status.

[0445] Step 3:

[0446] The user enters their username and password on the authentication screen. The terminal sends this information to the server, which then verifies it against the database information to perform authentication. If authentication is successful, the user receives a login notification. The input for this step is the username and password, and the output is the authentication result.

[0447] Step 4:

[0448] An authenticated user inputs the configuration information for a business task. The terminal sends this configuration information to the server, which uses a generated AI model to create an automated process for the task. In this step, the input is the task configuration information, and the output is the generated automated process.

[0449] Step 5:

[0450] The server monitors the progress of the generated automated process in real time. If an anomaly is detected, the server generates an alert and notifies the user via the terminal. The input for this step is the ongoing process data, and the output is the anomaly detection alert.

[0451] Step 6:

[0452] Users can check process progress and anomaly information in real time through the dashboard. The terminal displays this information visually, allowing users to understand the situation. The input for this step is process progress data and anomaly information, and the output is visual information on the dashboard.

[0453] Step 7:

[0454] Once an automated business process is complete, the server collects result data and user feedback and stores it in a database. The collected data is analyzed using generating AI models and machine learning models to extract areas for improvement and patterns. The input for this step is result data and feedback, and the output is the analysis results.

[0455] Step 8:

[0456] The server provides the user with extracted areas for improvement and patterns, allowing them to incorporate them into the design of new automation processes. The user readjusts the task settings based on the analysis results and executes the optimized process. The input for this step is the analysis results, and the output is the optimized task settings.

[0457] The above steps enable efficient management of factory robots and automation of operational tasks.

[0458] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0459] This invention relates to a system that combines an emotion engine for recognizing user emotions with an AI-powered system that efficiently automates tasks and operates with minimal resources. This system includes modules for account management, automation of business tasks, process monitoring and management, feedback collection and analysis, and emotion recognition. The specific processing of each module and its implementation examples are described below.

[0460] Account Management Module

[0461] User registration and authentication

[0462] The user uses a device to access the system and enter their account registration information. The device sends the entered information, such as username, password, and email address, to the server. The server stores the received information in its database, generates a verification link, and sends it to the registered email address. When the user clicks the verification link in the email, the device sends the request to the server, which receives it, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, the device sends this information to the server, and the server authenticates them by comparing it with the database.

[0463] Business task automation module

[0464] Setting up and automating business tasks

[0465] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model. The server trains the AI ​​model based on the task information and generates an automated process. The server sends the progress and results of the generated automated process to the terminal, which then provides the user with a dashboard that visualizes the process's progress and results.

[0466] Process monitoring and management module

[0467] Progress monitoring and anomaly detection

[0468] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. The server detects data that deviates from configured parameters and baseline values, and generates an alert when an anomaly is found. The terminal notifies the user of this alert information, and the user can check the progress and problems in real time on the dashboard screen.

[0469] Feedback collection and analysis module

[0470] Data collection and analysis

[0471] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server analyzes the stored data using AI / machine learning models to extract areas for improvement and patterns. The terminal provides the analysis results to the user, who then incorporates them into the design of new automated processes.

[0472] Emotion Engine Module

[0473] Emotion recognition and feedback

[0474] The server is equipped with an emotion engine that recognizes user emotions. When a user sets up a work task or inputs feedback for an automated process, the terminal simultaneously collects the user's emotional data and sends it to the server. The server analyzes this data using the emotion engine to understand the user's emotional state. For example, if a user is feeling stressed, the system uses this information to provide specific feedback to improve the user experience. It also extracts areas for improvement in the automated process based on the emotional data, aiming to improve the overall efficiency of the system.

[0475] External sales compatible module

[0476] Service customization and delivery

[0477] The server generates a standard service package for external sales based on accumulated know-how and sentiment data. Users customize the details of the external service on the settings screen, and the terminal sends this information to the server. The server reflects the settings and generates a new service package. This package is released, and while supporting users in use, the terminal collects feedback from users and sends it to the server.

[0478] Metaverse / Digital Twin Integration Module

[0479] Data transformation and model generation

[0480] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. Based on the converted data, the server generates a model for the metaverse or digital twin, and the terminal provides the user with a preview of the model. The user starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback, including sentiment data.

[0481] As a concrete example, when a user automates the creation of sales reports, they input business tasks and emotional feedback on a settings screen. The system analyzes this data to generate the optimal automation process and monitors the user's emotional state. If the user experiences stress, the server notifies them of appropriate suggestions and areas for improvement, and takes measures to reduce the user's burden. In this way, the system achieves efficient business automation and improved user experience with minimal resources.

[0482] The following describes the processing flow.

[0483] Account Management Module

[0484] User registration and authentication

[0485] Step 1:

[0486] The user enters their account registration information (username, password, email address).

[0487] Step 2:

[0488] The terminal sends the entered information to the server.

[0489] Step 3:

[0490] The server saves the received information to a database, generates a confirmation link, and sends it to the registered email address.

[0491] Step 4:

[0492] The user clicks the confirmation link sent to their email address.

[0493] Step 5:

[0494] The device sends a request to the server based on clicking the verification link.

[0495] Step 6:

[0496] The server receives the request and updates the authentication status. It then sends a notification to the user that they can log in.

[0497] Step 7:

[0498] The user enters their username and password on the login screen.

[0499] Step 8:

[0500] The terminal sends the entered information to the server.

[0501] Step 9:

[0502] The server verifies the information against the database and performs authentication. If authentication is successful, the user is provided with a dashboard screen.

[0503] Business task automation module

[0504] Setting up and automating business tasks

[0505] Step 1:

[0506] The user enters the business tasks they want to automate on the settings screen.

[0507] Step 2:

[0508] The terminal sends the configured work task information to the server.

[0509] Step 3:

[0510] The server analyzes the task information it receives and selects the appropriate AI model.

[0511] Step 4:

[0512] The server trains an AI model based on task information and generates an automated process.

[0513] Step 5:

[0514] The server sends the progress and results of the automated process it generates to the terminal.

[0515] Step 6:

[0516] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[0517] Process monitoring and management module

[0518] Progress monitoring and anomaly detection

[0519] Step 1:

[0520] The server monitors the progress of automated business processes in real time and records logs.

[0521] Step 2:

[0522] The database stores monitoring data and works in conjunction with anomaly detection algorithms.

[0523] Step 3:

[0524] The server detects data that deviates from the configured parameters and baseline values, and generates an alert when an anomaly is detected.

[0525] Step 4:

[0526] The terminal provides alert information to the user through an interface.

[0527] Step 5:

[0528] Users can check progress and issues in real time on the dashboard screen.

[0529] Feedback collection and analysis module

[0530] Data collection and analysis

[0531] Step 1:

[0532] The server continuously collects result data from automated processes and feedback from users.

[0533] Step 2:

[0534] The database systematically stores all collected data.

[0535] Step 3:

[0536] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns.

[0537] Step 4:

[0538] The terminal visualizes the analysis results through an interface that provides them to the user.

[0539] Step 5:

[0540] Users review the analysis results and incorporate them into the design of new automated processes.

[0541] Emotion Engine Module

[0542] Emotion recognition and feedback

[0543] Step 1:

[0544] When users set up work tasks and input feedback, they will also input emotional data.

[0545] Step 2:

[0546] The device sends user emotion data to the server.

[0547] Step 3:

[0548] The server analyzes emotional data using an emotion engine to understand the user's emotional state.

[0549] Step 4:

[0550] The server provides specific feedback based on sentiment analysis to improve the user experience.

[0551] Step 5:

[0552] For example, if a user is experiencing stress, the server will notify them with suggestions on how to address it.

[0553] Step 6:

[0554] Based on emotional data, the server identifies areas for improvement in automated processes, aiming to enhance the overall efficiency of the system.

[0555] External sales compatible module

[0556] Service customization and delivery

[0557] Step 1:

[0558] The server generates standard service packages for external sales based on accumulated know-how and emotional data.

[0559] Step 2:

[0560] Users can customize the details of external sales services on the settings screen.

[0561] Step 3:

[0562] The device sends customized configuration information to the server.

[0563] Step 4:

[0564] The server reflects the configuration and generates a new service package.

[0565] Step 5:

[0566] The server releases customized service packages and supports users in use.

[0567] Step 6:

[0568] The device collects feedback from users while they are using it and sends it to the server.

[0569] Metaverse / Digital Twin Integration Module

[0570] Data transformation and model generation

[0571] Step 1:

[0572] The server converts business data into formats for the metaverse or digital twin.

[0573] Step 2:

[0574] The database stores the converted data.

[0575] Step 3:

[0576] The server generates metaverse and digital twin models based on the converted data.

[0577] Step 4:

[0578] The device provides the user with a preview of the model.

[0579] Step 5:

[0580] Users initiate simulations within the metaverse or digital twin to verify business processes.

[0581] Step 6:

[0582] The terminal displays and provides the user with simulation results in real time.

[0583] Step 7:

[0584] The server analyzes the simulation results and accumulates feedback.

[0585] (Example 2)

[0586] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0587] Conventional business automation systems lack means of providing feedback and process improvement based on the user's emotional state. Therefore, while operational efficiency improves, there is a risk of user stress and dissatisfaction accumulating. Furthermore, anomaly detection and process progress monitoring are often insufficient, making it difficult to find optimal improvement measures. This invention aims to solve these problems, improve the user experience, and maximize operational efficiency.

[0588] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[0589] In this invention, the server includes means for receiving account registration information from a user and storing it in a database; means for sending a confirmation link to an email address based on the account registration information; means for receiving a click of the confirmation link from a user and updating the authentication status; means for receiving a username and password entered by the user from the login screen and authenticating by comparing them with the information in the database; means for inputting user emotion data and recognizing the user's emotional state; means for providing feedback to improve the user experience based on the recognized emotion data; means for receiving business task setting information and generating an automated process using an AI model; means for monitoring the progress of the generated automated process in real time and detecting anomalies; means for providing a dashboard to notify the user of the process progress and anomaly information; means for extracting areas for improvement in the automated process based on emotion data; means for collecting result data of the automated business process and feedback from users and storing it in a database; means for analyzing the stored data using an AI / machine learning model and extracting areas for improvement and patterns; means for providing the analysis results to the user and reflecting them in the design of a new automated process; and means for providing feedback based on the user's emotion data.

[0590] This enables business process automation that takes into account the user's emotional state, simultaneously achieving efficient business processing and improved user experience. Furthermore, real-time monitoring and anomaly detection optimize business processes, allowing for maximum results with minimal resources.

[0591] "Account registration information" refers to information necessary for user identification and authentication, and specifically includes username, password, email address, etc.

[0592] A "database" is a system or device used to store and manage various types of data, such as account registration information, work task information, results data, and feedback.

[0593] A "verification link" is a link sent to confirm the accuracy of your email address and to authenticate your account registration.

[0594] "Authentication status" refers to the information indicating whether a user has legitimate qualifications, and it is updated by clicking on confirmation links, etc.

[0595] "Emotional data" refers to data that represents the user's psychological state and is information that is analyzed through the emotion engine.

[0596] "Feedback" refers to information such as opinions, impressions, and evaluations from users, which is used for system improvement and process optimization.

[0597] A "business task" refers to the specific tasks and work content that a user sets out to perform in order to carry out their work.

[0598] An "AI model" refers to an artificial intelligence model used for purposes such as data analysis and business process automation, and is built based on machine learning algorithms.

[0599] An "automation process" is a series of processes or flows that automatically execute business tasks generated and configured by an AI model.

[0600] "Real-time monitoring" is a function that allows a system to instantly monitor ongoing business processes and understand their status.

[0601] "Anomaly detection" is a function that detects data or events that deviate from set standards or parameters, and notifies of problems early.

[0602] A "dashboard" is an information screen that allows users to visually check and manage the current status, progress, and any anomalies of a system.

[0603] An "AI / machine learning model" is an algorithm or model used for pattern learning and prediction based on large amounts of data, and is used for system analysis and optimization.

[0604] "Areas for improvement" refer to problems or areas for efficiency improvement in business processes and systems, and are identified through analysis results.

[0605] A "pattern" refers to a certain regularity or trend found within data, which is extracted through analysis.

[0606] A "new automation process" is an updated automation method that improves upon existing processes and enables more efficient work execution.

[0607] This invention relates to a system that combines efficient automation of tasks using AI with user emotion recognition. This system comprises modules for account management, task automation, process monitoring and management, feedback collection and analysis, and emotion recognition. The specific processing of each module and its implementation examples are described below.

[0608] Account Management Module

[0609] User registration and authentication

[0610] The user uses a device to access the system and enters account registration information such as username, password, and email address. The device sends this information to the server. The server stores the received information in its database, generates a verification link, and sends it to the registered email address. When the user clicks the verification link in the email, the device sends a request to the server. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server. The server performs authentication by comparing it with the database.

[0611] Business task automation module

[0612] Setting up and automating business tasks

[0613] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server. The server analyzes the received task information and selects an appropriate AI model. The server trains the AI ​​model based on the task information and generates an automated process. The server sends the progress and results of the generated automated process to the terminal, which then visualizes and presents this information to the user.

[0614] Process monitoring and management module

[0615] Progress monitoring and anomaly detection

[0616] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. When the server detects data that deviates from configured parameters or baseline values, it identifies anomalies and generates alerts. The terminal notifies the user of this alert information, and the user can check the progress and problems in real time on the dashboard screen.

[0617] Feedback collection and analysis module

[0618] Data collection and analysis

[0619] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server analyzes the stored data using AI / machine learning models to extract areas for improvement and patterns. The terminal provides the analysis results to the user, who then incorporates them into the design of new automated processes.

[0620] Emotion Engine Module

[0621] Emotion recognition and feedback

[0622] The server is equipped with an emotion engine that recognizes user emotions. When a user inputs feedback on setting up work tasks or automating processes, the terminal simultaneously collects user emotion data and sends it to the server. The server analyzes this data using the emotion engine to understand the user's emotional state. For example, if a user is feeling stressed, the system uses this information to provide specific feedback to improve the user experience. It also extracts areas for improvement in automated processes based on the emotion data, aiming to improve the overall efficiency of the system.

[0623] Specific example

[0624] For example, when a user automates the creation of sales reports, they enter the work tasks and emotional feedback in the settings screen. The system analyzes this data to generate the optimal automation process and monitors the user's emotional state. If the user is stressed, the server notifies them with appropriate suggestions and areas for improvement. An example of a specific prompt message is: "Please enter the settings for automating the creation of sales reports. Next, please select your recent emotional state from the following options: (1) Relaxed (2) Tired (3) Stressed."

[0625] This allows the system to efficiently automate tasks while considering the user's emotional state, thereby improving the user experience. Furthermore, real-time monitoring and anomaly detection optimize business processes, enabling maximum results with minimal resources.

[0626] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0627] Step 1:

[0628] The user enters their account registration information.

[0629] The user uses a terminal to access the system and enters their username, password, and email address. This becomes the input data. The terminal collects this data, formats it, and sends it to the server. The output is the transmission of account registration information to the server.

[0630] Step 2:

[0631] The device sends registration information to the server.

[0632] The terminal sends the account registration information entered by the user to the server. The server analyzes the received data and saves it to a database. This registers the user's account information in the system. The output is the account information stored in the database.

[0633] Step 3:

[0634] The server generates and sends a verification link.

[0635] The server generates a confirmation link based on the email address stored in the database and sends a confirmation email containing that link to the user. This is data processing based on the input data. The output is the confirmation email sent to the user.

[0636] Step 4:

[0637] The user clicked the confirmation link.

[0638] The user clicks the verification link in the email. The input indicating the link click is sent to the device. The output is the server sending of the request based on the verification link.

[0639] Step 5:

[0640] The device sends the request to the server.

[0641] The device sends a request to the server for the verification link clicked by the user. The server receives this request and updates the user's authentication status. The output is the updated authentication status.

[0642] Step 6:

[0643] The user enters their login information.

[0644] The user enters their username and password on the login screen and sends them to the terminal. This becomes the input data for login. The output is the transmission of the username and password from the terminal to the server.

[0645] Step 7:

[0646] The device sends login information to the server.

[0647] The terminal sends login information to the server, which then authenticates by comparing it against the database. The input is the received username and password, and the output is the authentication result.

[0648] Step 8:

[0649] Users set up work tasks.

[0650] The user uses a terminal to enter the business tasks they want to automate on the settings screen. This setting information becomes the input data. The output is sending this setting information to the server.

[0651] Step 9:

[0652] The device sends configuration information to the server.

[0653] The terminal sends user configuration information to the server. The server analyzes the received information and selects an appropriate AI model. The output is the selected AI model.

[0654] Step 10:

[0655] The server trains the AI ​​model.

[0656] The server trains an AI model based on task information. This process utilizes data processing and machine learning algorithms. The output is the model training result.

[0657] Step 11:

[0658] The server generates an automated process.

[0659] The server generates automated processes using a pre-trained AI model. The input is the training results and task information, and the output is the automated process.

[0660] Step 12:

[0661] The server monitors the progress in real time.

[0662] The server monitors the progress of automated business processes in real time and records logs. Input is progress data, and output is the log records.

[0663] Step 13:

[0664] The server detected an anomaly.

[0665] The server detects data that deviates from the configured parameters and baseline values, and identifies anomalies. Input is real-time progress data, and output is anomaly detection alerts.

[0666] Step 14:

[0667] The device notifies the user of alert information.

[0668] The terminal notifies the user of alert information received from the server. The input is the alert data from the server, and the output is the notification to the user.

[0669] Step 15:

[0670] Users can view this on the dashboard.

[0671] Users can check progress and issues on the dashboard. This allows users to understand the status of their work in real time. The output is the user's understanding and action.

[0672] Step 16:

[0673] The server collects result data and feedback.

[0674] The server collects result data from automated processes and user feedback, and stores it in a database. The input is result data and feedback, and the output is the stored database.

[0675] Step 17:

[0676] The server analyzes the data using AI / machine learning models.

[0677] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns. The input is the accumulated data, and the output is the analysis results.

[0678] Step 18:

[0679] The device provides the user with the analysis results.

[0680] The terminal provides the user with analysis results. The input is analysis result data from the server, and the output is the analysis information obtained by the user.

[0681] Step 19:

[0682] Users design new automation processes.

[0683] The user designs a new automated process based on the analysis results, further improving operational efficiency. The output is the new automated process.

[0684] Step 20:

[0685] Users enter emotional data

[0686] Users input their emotional data when setting up work tasks or providing feedback. This input represents data about the user's psychological state.

[0687] Step 21:

[0688] The device sends emotional data to the server.

[0689] The device sends the collected emotional data to the server. The input is emotional data from the user, and the output is data sent to the server.

[0690] Step 22:

[0691] The server performs emotion recognition and analysis.

[0692] The server uses an emotion engine to analyze emotional data and understand the user's emotional state. The input is emotional data, and the output is the analysis result.

[0693] Step 23:

[0694] The server provides feedback

[0695] The server provides feedback to improve the user experience based on the analysis results. The input is the sentiment analysis result, and the output is the specific feedback content.

[0696] Step 24:

[0697] The server extracts areas for process improvement based on emotional data.

[0698] The server extracts areas for improvement in the automated process based on emotional data. The input is emotional data and analysis results, and the output is the extracted areas for improvement.

[0699] (Application Example 2)

[0700] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0701] Conventional business automation systems often fail to consider the emotional state of users, leading to user burden and limiting improvements in operational efficiency. Furthermore, they lacked sufficient mechanisms for real-time collection and analysis of customer feedback, making immediate responses necessary for business improvement difficult. Additionally, real-time monitoring of business task progress and immediate responses to anomalies were inadequate. Solving these challenges was essential.

[0702] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving account registration information from a user and storing it in a database, means for sending a confirmation link to an email address based on the account registration information, means for receiving a click of the confirmation link from a user and updating the authentication status, means for receiving the username and password entered by the user from the login screen and authenticating them by comparing them with the information in the database, means for recognizing and analyzing the emotional state of staff, means for providing appropriate feedback and support based on the emotional state of staff, and means for monitoring the progress of automated work tasks in real time and detecting anomalies. This makes it possible to automate tasks while taking into account the emotional state of the user, thereby reducing the burden on staff and achieving improved work efficiency and immediate problem solving.

[0703] "Account registration information" refers to information such as the username, password, and email address that a user provides to access the system.

[0704] A "database" is a system that stores saved account registration information and business task data, and allows for searching and updating of that information.

[0705] A "confirmation link" is a URL sent via email to users to verify the validity of their registration information.

[0706] "Authentication status" is data that indicates the authentication status when a user accesses the system.

[0707] A "username" is a string of characters used to uniquely identify a user when accessing the system.

[0708] A "password" is confidential information used for security authentication when accessing a system.

[0709] "Staff" refers to individuals who perform duties at a physical store.

[0710] "Emotional state" refers to data that indicates the current psychological and emotional condition of a user or staff member.

[0711] "Feedback" refers to information collected from users and customers, such as opinions and evaluations, to help improve the system.

[0712] "Real-time" means that data is updated and processed almost instantly.

[0713] A "dashboard" is a screen that visually displays important information within a user interface.

[0714] An "AI model" is a mathematical model that uses artificial intelligence technology to automate and optimize specific tasks.

[0715] An "abnormal" refers to data or a situation that deviates from established standards or normal patterns.

[0716] A "business task" is a specific work item that requires automation and monitoring in order to perform business efficiently.

[0717] This invention is a system that combines an AI model and emotion recognition technology, primarily to improve operational efficiency in physical stores. Embodiments of this system are described below.

[0718] overview

[0719] This system is designed to reduce the burden on store staff and streamline operations. Using smart glasses, the system allows staff to monitor and manage work processes in real time. It also enables rapid collection of customer feedback and its use in improving operations. Furthermore, by recognizing staff emotional states and providing appropriate support when stressed, it improves employee satisfaction and work efficiency.

[0720] Program Overview

[0721] 1. Account Management:

[0722] Staff members use smart glasses to log into their accounts and check shift information and other details.

[0723] 2. Automation of business tasks:

[0724] Automate business tasks such as inventory management, checking displayed merchandise, and scheduling cleaning.

[0725] 3. Monitoring process progress:

[0726] It monitors the progress of tasks in real time and sends alerts if any anomalies occur.

[0727] 4. Feedback collection and analysis:

[0728] We collect customer feedback, analyze it using AI and machine learning, and identify areas for improvement.

[0729] 5. Emotion recognition:

[0730] Recognize the emotional state of staff and provide support, such as suggesting breaks if they are feeling stressed.

[0731] Hardware and software to be used

[0732] Hardware: Smart glasses (e.g., Google Glass®), servers, databases

[0733] Software: Emotion recognition library, task automation library, feedback collection library, dashboard display software

[0734] AI Models: AI / Machine Learning Models for Task Automation and Feedback Analysis

[0735] Processing flow

[0736] This system begins with staff logging into their accounts using smart glasses. Next, an AI model analyzes task information to automate work tasks and monitors progress in real time. Staff emotional states are analyzed by an emotion recognition library, and if stress is detected, an alert recommending a break is sent to the smart glasses. Customer feedback is collected in real time and analyzed by the AI ​​model. This results in specific operational improvement suggestions being presented on a dashboard.

[0737] Specific example

[0738] For example, a store employee arrives for work and logs in using smart glasses. When this employee enters an inventory check task into the system, the system automatically begins checking inventory. If the emotion recognition library detects the employee's emotional state as "stressed" during the process, the smart glasses suggest a break. Additionally, when a customer provides feedback about a product, the system analyzes the feedback in real time and displays suggestions for improving the display method on a dashboard.

[0739] Examples of prompts for generative AI models

[0740] Perform the following emotion recognition to detect the stress levels of your store staff:

[0741] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0742] Step 1:

[0743] The user puts on smart glasses and enters their account registration information (username, password). The device sends this information to the server. The server retrieves the corresponding user information from the database and performs authentication. If authentication is successful, a login success message is sent back to the device, and the user can log in to the system.

[0744] Step 2:

[0745] The user inputs configuration information for a business task (e.g., inventory management) through smart glasses. The device sends this task information to a server. The server analyzes the received task information, selects and trains an appropriate AI model, and generates an automated process. Once this automated process is generated, its progress and results are sent from the server to the device and presented to the user visually.

[0746] Step 3:

[0747] The server monitors the progress of the generated automated processes in real time. Progress data is sent to the server, and an anomaly detection algorithm detects data that deviates from the standard values. When an anomaly is detected, the server generates an alert and sends it to the terminal. The terminal notifies the user of the alert information and displays the details of the anomaly and countermeasures on the dashboard.

[0748] Step 4:

[0749] The user sends feedback (e.g., customer opinions) through smart glasses. The device sends the feedback data to a server. The server collects this feedback data and stores it in a database. The stored data is analyzed using AI and machine learning models to extract areas for improvement and patterns. The analysis results are sent from the server to the device and provided to the user.

[0750] Step 5:

[0751] The server uses an emotion recognition library to analyze the user's emotional state. The device sends data acquired from the camera and microphone to the server, which analyzes this data to recognize the user's emotional state. If "stress" is detected, the server generates appropriate feedback (e.g., a suggestion to take a break) and sends it to the device. The device then notifies the user of this feedback.

[0752] Step 6:

[0753] Customers provide feedback in-store using smart glasses. The device sends this feedback information to a server. The server analyzes the feedback data and generates improvement suggestions. The generated suggestions are sent to the device and presented to the user visually. This enables rapid business improvements that reflect customer feedback.

[0754] As an example of using a generative AI model, consider the prompt: "Perform the following emotion recognition to detect the stress level of store staff:" Based on this prompt, the AI ​​model performs emotion analysis.

[0755] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0756] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0757] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0758] [Second Embodiment]

[0759] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0760] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0761] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0762] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0763] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0764] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0765] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0766] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0767] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0768] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0769] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0770] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0771] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, and feedback collection and analysis. The specific processing of each module and its implementation examples are described below.

[0772] Account Management Module

[0773] User registration and authentication

[0774] The user uses a device to access the system and enter their account registration information. The device sends the entered information, such as username, password, and email address, to the server. The server stores the received information in a database and sends a confirmation link to the registered email address. When the user clicks the confirmation link in the email, the device sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server, which then authenticates them by comparing it with the information in the database.

[0775] Business task automation module

[0776] Setting up and automating business tasks

[0777] The user enters the business tasks they want to automate (e.g., report creation or data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model. The server then trains the AI ​​model based on the task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[0778] Process monitoring and management module

[0779] Progress monitoring and anomaly detection

[0780] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. The server detects data that deviates from configured parameters and baseline values, and generates an alert when an anomaly is found. The terminal notifies the user of this alert information, allowing them to check progress and problems on a real-time dashboard.

[0781] Feedback collection and analysis module

[0782] Data collection and analysis

[0783] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server then analyzes this data using AI / machine learning models to extract areas for improvement and patterns. The analysis results are then provided to users via their terminals, allowing them to incorporate them into the design of new automated processes.

[0784] External sales compatible module

[0785] Service customization and delivery

[0786] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. The customized service package is released, and support is provided to users during its use. The terminal collects feedback from users and sends it to the server.

[0787] Metaverse / Digital Twin Integration Module

[0788] Data transformation and model generation

[0789] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. Based on the converted data, the server generates a metaverse or digital twin model, and the terminal provides the user with a preview of the model. The user then starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[0790] As described above, each module works in conjunction with others to enable efficient automation, management, analysis, and feedback collection and sharing of operations, providing an environment in which companies can operate with minimal resources.

[0791] The following describes the processing flow.

[0792] Account Management Module

[0793] User registration and authentication

[0794] Step 1:

[0795] The user enters their account registration information (username, password, email address).

[0796] Step 2:

[0797] The terminal sends the entered information to the server.

[0798] Step 3:

[0799] The server saves the received information to a database, generates a confirmation link, and sends it to the registered email address.

[0800] Step 4:

[0801] The user clicks the confirmation link sent to their email address.

[0802] Step 5:

[0803] The device sends a request to the server based on clicking the verification link.

[0804] Step 6:

[0805] The server receives the request and updates the authentication status. It then sends a notification to the user that they can log in.

[0806] Step 7:

[0807] The user enters their username and password on the login screen.

[0808] Step 8:

[0809] The terminal sends the entered information to the server.

[0810] Step 9:

[0811] The server verifies the information against the database and performs authentication. If authentication is successful, the user is provided with a dashboard screen.

[0812] Business task automation module

[0813] Setting up and automating business tasks

[0814] Step 1:

[0815] The user enters the business tasks they want to automate on the settings screen.

[0816] Step 2:

[0817] The terminal sends the configured work task information to the server.

[0818] Step 3:

[0819] The server analyzes the task information it receives and selects the appropriate AI model.

[0820] Step 4:

[0821] The server trains an AI model based on task information and generates an automated process.

[0822] Step 5:

[0823] The server sends the progress and results of the automated process it generates to the terminal.

[0824] Step 6:

[0825] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[0826] Process monitoring and management module

[0827] Progress monitoring and anomaly detection

[0828] Step 1:

[0829] The server monitors the progress of automated business processes in real time and records logs.

[0830] Step 2:

[0831] The database stores monitoring data and works in conjunction with anomaly detection algorithms.

[0832] Step 3:

[0833] The server detects data that deviates from the configured parameters and baseline values, and generates an alert when an anomaly is detected.

[0834] Step 4:

[0835] The terminal provides alert information to the user through an interface.

[0836] Step 5:

[0837] Users can check progress and issues in real time on the dashboard screen.

[0838] Feedback collection and analysis module

[0839] Data collection and analysis

[0840] Step 1:

[0841] The server continuously collects result data from automated processes and feedback from users.

[0842] Step 2:

[0843] The database systematically stores all collected data.

[0844] Step 3:

[0845] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns.

[0846] Step 4:

[0847] The terminal visualizes the analysis results through an interface that provides them to the user.

[0848] Step 5:

[0849] Users review the analysis results and incorporate them into the design of new automated processes.

[0850] External sales compatible module

[0851] Service customization and delivery

[0852] Step 1:

[0853] The server generates standard service packages for external sales based on accumulated know-how.

[0854] Step 2:

[0855] Users can customize the details of external sales services on the settings screen.

[0856] Step 3:

[0857] The device sends customized configuration information to the server.

[0858] Step 4:

[0859] The server reflects the configuration and generates a new service package.

[0860] Step 5:

[0861] The server releases customized service packages and supports users in use.

[0862] Step 6:

[0863] The device collects feedback from users while they are using it and sends it to the server.

[0864] Metaverse / Digital Twin Integration Module

[0865] Data transformation and model generation

[0866] Step 1:

[0867] The server converts business data into formats for the metaverse or digital twin.

[0868] Step 2:

[0869] The database stores the converted data.

[0870] Step 3:

[0871] The server generates metaverse and digital twin models based on the converted data.

[0872] Step 4:

[0873] The device provides the user with a preview of the model.

[0874] Step 5:

[0875] Users initiate simulations within the metaverse or digital twin to verify business processes.

[0876] Step 6:

[0877] The terminal displays and provides the user with simulation results in real time.

[0878] Step 7:

[0879] The server analyzes the simulation results and accumulates feedback.

[0880] (Example 1)

[0881] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0882] Traditional systems relied on manual processes for automating, monitoring, and analyzing business processes, leading to decreased operational efficiency. Furthermore, effectively collecting and analyzing user feedback and incorporating it into new process design proved difficult. Additionally, the limited use of virtual spaces and digital replication hindered the easy simulation and verification of business processes.

[0883] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0884] In this invention, the server includes means for receiving account registration information from a user and storing it in a database; means for sending a confirmation link to an email address based on the account registration information; means for receiving a click of the confirmation link from the user and updating the authentication status; means for converting business data into a virtual space or digital replica format and storing it on an analysis platform; means for generating a virtual space or digital replica model based on the converted data; and means for providing the user with a preview of the generated model and initiating a simulation. This enables automation and efficiency improvements of business operations, real-time monitoring and anomaly detection, and effective collection and analysis of user feedback.

[0885] A "user" refers to anyone who uses this system to perform operations such as registering an account, setting up work tasks, logging in, and providing feedback.

[0886] A "server" refers to a computer system that receives requests from users, stores information in a database, and performs overall system processing, including training and running AI models, generating automated processes, monitoring progress, detecting anomalies, and generating models for virtual spaces and digital replicas.

[0887] "Database" refers to a digital storage system for permanently storing various types of data required by this system, such as account registration information, work task information, progress data, and feedback data.

[0888] A "verification link" refers to a temporary URL sent to the user's email address, which the user clicks to authenticate their account.

[0889] "Authentication status" refers to status information indicating whether the user has clicked the verification link and completed account authentication.

[0890] "Business tasks" refer to business processes, such as report creation and data entry, that users set up in this system.

[0891] A "generative AI model" refers to an artificial intelligence model trained to generate appropriate automation processes based on the user's business task information.

[0892] A "prompt" refers to the text information or question sentences that are input into a generative AI model.

[0893] "Anomaly detection" refers to the process of monitoring the progress of automated business processes and identifying data that deviates from set parameters or baseline values.

[0894] A "virtual space" refers to a digital environment that digitally reproduces the actual physical space for conducting simulations and model verification.

[0895] "Digital replication" refers to a virtual model that reproduces actual business processes and environments as digital data and is used for simulation and verification.

[0896] "Visualized display methods" refer to display methods that use dashboards, graphs, and charts to allow users to intuitively understand the progress of a process and any anomaly information.

[0897] "Analysis techniques" refer to algorithms and methods for analyzing large amounts of data stored in a database and extracting patterns and areas for improvement.

[0898] The above are the key terms and their definitions included in the claims of this system.

[0899] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, feedback collection and analysis, external sales support, and metaverse / digital twin integration. The specific operation of each module is described in detail below.

[0900] Account Management Module

[0901] User registration and authentication

[0902] The user accesses the system using a device and enters account registration information such as username, password, and email address. The device sends this information to the server, which stores the received information in a database (e.g., MySQL). The server then sends a verification link to the registered email address (e.g., using an email service). When the user clicks the verification link, the device sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server. The server authenticates the user by comparing it with the information in the database.

[0903] Business task automation module

[0904] Setting up and automating business tasks

[0905] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model (e.g., TensorFlow). The server then trains the AI ​​model based on this task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[0906] Process monitoring and management module

[0907] Progress monitoring and anomaly detection

[0908] The server monitors the progress of automated business processes in real time and logs the data (e.g., using a monitoring tool). The database stores the monitoring data and works with anomaly detection algorithms (e.g., Scikit-learn). The server detects data that deviates from configured parameters or baseline values ​​and generates an alert when an anomaly is found. The terminal notifies the user of this alert information and allows them to check progress and problems on a real-time dashboard.

[0909] Feedback collection and analysis module

[0910] Data collection and analysis

[0911] The server continuously collects result data from automated processes and user feedback, storing it in a database (e.g., PostgreSQL). The server then analyzes the stored data using an AI / machine learning model (e.g., PyTorch) to extract areas for improvement and patterns. The analysis results are provided to the user via a terminal, allowing the user to incorporate them into the design of new automated processes.

[0912] External sales compatible module

[0913] Service customization and delivery

[0914] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. It releases the customized service package and supports users who are using it. The terminal collects feedback from users and sends it to the server.

[0915] Metaverse / Digital Twin Integration Module

[0916] Data transformation and model generation

[0917] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. The server generates a model of the metaverse or digital twin based on the converted data (e.g., using Unity). The terminal provides the user with a preview of the model. The user starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[0918] Examples of specific cases and prompt statements

[0919] For example, a "mail service" can be used to send a confirmation link to a registered email address, and "MySQL" can be used as the database to store user information. Furthermore, "TensorFlow" or "PyTorch" can be used to train AI models. "Visualization tools" can be used to monitor the situation on the dashboard, and "Scikit-learn" can be utilized for anomaly detection. "3D modeling tools" can be used to generate models in virtual spaces or digital replicas.

[0920] Prompt example 1: "Send a confirmation email for new user registration and update the authentication status."

[0921] Prompt example 2: "Use a generative AI model to automate report generation tasks and visualize progress."

[0922] Prompt example 3: "Apply the anomaly detection algorithm and display an alert if an anomaly occurs."

[0923] Prompt example 4: "We will extract patterns through data analysis and identify areas for improvement."

[0924] As described above, each module works together through detailed and specific procedures, providing an environment where companies can efficiently automate their operations and operate with minimal resources.

[0925] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0926] Account Management Module

[0927] User registration and authentication

[0928] Step 1:

[0929] The user accesses the system using a terminal and enters account registration information such as username, password, and email address.

[0930] Input: Username, Password, Email Address

[0931] Output: The request the terminal sends to the server with this information.

[0932] Step 2:

[0933] The device sends the entered account registration information to the server.

[0934] Input: Username, password, and email address entered by the user.

[0935] Output: Data packets to be sent to the server

[0936] Step 3:

[0937] The server parses the information it receives and saves it to a database (for example, MySQL).

[0938] Input: Account registration information sent from the device

[0939] Output: Saving information to the database using SQL INSERT statements

[0940] Specific operation: Execute SQL queries using the Python sqlite3 module or MySQL client.

[0941] Step 4:

[0942] The server will send a confirmation link to the registered email address.

[0943] Input: User's email address

[0944] Output: Email containing a confirmation link

[0945] Specific operation: Use an email service (e.g., SendGrid) to send emails via API.

[0946] Step 5:

[0947] The user clicks the verification link, and the device sends an authentication request to the server.

[0948] Input: Link access via user click

[0949] Output: Request to the server

[0950] Specific action: The device sends an HTTP GET request to the server.

[0951] Step 6:

[0952] The server receives the request and updates the authentication status.

[0953] Input: Authentication request to the server

[0954] Output: Authentication status update

[0955] Specific action: Update the status field of the corresponding user in the database.

[0956] Step 7:

[0957] The user enters their username and password on the login screen, and the device sends that information to the server.

[0958] Input: Username, Password

[0959] Output: Login request to the server

[0960] Specific action: Send data using the terminal's form submission function.

[0961] Step 8:

[0962] The server compares the information with that in the database and performs authentication.

[0963] Input: Username, Password

[0964] Output: Authentication success return code or error message

[0965] Specific operation: Database matching is performed using an SQL SELECT statement.

[0966] ---

[0967] Business task automation module

[0968] Setting up and automating business tasks

[0969] Step 1:

[0970] The user enters the business tasks they want to automate (e.g., report creation, data entry) on the settings screen.

[0971] Input: Detailed information about the task

[0972] Output: The request the terminal sends to the server with this information.

[0973] Step 2:

[0974] The device sends configuration information to the server.

[0975] Input: Detailed information about the task

[0976] Output: Request for configuration information from the server

[0977] Step 3:

[0978] The server analyzes the task information and selects an appropriate AI model (e.g., TensorFlow).

[0979] Input: Detailed information about the task

[0980] Output: Selected AI models

[0981] Specific action: Execute the AI ​​model selection algorithm.

[0982] Step 4:

[0983] The server trains the AI ​​model and generates automated processes.

[0984] Input: Detailed information on the business task, selected AI model

[0985] Output: Trained model, generated automation process

[0986] Specific operation: Train the model using the TensorFlow library.

[0987] Step 5:

[0988] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[0989] Input: Progress data, result data

[0990] Output: Visualized dashboard

[0991] Specific actions: Use visualization tools (e.g., Grafana) to display data in graphs and charts.

[0992] ---

[0993] Process monitoring and management module

[0994] Progress monitoring and anomaly detection

[0995] Step 1:

[0996] The server monitors the progress of automated business processes in real time and records logs.

[0997] Input: Business process progress data

[0998] Output: Real-time monitoring data, recorded logs

[0999] Specific operation: Collect real-time data using a monitoring tool (e.g., Prometheus) and save it as a log.

[1000] Step 2:

[1001] The database stores the monitoring data.

[1002] Input: Real-time monitoring data

[1003] Output: Saved monitoring data

[1004] Specific operation: Execute an SQL INSERT statement into the database and save the data.

[1005] Step 3:

[1006] The server applies an anomaly detection algorithm.

[1007] Input: Stored monitoring data

[1008] Output: Anomaly detection result

[1009] Specific operation: Execute an anomaly detection algorithm using Scikit-learn.

[1010] Step 4:

[1011] The server detects an anomaly and generates an alert.

[1012] Input: Anomaly detection result

[1013] Output: Generated alerts

[1014] Specific action: Generate an alert message and output it in the specified format.

[1015] Step 5:

[1016] The device notifies the user of alert information.

[1017] Input: Generated alert

[1018] Output: Notification to the user

[1019] Specific operation: Uses a GUI to send pop-up notifications and email notifications.

[1020] Step 6:

[1021] The device will be able to check progress and issues in real time on a dashboard.

[1022] Input: Alert information, progress data

[1023] Output: Visualized dashboard

[1024] Specific actions: Display data using a visualization tool (e.g., Grafana).

[1025] ---

[1026] Feedback collection and analysis module

[1027] Data collection and analysis

[1028] Step 1:

[1029] The server continuously collects data from automated processes and user feedback, and stores it in a database.

[1030] Input: Result data, feedback data

[1031] Output: Stored data

[1032] Specific operation: Use the API to collect data and store it in the database.

[1033] Step 2:

[1034] The data accumulated by the server is analyzed using AI / machine learning models.

[1035] Input: Accumulated data

[1036] Output: Analysis results (areas for improvement and patterns)

[1037] Specific operation: Use PyTorch to feed the dataset into the model and perform analysis.

[1038] Step 3:

[1039] The device provides the user with the analysis results.

[1040] Input: Analysis results

[1041] Output: Display of analysis results to the user

[1042] Specific actions: Visualize and display results on the dashboard.

[1043] Step 4:

[1044] Users will incorporate their feedback into the design of new automation processes.

[1045] Input: Analysis results, feedback

[1046] Output: Newly designed automation process

[1047] Specific actions: Use operational design tools to design a new process.

[1048] ---

[1049] External sales compatible module

[1050] Service customization and delivery

[1051] Step 1:

[1052] The server generates the standard service package.

[1053] Input: Accumulated know-how

[1054] Output: Standard service package

[1055] Specific actions: Reading data from the database and executing a template generation script.

[1056] Step 2:

[1057] The user can customize the settings.

[1058] Input: Customization details

[1059] Output: Sending of configuration information by the terminal

[1060] Specific operation: Form input and submission via GUI.

[1061] Step 3:

[1062] The device sends customization information to the server.

[1063] Input: Customization details

[1064] Output: Request to the server

[1065] Specific operation: Send JSON data via an HTTP POST request.

[1066] Step 4:

[1067] The server will reflect the customizations.

[1068] Input: Customization Information

[1069] Output: Customized service package

[1070] Specific action: Generate a new package using a template and save it.

[1071] Step 5:

[1072] The server releases customized service packages and supports users in use.

[1073] Input: Customized package

[1074] Output: Released service packages, support information

[1075] Specific actions: Deploy and support using cloud services such as AWS.

[1076] Step 6:

[1077] The device collects feedback and sends it to the server.

[1078] Input: User feedback

[1079] Output: Sending feedback data to the server

[1080] Specific action: Submit data through the feedback form.

[1081] ---

[1082] Metaverse / Digital Twin Integration Module

[1083] Data transformation and model generation

[1084] Step 1:

[1085] The server converts business data into formats for the metaverse and digital twin.

[1086] Input: Business data

[1087] Output: Converted data

[1088] Specific operation: Executes a data conversion script (e.g., Python or C++).

[1089] Step 2:

[1090] The server saves the converted data to the database.

[1091] Input: Converted data

[1092] Output: Saved data

[1093] Specific action: Execute an SQL INSERT statement into the database.

[1094] Step 3:

[1095] The server generates metaverse and digital twin models based on the converted data.

[1096] Input: Converted data

[1097] Output: Generated model

[1098] Specific actions: Build the model using the Unity API.

[1099] Step 4:

[1100] The device provides the user with a preview of the model.

[1101] Input: Generated model

[1102] Output: Model preview screen

[1103] Specific action: Display the model using the 3D viewer.

[1104] Step 5:

[1105] The user starts a simulation within the metaverse or digital twin.

[1106] Input: Instruction to start simulation

[1107] Output: Run the simulation

[1108] Specific action: Click the Start button to begin the simulation.

[1109] Step 6:

[1110] The device displays the simulation results in real time.

[1111] Input: Simulation result data

[1112] Output: Simulation results displayed in real time

[1113] Specific actions: Display data in dashboards and graphs.

[1114] Step 7:

[1115] The server analyzes the simulation results and accumulates feedback.

[1116] Input: Simulation result data

[1117] Output: Analysis results, feedback

[1118] Specific operation: Analyze the result data and save it as Insights.

[1119] Through the above processing steps, each module is efficiently linked, enabling automation of operations, monitoring of progress, anomaly detection, feedback collection, customized services, and integration with the metaverse and digital twin.

[1120] (Application Example 1)

[1121] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1122] In modern factories, while efficiency and automation are essential, managing robots and setting and monitoring tasks are becoming increasingly complex. The lack of integrated real-time monitoring of work progress, anomaly detection, feedback collection and analysis, and account management for technicians and managers often hinders operational efficiency. Furthermore, the lack of established methods for designing and improving automation processes using generative AI models is slowing down the optimization of operations.

[1123] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1124] In this invention, the server includes means for receiving authentication information from a user and storing it in a database; means for sending a confirmation link via communication means based on the authentication information; means for receiving a click of the confirmation link from a user and updating the authentication status; means for receiving authentication information entered by a user from an authentication screen and performing authentication by comparing it with the information in the database; means for managing accounts of robot operators and administrators and performing access control; means for receiving setting information of business tasks from authenticated users and generating an automated process using a generation AI model; means for monitoring the progress of the generated automated process in real time and detecting anomalies; means for providing a dashboard that notifies the user of the progress of the process and anomaly information; means for collecting result data of the automated business process and feedback from users and storing it in a database; means for analyzing the stored data using a generation AI model and a machine learning model and extracting areas for improvement and patterns; and means for providing the user with the analysis results and reflecting them in the design of new automated processes. As a result, robot management and automation of business tasks in the factory are integrated, enabling improved work efficiency and faster anomaly detection.

[1125] "Authentication information" refers to information used to identify a user and verify their access rights.

[1126] A "database" is an electronic recording device used to efficiently store, search, and manage large amounts of data.

[1127] A "verification link" is a URL that users receive via email or other means and click to authenticate their account.

[1128] "Authentication status" refers to the state of information indicating whether a user has legitimate authority.

[1129] A "robot operator or manager" is a person responsible for operating and managing robots within a factory.

[1130] "Access control" is the process of restricting and managing access rights to information systems.

[1131] "Business tasks" refer to specific tasks or operations that are performed by robots within a factory.

[1132] A "generative AI model" is an artificial intelligence model trained using machine learning algorithms to generate automated processes for business tasks.

[1133] An "automation process" is a set of steps or processes that automatically execute specific tasks or operations.

[1134] "Real-time" refers to the process and display of data and information immediately as soon as it is generated.

[1135] "Anomaly detection" is the process of discovering behavior or errors that deviate from normal operation.

[1136] A "dashboard" is a user interface that visually displays multiple pieces of information, allowing users to check progress and any anomalies at a glance.

[1137] "Feedback" refers to information about opinions and results collected from users and systems.

[1138] "Analysis" is the process of processing and evaluating collected data to extract useful information.

[1139] "Areas for improvement" are parts of a business process that require change to optimize it.

[1140] A "pattern" refers to the regularity or characteristics found within data.

[1141] The system that realizes this application consists of multiple modules and aims to efficiently automate robot management and tasks within a factory. The hardware and software used, as well as specific implementation examples, are described below.

[1142] Hardware and software used

[1143] 1. Server

[1144] Database management systems: MySQL and PostgreSQL

[1145] AI and machine learning frameworks: TensorFlow and PyTorch

[1146] Web frameworks: Flask and Django

[1147] 2. Terminal

[1148] Personal computers, smartphones, tablets, and other devices that users use for input and display.

[1149] 3. User

[1150] Robot operators and managers

[1151] System configuration and details

[1152] 1. Account Management Module

[1153] The server first receives authentication information from the user and stores it in the database. Next, it sends a confirmation link via communication means based on that authentication information. When the user clicks the confirmation link, the server receives the request and updates the authentication status. It receives the authentication information entered by the user on the authentication screen, verifies it against the information in the database, and performs authentication. It also manages accounts for robot operators and administrators and performs access control.

[1154] 2. Business Task Automation Module

[1155] The system receives task configuration information from authenticated users and generates automated processes using a generated AI model. The server monitors this process in real time and detects anomalies. Users are provided with a dashboard that notifies them of process progress and anomaly information. For example, it can easily configure tasks for robots performing welding operations.

[1156] 3. Process Monitoring and Management Module

[1157] The server monitors the progress of automated business processes in real time and immediately generates an alert if an anomaly is detected. Users can monitor the process progress and anomaly information in real time.

[1158] 4. Feedback Collection and Analysis Module

[1159] The server collects result data from automated business processes and user feedback, and stores it in a database. The stored data is then analyzed using generative AI models and machine learning models to extract areas for improvement and patterns. The analysis results are provided to the user and incorporated into the design of new automated processes.

[1160] Specific examples and prompt statements

[1161] For robots performing welding tasks within a factory, the user inputs data to optimize the welding pattern. Data such as weld thickness, defect rate, and speed are collected, and if an AI model predicts an anomaly, it immediately alerts the administrator. The following is an example of a user prompt:

[1162] Example of a prompt:

[1163] "Username: user1, Password: pass123, Email: user1@example.com"

[1164] "Task name: Welding, Target: Part A, Schedule time: 1627884000, Parameters: {'Thickness': 0.5, 'Speed': 8}"

[1165] "Thickness: 0.7, Speed: 10"

[1166] By using this system, robot management and automation of work tasks within the factory can be carried out smoothly.

[1167] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1168] Step 1:

[1169] The user enters authentication information using a terminal. The terminal sends this information (including username, password, and email address) to the server. The server stores the received authentication information in its database and sends a confirmation link to the email address. In this step, the input is the user's authentication information, and the output is an email containing the confirmation link.

[1170] Step 2:

[1171] The user clicks a verification link in an email they received. The device sends a request to the server based on that click. The server receives the request and updates the user's authentication status in the database. The input for this step is the click of the verification link, and the output is the updated authentication status.

[1172] Step 3:

[1173] The user enters their username and password on the authentication screen. The terminal sends this information to the server, which then verifies it against the database information to perform authentication. If authentication is successful, the user receives a login notification. The input for this step is the username and password, and the output is the authentication result.

[1174] Step 4:

[1175] An authenticated user inputs the configuration information for a business task. The terminal sends this configuration information to the server, which uses a generated AI model to create an automated process for the task. In this step, the input is the task configuration information, and the output is the generated automated process.

[1176] Step 5:

[1177] The server monitors the progress of the generated automated process in real time. If an anomaly is detected, the server generates an alert and notifies the user via the terminal. The input for this step is the ongoing process data, and the output is the anomaly detection alert.

[1178] Step 6:

[1179] Users can check process progress and anomaly information in real time through the dashboard. The terminal displays this information visually, allowing users to understand the situation. The input for this step is process progress data and anomaly information, and the output is visual information on the dashboard.

[1180] Step 7:

[1181] Once an automated business process is complete, the server collects result data and user feedback and stores it in a database. The collected data is analyzed using generating AI models and machine learning models to extract areas for improvement and patterns. The input for this step is result data and feedback, and the output is the analysis results.

[1182] Step 8:

[1183] The server provides the user with extracted areas for improvement and patterns, allowing them to incorporate them into the design of new automation processes. The user readjusts the task settings based on the analysis results and executes the optimized process. The input for this step is the analysis results, and the output is the optimized task settings.

[1184] The above steps enable efficient management of factory robots and automation of operational tasks.

[1185] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1186] This invention relates to a system that combines an emotion engine for recognizing user emotions with an AI-powered system that efficiently automates tasks and operates with minimal resources. This system includes modules for account management, automation of business tasks, process monitoring and management, feedback collection and analysis, and emotion recognition. The specific processing of each module and its implementation examples are described below.

[1187] Account Management Module

[1188] User registration and authentication

[1189] The user uses a device to access the system and enter their account registration information. The device sends the entered information, such as username, password, and email address, to the server. The server stores the received information in its database, generates a verification link, and sends it to the registered email address. When the user clicks the verification link in the email, the device sends the request to the server, which receives it, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, the device sends this information to the server, and the server authenticates them by comparing it with the database.

[1190] Business task automation module

[1191] Setting up and automating business tasks

[1192] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model. The server trains the AI ​​model based on the task information and generates an automated process. The server sends the progress and results of the generated automated process to the terminal, which then provides the user with a dashboard that visualizes the process's progress and results.

[1193] Process monitoring and management module

[1194] Progress monitoring and anomaly detection

[1195] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. The server detects data that deviates from configured parameters and baseline values, and generates an alert when an anomaly is found. The terminal notifies the user of this alert information, and the user can check the progress and problems in real time on the dashboard screen.

[1196] Feedback collection and analysis module

[1197] Data collection and analysis

[1198] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server analyzes the stored data using AI / machine learning models to extract areas for improvement and patterns. The terminal provides the analysis results to the user, who then incorporates them into the design of new automated processes.

[1199] Emotion Engine Module

[1200] Emotion recognition and feedback

[1201] The server is equipped with an emotion engine that recognizes user emotions. When a user sets up a work task or inputs feedback for an automated process, the terminal simultaneously collects the user's emotional data and sends it to the server. The server analyzes this data using the emotion engine to understand the user's emotional state. For example, if a user is feeling stressed, the system uses this information to provide specific feedback to improve the user experience. It also extracts areas for improvement in the automated process based on the emotional data, aiming to improve the overall efficiency of the system.

[1202] External sales compatible module

[1203] Service customization and delivery

[1204] The server generates a standard service package for external sales based on accumulated know-how and sentiment data. Users customize the details of the external service on the settings screen, and the terminal sends this information to the server. The server reflects the settings and generates a new service package. This package is released, and while supporting users in use, the terminal collects feedback from users and sends it to the server.

[1205] Metaverse / Digital Twin Integration Module

[1206] Data transformation and model generation

[1207] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. Based on the converted data, the server generates a model for the metaverse or digital twin, and the terminal provides the user with a preview of the model. The user starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback, including sentiment data.

[1208] As a concrete example, when a user automates the creation of sales reports, they input business tasks and emotional feedback on a settings screen. The system analyzes this data to generate the optimal automation process and monitors the user's emotional state. If the user experiences stress, the server notifies them of appropriate suggestions and areas for improvement, and takes measures to reduce the user's burden. In this way, the system achieves efficient business automation and improved user experience with minimal resources.

[1209] The following describes the processing flow.

[1210] Account Management Module

[1211] User registration and authentication

[1212] Step 1:

[1213] The user enters their account registration information (username, password, email address).

[1214] Step 2:

[1215] The terminal sends the entered information to the server.

[1216] Step 3:

[1217] The server saves the received information to a database, generates a confirmation link, and sends it to the registered email address.

[1218] Step 4:

[1219] The user clicks the confirmation link sent to their email address.

[1220] Step 5:

[1221] The device sends a request to the server based on clicking the verification link.

[1222] Step 6:

[1223] The server receives the request and updates the authentication status. It then sends a notification to the user that they can log in.

[1224] Step 7:

[1225] The user enters their username and password on the login screen.

[1226] Step 8:

[1227] The terminal sends the entered information to the server.

[1228] Step 9:

[1229] The server verifies the information against the database and performs authentication. If authentication is successful, the user is provided with a dashboard screen.

[1230] Business task automation module

[1231] Setting up and automating business tasks

[1232] Step 1:

[1233] The user enters the business tasks they want to automate on the settings screen.

[1234] Step 2:

[1235] The terminal sends the configured work task information to the server.

[1236] Step 3:

[1237] The server analyzes the task information it receives and selects the appropriate AI model.

[1238] Step 4:

[1239] The server trains an AI model based on task information and generates an automated process.

[1240] Step 5:

[1241] The server sends the progress and results of the automated process it generates to the terminal.

[1242] Step 6:

[1243] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[1244] Process monitoring and management module

[1245] Progress monitoring and anomaly detection

[1246] Step 1:

[1247] The server monitors the progress of automated business processes in real time and records logs.

[1248] Step 2:

[1249] The database stores monitoring data and works in conjunction with anomaly detection algorithms.

[1250] Step 3:

[1251] The server detects data that deviates from the configured parameters and baseline values, and generates an alert when an anomaly is detected.

[1252] Step 4:

[1253] The terminal provides alert information to the user through an interface.

[1254] Step 5:

[1255] Users can check progress and issues in real time on the dashboard screen.

[1256] Feedback collection and analysis module

[1257] Data collection and analysis

[1258] Step 1:

[1259] The server continuously collects result data from automated processes and feedback from users.

[1260] Step 2:

[1261] The database systematically stores all collected data.

[1262] Step 3:

[1263] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns.

[1264] Step 4:

[1265] The terminal visualizes the analysis results through an interface that provides them to the user.

[1266] Step 5:

[1267] Users review the analysis results and incorporate them into the design of new automated processes.

[1268] Emotion Engine Module

[1269] Emotion recognition and feedback

[1270] Step 1:

[1271] When users set up work tasks and input feedback, they will also input emotional data.

[1272] Step 2:

[1273] The device sends user emotion data to the server.

[1274] Step 3:

[1275] The server analyzes emotional data using an emotion engine to understand the user's emotional state.

[1276] Step 4:

[1277] The server provides specific feedback based on sentiment analysis to improve the user experience.

[1278] Step 5:

[1279] For example, if a user is experiencing stress, the server will notify them with suggestions on how to address it.

[1280] Step 6:

[1281] Based on emotional data, the server identifies areas for improvement in automated processes, aiming to enhance the overall efficiency of the system.

[1282] External sales compatible module

[1283] Service customization and delivery

[1284] Step 1:

[1285] The server generates standard service packages for external sales based on accumulated know-how and emotional data.

[1286] Step 2:

[1287] Users can customize the details of external sales services on the settings screen.

[1288] Step 3:

[1289] The device sends customized configuration information to the server.

[1290] Step 4:

[1291] The server reflects the configuration and generates a new service package.

[1292] Step 5:

[1293] The server releases customized service packages and supports users in use.

[1294] Step 6:

[1295] The device collects feedback from users while they are using it and sends it to the server.

[1296] Metaverse / Digital Twin Integration Module

[1297] Data transformation and model generation

[1298] Step 1:

[1299] The server converts business data into formats for the metaverse or digital twin.

[1300] Step 2:

[1301] The database stores the converted data.

[1302] Step 3:

[1303] The server generates metaverse and digital twin models based on the converted data.

[1304] Step 4:

[1305] The device provides the user with a preview of the model.

[1306] Step 5:

[1307] Users initiate simulations within the metaverse or digital twin to verify business processes.

[1308] Step 6:

[1309] The terminal displays and provides the user with simulation results in real time.

[1310] Step 7:

[1311] The server analyzes the simulation results and accumulates feedback.

[1312] (Example 2)

[1313] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[1314] Conventional business automation systems lack means of providing feedback and process improvement based on the user's emotional state. Therefore, while operational efficiency improves, there is a risk of user stress and dissatisfaction accumulating. Furthermore, anomaly detection and process progress monitoring are often insufficient, making it difficult to find optimal improvement measures. This invention aims to solve these problems, improve the user experience, and maximize operational efficiency.

[1315] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[1316] In this invention, the server includes means for receiving account registration information from a user and storing it in a database; means for sending a confirmation link to an email address based on the account registration information; means for receiving a click of the confirmation link from a user and updating the authentication status; means for receiving a username and password entered by the user from the login screen and authenticating by comparing them with the information in the database; means for inputting user emotion data and recognizing the user's emotional state; means for providing feedback to improve the user experience based on the recognized emotion data; means for receiving business task setting information and generating an automated process using an AI model; means for monitoring the progress of the generated automated process in real time and detecting anomalies; means for providing a dashboard to notify the user of the process progress and anomaly information; means for extracting areas for improvement in the automated process based on emotion data; means for collecting result data of the automated business process and feedback from users and storing it in a database; means for analyzing the stored data using an AI / machine learning model and extracting areas for improvement and patterns; means for providing the analysis results to the user and reflecting them in the design of a new automated process; and means for providing feedback based on the user's emotion data.

[1317] This enables business process automation that takes into account the user's emotional state, simultaneously achieving efficient business processing and improved user experience. Furthermore, real-time monitoring and anomaly detection optimize business processes, allowing for maximum results with minimal resources.

[1318] "Account registration information" refers to information necessary for user identification and authentication, and specifically includes username, password, email address, etc.

[1319] A "database" is a system or device used to store and manage various types of data, such as account registration information, work task information, results data, and feedback.

[1320] A "verification link" is a link sent to confirm the accuracy of your email address and to authenticate your account registration.

[1321] "Authentication status" refers to the information indicating whether a user has legitimate qualifications, and it is updated by clicking on confirmation links, etc.

[1322] "Emotional data" refers to data that represents the user's psychological state and is information that is analyzed through the emotion engine.

[1323] "Feedback" refers to information such as opinions, impressions, and evaluations from users, which is used for system improvement and process optimization.

[1324] A "business task" refers to the specific tasks and work content that a user sets out to perform in order to carry out their work.

[1325] An "AI model" refers to an artificial intelligence model used for purposes such as data analysis and business process automation, and is built based on machine learning algorithms.

[1326] An "automation process" is a series of processes or flows that automatically execute business tasks generated and configured by an AI model.

[1327] "Real-time monitoring" is a function that allows a system to instantly monitor ongoing business processes and understand their status.

[1328] "Anomaly detection" is a function that detects data or events that deviate from set standards or parameters, and notifies of problems early.

[1329] A "dashboard" is an information screen that allows users to visually check and manage the current status, progress, and any anomalies of a system.

[1330] An "AI / machine learning model" is an algorithm or model used for pattern learning and prediction based on large amounts of data, and is used for system analysis and optimization.

[1331] "Areas for improvement" refer to problems or areas for efficiency improvement in business processes and systems, and are identified through analysis results.

[1332] A "pattern" refers to a certain regularity or trend found within data, which is extracted through analysis.

[1333] A "new automation process" is an updated automation method that improves upon existing processes and enables more efficient work execution.

[1334] This invention relates to a system that combines efficient automation of tasks using AI with user emotion recognition. This system comprises modules for account management, task automation, process monitoring and management, feedback collection and analysis, and emotion recognition. The specific processing of each module and its implementation examples are described below.

[1335] Account Management Module

[1336] User registration and authentication

[1337] The user uses a device to access the system and enters account registration information such as username, password, and email address. The device sends this information to the server. The server stores the received information in its database, generates a verification link, and sends it to the registered email address. When the user clicks the verification link in the email, the device sends a request to the server. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server. The server performs authentication by comparing it with the database.

[1338] Business task automation module

[1339] Setting up and automating business tasks

[1340] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server. The server analyzes the received task information and selects an appropriate AI model. The server trains the AI ​​model based on the task information and generates an automated process. The server sends the progress and results of the generated automated process to the terminal, which then visualizes and presents this information to the user.

[1341] Process monitoring and management module

[1342] Progress monitoring and anomaly detection

[1343] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. When the server detects data that deviates from configured parameters or baseline values, it identifies anomalies and generates alerts. The terminal notifies the user of this alert information, and the user can check the progress and problems in real time on the dashboard screen.

[1344] Feedback collection and analysis module

[1345] Data collection and analysis

[1346] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server analyzes the stored data using AI / machine learning models to extract areas for improvement and patterns. The terminal provides the analysis results to the user, who then incorporates them into the design of new automated processes.

[1347] Emotion Engine Module

[1348] Emotion recognition and feedback

[1349] The server is equipped with an emotion engine that recognizes user emotions. When a user inputs feedback on setting up work tasks or automating processes, the terminal simultaneously collects user emotion data and sends it to the server. The server analyzes this data using the emotion engine to understand the user's emotional state. For example, if a user is feeling stressed, the system uses this information to provide specific feedback to improve the user experience. It also extracts areas for improvement in automated processes based on the emotion data, aiming to improve the overall efficiency of the system.

[1350] Specific example

[1351] For example, when a user automates the creation of sales reports, they enter the work tasks and emotional feedback in the settings screen. The system analyzes this data to generate the optimal automation process and monitors the user's emotional state. If the user is stressed, the server notifies them with appropriate suggestions and areas for improvement. An example of a specific prompt message is: "Please enter the settings for automating the creation of sales reports. Next, please select your recent emotional state from the following options: (1) Relaxed (2) Tired (3) Stressed."

[1352] This allows the system to efficiently automate tasks while considering the user's emotional state, thereby improving the user experience. Furthermore, real-time monitoring and anomaly detection optimize business processes, enabling maximum results with minimal resources.

[1353] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1354] Step 1:

[1355] The user enters their account registration information.

[1356] The user uses a terminal to access the system and enters their username, password, and email address. This becomes the input data. The terminal collects this data, formats it, and sends it to the server. The output is the transmission of account registration information to the server.

[1357] Step 2:

[1358] The device sends registration information to the server.

[1359] The terminal sends the account registration information entered by the user to the server. The server analyzes the received data and saves it to a database. This registers the user's account information in the system. The output is the account information stored in the database.

[1360] Step 3:

[1361] The server generates and sends a verification link.

[1362] The server generates a confirmation link based on the email address stored in the database and sends a confirmation email containing that link to the user. This is data processing based on the input data. The output is the confirmation email sent to the user.

[1363] Step 4:

[1364] The user clicked the confirmation link.

[1365] The user clicks the verification link in the email. The input indicating the link click is sent to the device. The output is the server sending of the request based on the verification link.

[1366] Step 5:

[1367] The device sends the request to the server.

[1368] The device sends a request to the server for the verification link clicked by the user. The server receives this request and updates the user's authentication status. The output is the updated authentication status.

[1369] Step 6:

[1370] The user enters their login information.

[1371] The user enters their username and password on the login screen and sends them to the terminal. This becomes the input data for login. The output is the transmission of the username and password from the terminal to the server.

[1372] Step 7:

[1373] The device sends login information to the server.

[1374] The terminal sends login information to the server, which then authenticates by comparing it against the database. The input is the received username and password, and the output is the authentication result.

[1375] Step 8:

[1376] Users set up work tasks.

[1377] The user uses a terminal to enter the business tasks they want to automate on the settings screen. This setting information becomes the input data. The output is sending this setting information to the server.

[1378] Step 9:

[1379] The device sends configuration information to the server.

[1380] The terminal sends user configuration information to the server. The server analyzes the received information and selects an appropriate AI model. The output is the selected AI model.

[1381] Step 10:

[1382] The server trains the AI ​​model.

[1383] The server trains an AI model based on task information. This process utilizes data processing and machine learning algorithms. The output is the model training result.

[1384] Step 11:

[1385] The server generates an automated process.

[1386] The server generates automated processes using a pre-trained AI model. The input is the training results and task information, and the output is the automated process.

[1387] Step 12:

[1388] The server monitors the progress in real time.

[1389] The server monitors the progress of automated business processes in real time and records logs. Input is progress data, and output is the log records.

[1390] Step 13:

[1391] The server detected an anomaly.

[1392] The server detects data that deviates from the configured parameters and baseline values, and identifies anomalies. Input is real-time progress data, and output is anomaly detection alerts.

[1393] Step 14:

[1394] The device notifies the user of alert information.

[1395] The terminal notifies the user of alert information received from the server. The input is the alert data from the server, and the output is the notification to the user.

[1396] Step 15:

[1397] Users can view this on the dashboard.

[1398] Users can check progress and issues on the dashboard. This allows users to understand the status of their work in real time. The output is the user's understanding and action.

[1399] Step 16:

[1400] The server collects result data and feedback.

[1401] The server collects result data from automated processes and user feedback, and stores it in a database. The input is result data and feedback, and the output is the stored database.

[1402] Step 17:

[1403] The server analyzes the data using AI / machine learning models.

[1404] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns. The input is the accumulated data, and the output is the analysis results.

[1405] Step 18:

[1406] The device provides the user with the analysis results.

[1407] The terminal provides the user with analysis results. The input is analysis result data from the server, and the output is the analysis information obtained by the user.

[1408] Step 19:

[1409] Users design new automation processes.

[1410] The user designs a new automated process based on the analysis results, further improving operational efficiency. The output is the new automated process.

[1411] Step 20:

[1412] Users enter emotional data

[1413] Users input their emotional data when setting up work tasks or providing feedback. This input represents data about the user's psychological state.

[1414] Step 21:

[1415] The device sends emotional data to the server.

[1416] The device sends the collected emotional data to the server. The input is emotional data from the user, and the output is data sent to the server.

[1417] Step 22:

[1418] The server performs emotion recognition and analysis.

[1419] The server uses an emotion engine to analyze emotional data and understand the user's emotional state. The input is emotional data, and the output is the analysis result.

[1420] Step 23:

[1421] The server provides feedback

[1422] The server provides feedback to improve the user experience based on the analysis results. The input is the sentiment analysis result, and the output is the specific feedback content.

[1423] Step 24:

[1424] The server extracts areas for process improvement based on emotional data.

[1425] The server extracts areas for improvement in the automated process based on emotional data. The input is emotional data and analysis results, and the output is the extracted areas for improvement.

[1426] (Application Example 2)

[1427] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[1428] Conventional business automation systems often fail to consider the emotional state of users, leading to user burden and limiting improvements in operational efficiency. Furthermore, they lacked sufficient mechanisms for real-time collection and analysis of customer feedback, making immediate responses necessary for business improvement difficult. Additionally, real-time monitoring of business task progress and immediate responses to anomalies were inadequate. Solving these challenges was essential.

[1429] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving account registration information from a user and storing it in a database, means for sending a confirmation link to an email address based on the account registration information, means for receiving a click of the confirmation link from a user and updating the authentication status, means for receiving the username and password entered by the user from the login screen and authenticating them by comparing them with the information in the database, means for recognizing and analyzing the emotional state of staff, means for providing appropriate feedback and support based on the emotional state of staff, and means for monitoring the progress of automated work tasks in real time and detecting anomalies. This makes it possible to automate tasks while taking into account the emotional state of the user, thereby reducing the burden on staff and achieving improved work efficiency and immediate problem solving.

[1430] "Account registration information" refers to information such as the username, password, and email address that a user provides to access the system.

[1431] A "database" is a system that stores saved account registration information and business task data, and allows for searching and updating of that information.

[1432] A "confirmation link" is a URL sent via email to users to verify the validity of their registration information.

[1433] "Authentication status" is data that indicates the authentication status when a user accesses the system.

[1434] A "username" is a string of characters used to uniquely identify a user when accessing the system.

[1435] A "password" is confidential information used for security authentication when accessing a system.

[1436] "Staff" refers to individuals who perform duties at a physical store.

[1437] "Emotional state" refers to data that indicates the current psychological and emotional condition of a user or staff member.

[1438] "Feedback" refers to information collected from users and customers, such as opinions and evaluations, to help improve the system.

[1439] "Real-time" means that data is updated and processed almost instantly.

[1440] A "dashboard" is a screen that visually displays important information within a user interface.

[1441] An "AI model" is a mathematical model that uses artificial intelligence technology to automate and optimize specific tasks.

[1442] An "abnormal" refers to data or a situation that deviates from established standards or normal patterns.

[1443] A "business task" is a specific work item that requires automation and monitoring in order to perform business efficiently.

[1444] This invention is a system that combines an AI model and emotion recognition technology, primarily to improve operational efficiency in physical stores. Embodiments of this system are described below.

[1445] overview

[1446] This system is designed to reduce the burden on store staff and streamline operations. Using smart glasses, the system allows staff to monitor and manage work processes in real time. It also enables rapid collection of customer feedback and its use in improving operations. Furthermore, by recognizing staff emotional states and providing appropriate support when stressed, it improves employee satisfaction and work efficiency.

[1447] Program Overview

[1448] 1. Account Management:

[1449] Staff members use smart glasses to log into their accounts and check shift information and other details.

[1450] 2. Automation of business tasks:

[1451] Automate business tasks such as inventory management, checking displayed merchandise, and scheduling cleaning.

[1452] 3. Monitoring process progress:

[1453] It monitors the progress of tasks in real time and sends alerts if any anomalies occur.

[1454] 4. Feedback collection and analysis:

[1455] We collect customer feedback, analyze it using AI and machine learning, and identify areas for improvement.

[1456] 5. Emotion recognition:

[1457] Recognize the emotional state of staff and provide support, such as suggesting breaks if they are feeling stressed.

[1458] Hardware and software to be used

[1459] Hardware: Smart glasses (e.g., Google Glass), servers, databases

[1460] Software: Emotion recognition library, task automation library, feedback collection library, dashboard display software

[1461] AI Models: AI / Machine Learning Models for Task Automation and Feedback Analysis

[1462] Processing flow

[1463] This system begins with staff logging into their accounts using smart glasses. Next, an AI model analyzes task information to automate work tasks and monitors progress in real time. Staff emotional states are analyzed by an emotion recognition library, and if stress is detected, an alert recommending a break is sent to the smart glasses. Customer feedback is collected in real time and analyzed by the AI ​​model. This results in specific operational improvement suggestions being presented on a dashboard.

[1464] Specific example

[1465] For example, a store employee arrives for work and logs in using smart glasses. When this employee enters an inventory check task into the system, the system automatically begins checking inventory. If the emotion recognition library detects the employee's emotional state as "stressed" during the process, the smart glasses suggest a break. Additionally, when a customer provides feedback about a product, the system analyzes the feedback in real time and displays suggestions for improving the display method on a dashboard.

[1466] Examples of prompts for generative AI models

[1467] Perform the following emotion recognition to detect the stress levels of your store staff:

[1468] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1469] Step 1:

[1470] The user puts on smart glasses and enters their account registration information (username, password). The device sends this information to the server. The server retrieves the corresponding user information from the database and performs authentication. If authentication is successful, a login success message is sent back to the device, and the user can log in to the system.

[1471] Step 2:

[1472] The user inputs configuration information for a business task (e.g., inventory management) through smart glasses. The device sends this task information to a server. The server analyzes the received task information, selects and trains an appropriate AI model, and generates an automated process. Once this automated process is generated, its progress and results are sent from the server to the device and presented to the user visually.

[1473] Step 3:

[1474] The server monitors the progress of the generated automated processes in real time. Progress data is sent to the server, and an anomaly detection algorithm detects data that deviates from the standard values. When an anomaly is detected, the server generates an alert and sends it to the terminal. The terminal notifies the user of the alert information and displays the details of the anomaly and countermeasures on the dashboard.

[1475] Step 4:

[1476] The user sends feedback (e.g., customer opinions) through smart glasses. The device sends the feedback data to a server. The server collects this feedback data and stores it in a database. The stored data is analyzed using AI and machine learning models to extract areas for improvement and patterns. The analysis results are sent from the server to the device and provided to the user.

[1477] Step 5:

[1478] The server uses an emotion recognition library to analyze the user's emotional state. The device sends data acquired from the camera and microphone to the server, which analyzes this data to recognize the user's emotional state. If "stress" is detected, the server generates appropriate feedback (e.g., a suggestion to take a break) and sends it to the device. The device then notifies the user of this feedback.

[1479] Step 6:

[1480] Customers provide feedback in-store using smart glasses. The device sends this feedback information to a server. The server analyzes the feedback data and generates improvement suggestions. The generated suggestions are sent to the device and presented to the user visually. This enables rapid business improvements that reflect customer feedback.

[1481] As an example of using a generative AI model, consider the prompt: "Perform the following emotion recognition to detect the stress level of store staff:" Based on this prompt, the AI ​​model performs emotion analysis.

[1482] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1483] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1484] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1485] [Third Embodiment]

[1486] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1487] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1488] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1489] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1490] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1491] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1492] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1493] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1494] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1495] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1496] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1497] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1498] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, and feedback collection and analysis. The specific processing of each module and its implementation examples are described below.

[1499] Account Management Module

[1500] User registration and authentication

[1501] The user uses a device to access the system and enter their account registration information. The device sends the entered information, such as username, password, and email address, to the server. The server stores the received information in a database and sends a confirmation link to the registered email address. When the user clicks the confirmation link in the email, the device sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server, which then authenticates them by comparing it with the information in the database.

[1502] Business task automation module

[1503] Setting up and automating business tasks

[1504] The user enters the business tasks they want to automate (e.g., report creation or data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model. The server then trains the AI ​​model based on the task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[1505] Process monitoring and management module

[1506] Progress monitoring and anomaly detection

[1507] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. The server detects data that deviates from configured parameters and baseline values, and generates an alert when an anomaly is found. The terminal notifies the user of this alert information, allowing them to check progress and problems on a real-time dashboard.

[1508] Feedback collection and analysis module

[1509] Data collection and analysis

[1510] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server then analyzes this data using AI / machine learning models to extract areas for improvement and patterns. The analysis results are then provided to users via their terminals, allowing them to incorporate them into the design of new automated processes.

[1511] External sales compatible module

[1512] Service customization and delivery

[1513] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. The customized service package is released, and support is provided to users during its use. The terminal collects feedback from users and sends it to the server.

[1514] Metaverse / Digital Twin Integration Module

[1515] Data transformation and model generation

[1516] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. Based on the converted data, the server generates a metaverse or digital twin model, and the terminal provides the user with a preview of the model. The user then starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[1517] As described above, each module works in conjunction with others to enable efficient automation, management, analysis, and feedback collection and sharing of operations, providing an environment in which companies can operate with minimal resources.

[1518] The following describes the processing flow.

[1519] Account Management Module

[1520] User registration and authentication

[1521] Step 1:

[1522] The user enters their account registration information (username, password, email address).

[1523] Step 2:

[1524] The terminal sends the entered information to the server.

[1525] Step 3:

[1526] The server saves the received information to a database, generates a confirmation link, and sends it to the registered email address.

[1527] Step 4:

[1528] The user clicks the confirmation link sent to their email address.

[1529] Step 5:

[1530] The device sends a request to the server based on clicking the verification link.

[1531] Step 6:

[1532] The server receives the request and updates the authentication status. It then sends a notification to the user that they can log in.

[1533] Step 7:

[1534] The user enters their username and password on the login screen.

[1535] Step 8:

[1536] The terminal sends the entered information to the server.

[1537] Step 9:

[1538] The server verifies the information against the database and performs authentication. If authentication is successful, the user is provided with a dashboard screen.

[1539] Business task automation module

[1540] Setting up and automating business tasks

[1541] Step 1:

[1542] The user enters the business tasks they want to automate on the settings screen.

[1543] Step 2:

[1544] The terminal sends the configured work task information to the server.

[1545] Step 3:

[1546] The server analyzes the task information it receives and selects the appropriate AI model.

[1547] Step 4:

[1548] The server trains an AI model based on task information and generates an automated process.

[1549] Step 5:

[1550] The server sends the progress and results of the automated process it generates to the terminal.

[1551] Step 6:

[1552] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[1553] Process monitoring and management module

[1554] Progress monitoring and anomaly detection

[1555] Step 1:

[1556] The server monitors the progress of automated business processes in real time and records logs.

[1557] Step 2:

[1558] The database stores monitoring data and works in conjunction with anomaly detection algorithms.

[1559] Step 3:

[1560] The server detects data that deviates from the configured parameters and baseline values, and generates an alert when an anomaly is detected.

[1561] Step 4:

[1562] The terminal provides alert information to the user through an interface.

[1563] Step 5:

[1564] Users can check progress and issues in real time on the dashboard screen.

[1565] Feedback collection and analysis module

[1566] Data collection and analysis

[1567] Step 1:

[1568] The server continuously collects result data from automated processes and feedback from users.

[1569] Step 2:

[1570] The database systematically stores all collected data.

[1571] Step 3:

[1572] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns.

[1573] Step 4:

[1574] The terminal visualizes the analysis results through an interface that provides them to the user.

[1575] Step 5:

[1576] Users review the analysis results and incorporate them into the design of new automated processes.

[1577] External sales compatible module

[1578] Service customization and delivery

[1579] Step 1:

[1580] The server generates standard service packages for external sales based on accumulated know-how.

[1581] Step 2:

[1582] Users can customize the details of external sales services on the settings screen.

[1583] Step 3:

[1584] The device sends customized configuration information to the server.

[1585] Step 4:

[1586] The server reflects the configuration and generates a new service package.

[1587] Step 5:

[1588] The server releases customized service packages and supports users in use.

[1589] Step 6:

[1590] The device collects feedback from users while they are using it and sends it to the server.

[1591] Metaverse / Digital Twin Integration Module

[1592] Data transformation and model generation

[1593] Step 1:

[1594] The server converts business data into formats for the metaverse or digital twin.

[1595] Step 2:

[1596] The database stores the converted data.

[1597] Step 3:

[1598] The server generates metaverse and digital twin models based on the converted data.

[1599] Step 4:

[1600] The device provides the user with a preview of the model.

[1601] Step 5:

[1602] Users initiate simulations within the metaverse or digital twin to verify business processes.

[1603] Step 6:

[1604] The terminal displays and provides the user with simulation results in real time.

[1605] Step 7:

[1606] The server analyzes the simulation results and accumulates feedback.

[1607] (Example 1)

[1608] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1609] Traditional systems relied on manual processes for automating, monitoring, and analyzing business processes, leading to decreased operational efficiency. Furthermore, effectively collecting and analyzing user feedback and incorporating it into new process design proved difficult. Additionally, the limited use of virtual spaces and digital replication hindered the easy simulation and verification of business processes.

[1610] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1611] In this invention, the server includes means for receiving account registration information from a user and storing it in a database; means for sending a confirmation link to an email address based on the account registration information; means for receiving a click of the confirmation link from the user and updating the authentication status; means for converting business data into a virtual space or digital replica format and storing it on an analysis platform; means for generating a virtual space or digital replica model based on the converted data; and means for providing the user with a preview of the generated model and initiating a simulation. This enables automation and efficiency improvements of business operations, real-time monitoring and anomaly detection, and effective collection and analysis of user feedback.

[1612] A "user" refers to anyone who uses this system to perform operations such as registering an account, setting up work tasks, logging in, and providing feedback.

[1613] A "server" refers to a computer system that receives requests from users, stores information in a database, and performs overall system processing, including training and running AI models, generating automated processes, monitoring progress, detecting anomalies, and generating models for virtual spaces and digital replicas.

[1614] "Database" refers to a digital storage system for permanently storing various types of data required by this system, such as account registration information, work task information, progress data, and feedback data.

[1615] A "verification link" refers to a temporary URL sent to the user's email address, which the user clicks to authenticate their account.

[1616] "Authentication status" refers to status information indicating whether the user has clicked the verification link and completed account authentication.

[1617] "Business tasks" refer to business processes, such as report creation and data entry, that users set up in this system.

[1618] A "generative AI model" refers to an artificial intelligence model trained to generate appropriate automation processes based on the user's business task information.

[1619] A "prompt" refers to the text information or question sentences that are input into a generative AI model.

[1620] "Anomaly detection" refers to the process of monitoring the progress of automated business processes and identifying data that deviates from set parameters or baseline values.

[1621] A "virtual space" refers to a digital environment that digitally reproduces the actual physical space for conducting simulations and model verification.

[1622] "Digital replication" refers to a virtual model that reproduces actual business processes and environments as digital data and is used for simulation and verification.

[1623] "Visualized display methods" refer to display methods that use dashboards, graphs, and charts to allow users to intuitively understand the progress of a process and any anomaly information.

[1624] "Analysis techniques" refer to algorithms and methods for analyzing large amounts of data stored in a database and extracting patterns and areas for improvement.

[1625] The above are the key terms and their definitions included in the claims of this system.

[1626] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, feedback collection and analysis, external sales support, and metaverse / digital twin integration. The specific operation of each module is described in detail below.

[1627] Account Management Module

[1628] User registration and authentication

[1629] The user accesses the system using a device and enters account registration information such as username, password, and email address. The device sends this information to the server, which stores the received information in a database (e.g., MySQL). The server then sends a verification link to the registered email address (e.g., using an email service). When the user clicks the verification link, the device sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server. The server authenticates the user by comparing it with the information in the database.

[1630] Business task automation module

[1631] Setting up and automating business tasks

[1632] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model (e.g., TensorFlow). The server then trains the AI ​​model based on this task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[1633] Process monitoring and management module

[1634] Progress monitoring and anomaly detection

[1635] The server monitors the progress of automated business processes in real time and logs the data (e.g., using a monitoring tool). The database stores the monitoring data and works with anomaly detection algorithms (e.g., Scikit-learn). The server detects data that deviates from configured parameters or baseline values ​​and generates an alert when an anomaly is found. The terminal notifies the user of this alert information and allows them to check progress and problems on a real-time dashboard.

[1636] Feedback collection and analysis module

[1637] Data collection and analysis

[1638] The server continuously collects result data from automated processes and user feedback, storing it in a database (e.g., PostgreSQL). The server then analyzes the stored data using an AI / machine learning model (e.g., PyTorch) to extract areas for improvement and patterns. The analysis results are provided to the user via a terminal, allowing the user to incorporate them into the design of new automated processes.

[1639] External sales compatible module

[1640] Service customization and delivery

[1641] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. It releases the customized service package and supports users who are using it. The terminal collects feedback from users and sends it to the server.

[1642] Metaverse / Digital Twin Integration Module

[1643] Data transformation and model generation

[1644] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. The server generates a model of the metaverse or digital twin based on the converted data (e.g., using Unity). The terminal provides the user with a preview of the model. The user starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[1645] Examples of specific cases and prompt statements

[1646] For example, a "mail service" can be used to send a confirmation link to a registered email address, and "MySQL" can be used as the database to store user information. Furthermore, "TensorFlow" or "PyTorch" can be used to train AI models. "Visualization tools" can be used to monitor the situation on the dashboard, and "Scikit-learn" can be utilized for anomaly detection. "3D modeling tools" can be used to generate models in virtual spaces or digital replicas.

[1647] Prompt example 1: "Send a confirmation email for new user registration and update the authentication status."

[1648] Prompt example 2: "Use a generative AI model to automate report generation tasks and visualize progress."

[1649] Prompt example 3: "Apply the anomaly detection algorithm and display an alert if an anomaly occurs."

[1650] Prompt example 4: "We will extract patterns through data analysis and identify areas for improvement."

[1651] As described above, each module works together through detailed and specific procedures, providing an environment where companies can efficiently automate their operations and operate with minimal resources.

[1652] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1653] Account Management Module

[1654] User registration and authentication

[1655] Step 1:

[1656] The user accesses the system using a terminal and enters account registration information such as username, password, and email address.

[1657] Input: Username, Password, Email Address

[1658] Output: The request the terminal sends to the server with this information.

[1659] Step 2:

[1660] The device sends the entered account registration information to the server.

[1661] Input: Username, password, and email address entered by the user.

[1662] Output: Data packets to be sent to the server

[1663] Step 3:

[1664] The server parses the information it receives and saves it to a database (for example, MySQL).

[1665] Input: Account registration information sent from the device

[1666] Output: Saving information to the database using SQL INSERT statements

[1667] Specific operation: Execute SQL queries using the Python sqlite3 module or MySQL client.

[1668] Step 4:

[1669] The server will send a confirmation link to the registered email address.

[1670] Input: User's email address

[1671] Output: Email containing a confirmation link

[1672] Specific operation: Use an email service (e.g., SendGrid) to send emails via API.

[1673] Step 5:

[1674] The user clicks the verification link, and the device sends an authentication request to the server.

[1675] Input: Link access via user click

[1676] Output: Request to the server

[1677] Specific action: The device sends an HTTP GET request to the server.

[1678] Step 6:

[1679] The server receives the request and updates the authentication status.

[1680] Input: Authentication request to the server

[1681] Output: Authentication status update

[1682] Specific action: Update the status field of the corresponding user in the database.

[1683] Step 7:

[1684] The user enters their username and password on the login screen, and the device sends that information to the server.

[1685] Input: Username, Password

[1686] Output: Login request to the server

[1687] Specific action: Send data using the terminal's form submission function.

[1688] Step 8:

[1689] The server compares the information with that in the database and performs authentication.

[1690] Input: Username, Password

[1691] Output: Authentication success return code or error message

[1692] Specific operation: Database matching is performed using an SQL SELECT statement.

[1693] ---

[1694] Business task automation module

[1695] Setting up and automating business tasks

[1696] Step 1:

[1697] The user enters the business tasks they want to automate (e.g., report creation, data entry) on the settings screen.

[1698] Input: Detailed information about the task

[1699] Output: The request the terminal sends to the server with this information.

[1700] Step 2:

[1701] The device sends configuration information to the server.

[1702] Input: Detailed information about the task

[1703] Output: Request for configuration information from the server

[1704] Step 3:

[1705] The server analyzes the task information and selects an appropriate AI model (e.g., TensorFlow).

[1706] Input: Detailed information about the task

[1707] Output: Selected AI models

[1708] Specific action: Execute the AI ​​model selection algorithm.

[1709] Step 4:

[1710] The server trains the AI ​​model and generates automated processes.

[1711] Input: Detailed information on the business task, selected AI model

[1712] Output: Trained model, generated automation process

[1713] Specific operation: Train the model using the TensorFlow library.

[1714] Step 5:

[1715] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[1716] Input: Progress data, result data

[1717] Output: Visualized dashboard

[1718] Specific actions: Use visualization tools (e.g., Grafana) to display data in graphs and charts.

[1719] ---

[1720] Process monitoring and management module

[1721] Progress monitoring and anomaly detection

[1722] Step 1:

[1723] The server monitors the progress of automated business processes in real time and records logs.

[1724] Input: Business process progress data

[1725] Output: Real-time monitoring data, recorded logs

[1726] Specific operation: Collect real-time data using a monitoring tool (e.g., Prometheus) and save it as a log.

[1727] Step 2:

[1728] The database stores the monitoring data.

[1729] Input: Real-time monitoring data

[1730] Output: Saved monitoring data

[1731] Specific operation: Execute an SQL INSERT statement into the database and save the data.

[1732] Step 3:

[1733] The server applies an anomaly detection algorithm.

[1734] Input: Stored monitoring data

[1735] Output: Anomaly detection result

[1736] Specific operation: Execute an anomaly detection algorithm using Scikit-learn.

[1737] Step 4:

[1738] The server detects an anomaly and generates an alert.

[1739] Input: Anomaly detection result

[1740] Output: Generated alerts

[1741] Specific action: Generate an alert message and output it in the specified format.

[1742] Step 5:

[1743] The device notifies the user of alert information.

[1744] Input: Generated alert

[1745] Output: Notification to the user

[1746] Specific operation: Uses a GUI to send pop-up notifications and email notifications.

[1747] Step 6:

[1748] The device will be able to check progress and issues in real time on a dashboard.

[1749] Input: Alert information, progress data

[1750] Output: Visualized dashboard

[1751] Specific actions: Display data using a visualization tool (e.g., Grafana).

[1752] ---

[1753] Feedback collection and analysis module

[1754] Data collection and analysis

[1755] Step 1:

[1756] The server continuously collects data from automated processes and user feedback, and stores it in a database.

[1757] Input: Result data, feedback data

[1758] Output: Stored data

[1759] Specific operation: Use the API to collect data and store it in the database.

[1760] Step 2:

[1761] The data accumulated by the server is analyzed using AI / machine learning models.

[1762] Input: Accumulated data

[1763] Output: Analysis results (areas for improvement and patterns)

[1764] Specific operation: Use PyTorch to feed the dataset into the model and perform analysis.

[1765] Step 3:

[1766] The device provides the user with the analysis results.

[1767] Input: Analysis results

[1768] Output: Display of analysis results to the user

[1769] Specific actions: Visualize and display results on the dashboard.

[1770] Step 4:

[1771] Users will incorporate their feedback into the design of new automation processes.

[1772] Input: Analysis results, feedback

[1773] Output: Newly designed automation process

[1774] Specific actions: Use operational design tools to design a new process.

[1775] ---

[1776] External sales compatible module

[1777] Service customization and delivery

[1778] Step 1:

[1779] The server generates the standard service package.

[1780] Input: Accumulated know-how

[1781] Output: Standard service package

[1782] Specific actions: Reading data from the database and executing a template generation script.

[1783] Step 2:

[1784] The user can customize the settings.

[1785] Input: Customization details

[1786] Output: Sending of configuration information by the terminal

[1787] Specific operation: Form input and submission via GUI.

[1788] Step 3:

[1789] The device sends customization information to the server.

[1790] Input: Customization details

[1791] Output: Request to the server

[1792] Specific operation: Send JSON data via an HTTP POST request.

[1793] Step 4:

[1794] The server will reflect the customizations.

[1795] Input: Customization Information

[1796] Output: Customized service package

[1797] Specific action: Generate a new package using a template and save it.

[1798] Step 5:

[1799] The server releases customized service packages and supports users in use.

[1800] Input: Customized package

[1801] Output: Released service packages, support information

[1802] Specific actions: Deploy and support using cloud services such as AWS.

[1803] Step 6:

[1804] The device collects feedback and sends it to the server.

[1805] Input: User feedback

[1806] Output: Sending feedback data to the server

[1807] Specific action: Submit data through the feedback form.

[1808] ---

[1809] Metaverse / Digital Twin Integration Module

[1810] Data transformation and model generation

[1811] Step 1:

[1812] The server converts business data into formats for the metaverse and digital twin.

[1813] Input: Business data

[1814] Output: Converted data

[1815] Specific operation: Executes a data conversion script (e.g., Python or C++).

[1816] Step 2:

[1817] The server saves the converted data to the database.

[1818] Input: Converted data

[1819] Output: Saved data

[1820] Specific action: Execute an SQL INSERT statement into the database.

[1821] Step 3:

[1822] The server generates metaverse and digital twin models based on the converted data.

[1823] Input: Converted data

[1824] Output: Generated model

[1825] Specific actions: Build the model using the Unity API.

[1826] Step 4:

[1827] The device provides the user with a preview of the model.

[1828] Input: Generated model

[1829] Output: Model preview screen

[1830] Specific action: Display the model using the 3D viewer.

[1831] Step 5:

[1832] The user starts a simulation within the metaverse or digital twin.

[1833] Input: Instruction to start simulation

[1834] Output: Run the simulation

[1835] Specific action: Click the Start button to begin the simulation.

[1836] Step 6:

[1837] The device displays the simulation results in real time.

[1838] Input: Simulation result data

[1839] Output: Simulation results displayed in real time

[1840] Specific actions: Display data in dashboards and graphs.

[1841] Step 7:

[1842] The server analyzes the simulation results and accumulates feedback.

[1843] Input: Simulation result data

[1844] Output: Analysis results, feedback

[1845] Specific operation: Analyze the result data and save it as Insights.

[1846] Through the above processing steps, each module is efficiently linked, enabling automation of operations, monitoring of progress, anomaly detection, feedback collection, customized services, and integration with the metaverse and digital twin.

[1847] (Application Example 1)

[1848] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1849] In modern factories, while efficiency and automation are essential, managing robots and setting and monitoring tasks are becoming increasingly complex. The lack of integrated real-time monitoring of work progress, anomaly detection, feedback collection and analysis, and account management for technicians and managers often hinders operational efficiency. Furthermore, the lack of established methods for designing and improving automation processes using generative AI models is slowing down the optimization of operations.

[1850] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1851] In this invention, the server includes means for receiving authentication information from a user and storing it in a database; means for sending a confirmation link via communication means based on the authentication information; means for receiving a click of the confirmation link from a user and updating the authentication status; means for receiving authentication information entered by a user from an authentication screen and performing authentication by comparing it with the information in the database; means for managing accounts of robot operators and administrators and performing access control; means for receiving setting information of business tasks from authenticated users and generating an automated process using a generation AI model; means for monitoring the progress of the generated automated process in real time and detecting anomalies; means for providing a dashboard that notifies the user of the progress of the process and anomaly information; means for collecting result data of the automated business process and feedback from users and storing it in a database; means for analyzing the stored data using a generation AI model and a machine learning model and extracting areas for improvement and patterns; and means for providing the user with the analysis results and reflecting them in the design of new automated processes. As a result, robot management and automation of business tasks in the factory are integrated, enabling improved work efficiency and faster anomaly detection.

[1852] "Authentication information" refers to information used to identify a user and verify their access rights.

[1853] A "database" is an electronic recording device used to efficiently store, search, and manage large amounts of data.

[1854] A "verification link" is a URL that users receive via email or other means and click to authenticate their account.

[1855] "Authentication status" refers to the state of information indicating whether a user has legitimate authority.

[1856] A "robot operator or manager" is a person responsible for operating and managing robots within a factory.

[1857] "Access control" is the process of restricting and managing access rights to information systems.

[1858] "Business tasks" refer to specific tasks or operations that are performed by robots within a factory.

[1859] A "generative AI model" is an artificial intelligence model trained using machine learning algorithms to generate automated processes for business tasks.

[1860] An "automation process" is a set of steps or processes that automatically execute specific tasks or operations.

[1861] "Real-time" refers to the process and display of data and information immediately as soon as it is generated.

[1862] "Anomaly detection" is the process of discovering behavior or errors that deviate from normal operation.

[1863] A "dashboard" is a user interface that visually displays multiple pieces of information, allowing users to check progress and any anomalies at a glance.

[1864] "Feedback" refers to information about opinions and results collected from users and systems.

[1865] "Analysis" is the process of processing and evaluating collected data to extract useful information.

[1866] "Areas for improvement" are parts of a business process that require change to optimize it.

[1867] A "pattern" refers to the regularity or characteristics found within data.

[1868] The system that realizes this application consists of multiple modules and aims to efficiently automate robot management and tasks within a factory. The hardware and software used, as well as specific implementation examples, are described below.

[1869] Hardware and software used

[1870] 1. Server

[1871] Database management systems: MySQL and PostgreSQL

[1872] AI and machine learning frameworks: TensorFlow and PyTorch

[1873] Web frameworks: Flask and Django

[1874] 2. Terminal

[1875] Personal computers, smartphones, tablets, and other devices that users use for input and display.

[1876] 3. User

[1877] Robot operators and managers

[1878] System configuration and details

[1879] 1. Account Management Module

[1880] The server first receives authentication information from the user and stores it in the database. Next, it sends a confirmation link via communication means based on that authentication information. When the user clicks the confirmation link, the server receives the request and updates the authentication status. It receives the authentication information entered by the user on the authentication screen, verifies it against the information in the database, and performs authentication. It also manages accounts for robot operators and administrators and performs access control.

[1881] 2. Business Task Automation Module

[1882] The system receives task configuration information from authenticated users and generates automated processes using a generated AI model. The server monitors this process in real time and detects anomalies. Users are provided with a dashboard that notifies them of process progress and anomaly information. For example, it can easily configure tasks for robots performing welding operations.

[1883] 3. Process Monitoring and Management Module

[1884] The server monitors the progress of automated business processes in real time and immediately generates an alert if an anomaly is detected. Users can monitor the process progress and anomaly information in real time.

[1885] 4. Feedback Collection and Analysis Module

[1886] The server collects result data from automated business processes and user feedback, and stores it in a database. The stored data is then analyzed using generative AI models and machine learning models to extract areas for improvement and patterns. The analysis results are provided to the user and incorporated into the design of new automated processes.

[1887] Specific examples and prompt statements

[1888] For robots performing welding tasks within a factory, the user inputs data to optimize the welding pattern. Data such as weld thickness, defect rate, and speed are collected, and if an AI model predicts an anomaly, it immediately alerts the administrator. The following is an example of a user prompt:

[1889] Example of a prompt:

[1890] "Username: user1, Password: pass123, Email: user1@example.com"

[1891] "Task name: Welding, Target: Part A, Schedule time: 1627884000, Parameters: {'Thickness': 0.5, 'Speed': 8}"

[1892] "Thickness: 0.7, Speed: 10"

[1893] By using this system, robot management and automation of work tasks within the factory can be carried out smoothly.

[1894] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1895] Step 1:

[1896] The user enters authentication information using a terminal. The terminal sends this information (including username, password, and email address) to the server. The server stores the received authentication information in its database and sends a confirmation link to the email address. In this step, the input is the user's authentication information, and the output is an email containing the confirmation link.

[1897] Step 2:

[1898] The user clicks a verification link in an email they received. The device sends a request to the server based on that click. The server receives the request and updates the user's authentication status in the database. The input for this step is the click of the verification link, and the output is the updated authentication status.

[1899] Step 3:

[1900] The user enters their username and password on the authentication screen. The terminal sends this information to the server, which then verifies it against the database information to perform authentication. If authentication is successful, the user receives a login notification. The input for this step is the username and password, and the output is the authentication result.

[1901] Step 4:

[1902] An authenticated user inputs the configuration information for a business task. The terminal sends this configuration information to the server, which uses a generated AI model to create an automated process for the task. In this step, the input is the task configuration information, and the output is the generated automated process.

[1903] Step 5:

[1904] The server monitors the progress of the generated automated process in real time. If an anomaly is detected, the server generates an alert and notifies the user via the terminal. The input for this step is the ongoing process data, and the output is the anomaly detection alert.

[1905] Step 6:

[1906] Users can check process progress and anomaly information in real time through the dashboard. The terminal displays this information visually, allowing users to understand the situation. The input for this step is process progress data and anomaly information, and the output is visual information on the dashboard.

[1907] Step 7:

[1908] Once an automated business process is complete, the server collects result data and user feedback and stores it in a database. The collected data is analyzed using generating AI models and machine learning models to extract areas for improvement and patterns. The input for this step is result data and feedback, and the output is the analysis results.

[1909] Step 8:

[1910] The server provides the user with extracted areas for improvement and patterns, allowing them to incorporate them into the design of new automation processes. The user readjusts the task settings based on the analysis results and executes the optimized process. The input for this step is the analysis results, and the output is the optimized task settings.

[1911] The above steps enable efficient management of factory robots and automation of operational tasks.

[1912] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1913] This invention relates to a system that combines an emotion engine for recognizing user emotions with an AI-powered system that efficiently automates tasks and operates with minimal resources. This system includes modules for account management, automation of business tasks, process monitoring and management, feedback collection and analysis, and emotion recognition. The specific processing of each module and its implementation examples are described below.

[1914] Account Management Module

[1915] User registration and authentication

[1916] The user uses a device to access the system and enter their account registration information. The device sends the entered information, such as username, password, and email address, to the server. The server stores the received information in its database, generates a verification link, and sends it to the registered email address. When the user clicks the verification link in the email, the device sends the request to the server, which receives it, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, the device sends this information to the server, and the server authenticates them by comparing it with the database.

[1917] Business task automation module

[1918] Setting up and automating business tasks

[1919] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model. The server trains the AI ​​model based on the task information and generates an automated process. The server sends the progress and results of the generated automated process to the terminal, which then provides the user with a dashboard that visualizes the process's progress and results.

[1920] Process monitoring and management module

[1921] Progress monitoring and anomaly detection

[1922] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. The server detects data that deviates from configured parameters and baseline values, and generates an alert when an anomaly is found. The terminal notifies the user of this alert information, and the user can check the progress and problems in real time on the dashboard screen.

[1923] Feedback collection and analysis module

[1924] Data collection and analysis

[1925] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server analyzes the stored data using AI / machine learning models to extract areas for improvement and patterns. The terminal provides the analysis results to the user, who then incorporates them into the design of new automated processes.

[1926] Emotion Engine Module

[1927] Emotion recognition and feedback

[1928] The server is equipped with an emotion engine that recognizes user emotions. When a user sets up a work task or inputs feedback for an automated process, the terminal simultaneously collects the user's emotional data and sends it to the server. The server analyzes this data using the emotion engine to understand the user's emotional state. For example, if a user is feeling stressed, the system uses this information to provide specific feedback to improve the user experience. It also extracts areas for improvement in the automated process based on the emotional data, aiming to improve the overall efficiency of the system.

[1929] External sales compatible module

[1930] Service customization and delivery

[1931] The server generates a standard service package for external sales based on accumulated know-how and sentiment data. Users customize the details of the external service on the settings screen, and the terminal sends this information to the server. The server reflects the settings and generates a new service package. This package is released, and while supporting users in use, the terminal collects feedback from users and sends it to the server.

[1932] Metaverse / Digital Twin Integration Module

[1933] Data transformation and model generation

[1934] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. Based on the converted data, the server generates a model for the metaverse or digital twin, and the terminal provides the user with a preview of the model. The user starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback, including sentiment data.

[1935] As a concrete example, when a user automates the creation of sales reports, they input business tasks and emotional feedback on a settings screen. The system analyzes this data to generate the optimal automation process and monitors the user's emotional state. If the user experiences stress, the server notifies them of appropriate suggestions and areas for improvement, and takes measures to reduce the user's burden. In this way, the system achieves efficient business automation and improved user experience with minimal resources.

[1936] The following describes the processing flow.

[1937] Account Management Module

[1938] User registration and authentication

[1939] Step 1:

[1940] The user enters their account registration information (username, password, email address).

[1941] Step 2:

[1942] The terminal sends the entered information to the server.

[1943] Step 3:

[1944] The server saves the received information to a database, generates a confirmation link, and sends it to the registered email address.

[1945] Step 4:

[1946] The user clicks the confirmation link sent to their email address.

[1947] Step 5:

[1948] The device sends a request to the server based on clicking the verification link.

[1949] Step 6:

[1950] The server receives the request and updates the authentication status. It then sends a notification to the user that they can log in.

[1951] Step 7:

[1952] The user enters their username and password on the login screen.

[1953] Step 8:

[1954] The terminal sends the entered information to the server.

[1955] Step 9:

[1956] The server verifies the information against the database and performs authentication. If authentication is successful, the user is provided with a dashboard screen.

[1957] Business task automation module

[1958] Setting up and automating business tasks

[1959] Step 1:

[1960] The user enters the business tasks they want to automate on the settings screen.

[1961] Step 2:

[1962] The terminal sends the configured work task information to the server.

[1963] Step 3:

[1964] The server analyzes the task information it receives and selects the appropriate AI model.

[1965] Step 4:

[1966] The server trains an AI model based on task information and generates an automated process.

[1967] Step 5:

[1968] The server sends the progress and results of the automated process it generates to the terminal.

[1969] Step 6:

[1970] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[1971] Process monitoring and management module

[1972] Progress monitoring and anomaly detection

[1973] Step 1:

[1974] The server monitors the progress of automated business processes in real time and records logs.

[1975] Step 2:

[1976] The database stores monitoring data and works in conjunction with anomaly detection algorithms.

[1977] Step 3:

[1978] The server detects data that deviates from the configured parameters and baseline values, and generates an alert when an anomaly is detected.

[1979] Step 4:

[1980] The terminal provides alert information to the user through an interface.

[1981] Step 5:

[1982] Users can check progress and issues in real time on the dashboard screen.

[1983] Feedback collection and analysis module

[1984] Data collection and analysis

[1985] Step 1:

[1986] The server continuously collects result data from automated processes and feedback from users.

[1987] Step 2:

[1988] The database systematically stores all collected data.

[1989] Step 3:

[1990] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns.

[1991] Step 4:

[1992] The terminal visualizes the analysis results through an interface that provides them to the user.

[1993] Step 5:

[1994] Users review the analysis results and incorporate them into the design of new automated processes.

[1995] Emotion Engine Module

[1996] Emotion recognition and feedback

[1997] Step 1:

[1998] When users set up work tasks and input feedback, they will also input emotional data.

[1999] Step 2:

[2000] The device sends user emotion data to the server.

[2001] Step 3:

[2002] The server analyzes emotional data using an emotion engine to understand the user's emotional state.

[2003] Step 4:

[2004] The server provides specific feedback based on sentiment analysis to improve the user experience.

[2005] Step 5:

[2006] For example, if a user is experiencing stress, the server will notify them with suggestions on how to address it.

[2007] Step 6:

[2008] Based on emotional data, the server identifies areas for improvement in automated processes, aiming to enhance the overall efficiency of the system.

[2009] External sales compatible module

[2010] Service customization and delivery

[2011] Step 1:

[2012] The server generates standard service packages for external sales based on accumulated know-how and emotional data.

[2013] Step 2:

[2014] Users can customize the details of external sales services on the settings screen.

[2015] Step 3:

[2016] The device sends customized configuration information to the server.

[2017] Step 4:

[2018] The server reflects the configuration and generates a new service package.

[2019] Step 5:

[2020] The server releases customized service packages and supports users in use.

[2021] Step 6:

[2022] The device collects feedback from users while they are using it and sends it to the server.

[2023] Metaverse / Digital Twin Integration Module

[2024] Data transformation and model generation

[2025] Step 1:

[2026] The server converts business data into formats for the metaverse or digital twin.

[2027] Step 2:

[2028] The database stores the converted data.

[2029] Step 3:

[2030] The server generates metaverse and digital twin models based on the converted data.

[2031] Step 4:

[2032] The device provides the user with a preview of the model.

[2033] Step 5:

[2034] Users initiate simulations within the metaverse or digital twin to verify business processes.

[2035] Step 6:

[2036] The terminal displays and provides the user with simulation results in real time.

[2037] Step 7:

[2038] The server analyzes the simulation results and accumulates feedback.

[2039] (Example 2)

[2040] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2041] Conventional business automation systems lack means of providing feedback and process improvement based on the user's emotional state. Therefore, while operational efficiency improves, there is a risk of user stress and dissatisfaction accumulating. Furthermore, anomaly detection and process progress monitoring are often insufficient, making it difficult to find optimal improvement measures. This invention aims to solve these problems, improve the user experience, and maximize operational efficiency.

[2042] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means.

[2043] In this invention, the server includes means for receiving account registration information from a user and storing it in a database; means for sending a confirmation link to an email address based on the account registration information; means for receiving a click of the confirmation link from a user and updating the authentication status; means for receiving a username and password entered by the user from the login screen and authenticating by comparing them with the information in the database; means for inputting user emotion data and recognizing the user's emotional state; means for providing feedback to improve the user experience based on the recognized emotion data; means for receiving business task setting information and generating an automated process using an AI model; means for monitoring the progress of the generated automated process in real time and detecting anomalies; means for providing a dashboard to notify the user of the process progress and anomaly information; means for extracting areas for improvement in the automated process based on emotion data; means for collecting result data of the automated business process and feedback from users and storing it in a database; means for analyzing the stored data using an AI / machine learning model and extracting areas for improvement and patterns; means for providing the analysis results to the user and reflecting them in the design of a new automated process; and means for providing feedback based on the user's emotion data.

[2044] This enables business process automation that takes into account the user's emotional state, simultaneously achieving efficient business processing and improved user experience. Furthermore, real-time monitoring and anomaly detection optimize business processes, allowing for maximum results with minimal resources.

[2045] "Account registration information" refers to information necessary for user identification and authentication, and specifically includes username, password, email address, etc.

[2046] A "database" is a system or device used to store and manage various types of data, such as account registration information, work task information, results data, and feedback.

[2047] A "verification link" is a link sent to confirm the accuracy of your email address and to authenticate your account registration.

[2048] "Authentication status" refers to the information indicating whether a user has legitimate qualifications, and it is updated by clicking on confirmation links, etc.

[2049] "Emotional data" refers to data that represents the user's psychological state and is information that is analyzed through the emotion engine.

[2050] "Feedback" refers to information such as opinions, impressions, and evaluations from users, which is used for system improvement and process optimization.

[2051] A "business task" refers to the specific tasks and work content that a user sets out to perform in order to carry out their work.

[2052] An "AI model" refers to an artificial intelligence model used for purposes such as data analysis and business process automation, and is built based on machine learning algorithms.

[2053] An "automation process" is a series of processes or flows that automatically execute business tasks generated and configured by an AI model.

[2054] "Real-time monitoring" is a function that allows a system to instantly monitor ongoing business processes and understand their status.

[2055] "Anomaly detection" is a function that detects data or events that deviate from set standards or parameters, and notifies of problems early.

[2056] A "dashboard" is an information screen that allows users to visually check and manage the current status, progress, and any anomalies of a system.

[2057] An "AI / machine learning model" is an algorithm or model used for pattern learning and prediction based on large amounts of data, and is used for system analysis and optimization.

[2058] "Areas for improvement" refer to problems or areas for efficiency improvement in business processes and systems, and are identified through analysis results.

[2059] A "pattern" refers to a certain regularity or trend found within data, which is extracted through analysis.

[2060] A "new automation process" is an updated automation method that improves upon existing processes and enables more efficient work execution.

[2061] This invention relates to a system that combines efficient automation of tasks using AI with user emotion recognition. This system comprises modules for account management, task automation, process monitoring and management, feedback collection and analysis, and emotion recognition. The specific processing of each module and its implementation examples are described below.

[2062] Account Management Module

[2063] User registration and authentication

[2064] The user uses a device to access the system and enters account registration information such as username, password, and email address. The device sends this information to the server. The server stores the received information in its database, generates a verification link, and sends it to the registered email address. When the user clicks the verification link in the email, the device sends a request to the server. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server. The server performs authentication by comparing it with the database.

[2065] Business task automation module

[2066] Setting up and automating business tasks

[2067] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server. The server analyzes the received task information and selects an appropriate AI model. The server trains the AI ​​model based on the task information and generates an automated process. The server sends the progress and results of the generated automated process to the terminal, which then visualizes and presents this information to the user.

[2068] Process monitoring and management module

[2069] Progress monitoring and anomaly detection

[2070] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. When the server detects data that deviates from configured parameters or baseline values, it identifies anomalies and generates alerts. The terminal notifies the user of this alert information, and the user can check the progress and problems in real time on the dashboard screen.

[2071] Feedback collection and analysis module

[2072] Data collection and analysis

[2073] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server analyzes the stored data using AI / machine learning models to extract areas for improvement and patterns. The terminal provides the analysis results to the user, who then incorporates them into the design of new automated processes.

[2074] Emotion Engine Module

[2075] Emotion recognition and feedback

[2076] The server is equipped with an emotion engine that recognizes user emotions. When a user inputs feedback on setting up work tasks or automating processes, the terminal simultaneously collects user emotion data and sends it to the server. The server analyzes this data using the emotion engine to understand the user's emotional state. For example, if a user is feeling stressed, the system uses this information to provide specific feedback to improve the user experience. It also extracts areas for improvement in automated processes based on the emotion data, aiming to improve the overall efficiency of the system.

[2077] Specific example

[2078] For example, when a user automates the creation of sales reports, they enter the work tasks and emotional feedback in the settings screen. The system analyzes this data to generate the optimal automation process and monitors the user's emotional state. If the user is stressed, the server notifies them with appropriate suggestions and areas for improvement. An example of a specific prompt message is: "Please enter the settings for automating the creation of sales reports. Next, please select your recent emotional state from the following options: (1) Relaxed (2) Tired (3) Stressed."

[2079] This allows the system to efficiently automate tasks while considering the user's emotional state, thereby improving the user experience. Furthermore, real-time monitoring and anomaly detection optimize business processes, enabling maximum results with minimal resources.

[2080] The flow of the specific processing in Example 2 will be explained using Figure 13.

[2081] Step 1:

[2082] The user enters their account registration information.

[2083] The user uses a terminal to access the system and enters their username, password, and email address. This becomes the input data. The terminal collects this data, formats it, and sends it to the server. The output is the transmission of account registration information to the server.

[2084] Step 2:

[2085] The device sends registration information to the server.

[2086] The terminal sends the account registration information entered by the user to the server. The server analyzes the received data and saves it to a database. This registers the user's account information in the system. The output is the account information stored in the database.

[2087] Step 3:

[2088] The server generates and sends a verification link.

[2089] The server generates a confirmation link based on the email address stored in the database and sends a confirmation email containing that link to the user. This is data processing based on the input data. The output is the confirmation email sent to the user.

[2090] Step 4:

[2091] The user clicked the confirmation link.

[2092] The user clicks the verification link in the email. The input indicating the link click is sent to the device. The output is the server sending of the request based on the verification link.

[2093] Step 5:

[2094] The device sends the request to the server.

[2095] The device sends a request to the server for the verification link clicked by the user. The server receives this request and updates the user's authentication status. The output is the updated authentication status.

[2096] Step 6:

[2097] The user enters their login information.

[2098] The user enters their username and password on the login screen and sends them to the terminal. This becomes the input data for login. The output is the transmission of the username and password from the terminal to the server.

[2099] Step 7:

[2100] The device sends login information to the server.

[2101] The terminal sends login information to the server, which then authenticates by comparing it against the database. The input is the received username and password, and the output is the authentication result.

[2102] Step 8:

[2103] Users set up work tasks.

[2104] The user uses a terminal to enter the business tasks they want to automate on the settings screen. This setting information becomes the input data. The output is sending this setting information to the server.

[2105] Step 9:

[2106] The device sends configuration information to the server.

[2107] The terminal sends user configuration information to the server. The server analyzes the received information and selects an appropriate AI model. The output is the selected AI model.

[2108] Step 10:

[2109] The server trains the AI ​​model.

[2110] The server trains an AI model based on task information. This process utilizes data processing and machine learning algorithms. The output is the model training result.

[2111] Step 11:

[2112] The server generates an automated process.

[2113] The server generates automated processes using a pre-trained AI model. The input is the training results and task information, and the output is the automated process.

[2114] Step 12:

[2115] The server monitors the progress in real time.

[2116] The server monitors the progress of automated business processes in real time and records logs. Input is progress data, and output is the log records.

[2117] Step 13:

[2118] The server detected an anomaly.

[2119] The server detects data that deviates from the configured parameters and baseline values, and identifies anomalies. Input is real-time progress data, and output is anomaly detection alerts.

[2120] Step 14:

[2121] The device notifies the user of alert information.

[2122] The terminal notifies the user of alert information received from the server. The input is the alert data from the server, and the output is the notification to the user.

[2123] Step 15:

[2124] Users can view this on the dashboard.

[2125] Users can check progress and issues on the dashboard. This allows users to understand the status of their work in real time. The output is the user's understanding and action.

[2126] Step 16:

[2127] The server collects result data and feedback.

[2128] The server collects result data from automated processes and user feedback, and stores it in a database. The input is result data and feedback, and the output is the stored database.

[2129] Step 17:

[2130] The server analyzes the data using AI / machine learning models.

[2131] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns. The input is the accumulated data, and the output is the analysis results.

[2132] Step 18:

[2133] The device provides the user with the analysis results.

[2134] The terminal provides the user with analysis results. The input is analysis result data from the server, and the output is the analysis information obtained by the user.

[2135] Step 19:

[2136] Users design new automation processes.

[2137] The user designs a new automated process based on the analysis results, further improving operational efficiency. The output is the new automated process.

[2138] Step 20:

[2139] Users enter emotional data

[2140] Users input their emotional data when setting up work tasks or providing feedback. This input represents data about the user's psychological state.

[2141] Step 21:

[2142] The device sends emotional data to the server.

[2143] The device sends the collected emotional data to the server. The input is emotional data from the user, and the output is data sent to the server.

[2144] Step 22:

[2145] The server performs emotion recognition and analysis.

[2146] The server uses an emotion engine to analyze emotional data and understand the user's emotional state. The input is emotional data, and the output is the analysis result.

[2147] Step 23:

[2148] The server provides feedback

[2149] The server provides feedback to improve the user experience based on the analysis results. The input is the sentiment analysis result, and the output is the specific feedback content.

[2150] Step 24:

[2151] The server extracts areas for process improvement based on emotional data.

[2152] The server extracts areas for improvement in the automated process based on emotional data. The input is emotional data and analysis results, and the output is the extracted areas for improvement.

[2153] (Application Example 2)

[2154] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[2155] Conventional business automation systems often fail to consider the emotional state of users, leading to user burden and limiting improvements in operational efficiency. Furthermore, they lacked sufficient mechanisms for real-time collection and analysis of customer feedback, making immediate responses necessary for business improvement difficult. Additionally, real-time monitoring of business task progress and immediate responses to anomalies were inadequate. Solving these challenges was essential.

[2156] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving account registration information from a user and storing it in a database, means for sending a confirmation link to an email address based on the account registration information, means for receiving a click of the confirmation link from a user and updating the authentication status, means for receiving the username and password entered by the user from the login screen and authenticating them by comparing them with the information in the database, means for recognizing and analyzing the emotional state of staff, means for providing appropriate feedback and support based on the emotional state of staff, and means for monitoring the progress of automated work tasks in real time and detecting anomalies. This makes it possible to automate tasks while taking into account the emotional state of the user, thereby reducing the burden on staff and achieving improved work efficiency and immediate problem solving.

[2157] "Account registration information" refers to information such as the username, password, and email address that a user provides to access the system.

[2158] A "database" is a system that stores saved account registration information and business task data, and allows for searching and updating of that information.

[2159] A "confirmation link" is a URL sent via email to users to verify the validity of their registration information.

[2160] "Authentication status" is data that indicates the authentication status when a user accesses the system.

[2161] A "username" is a string of characters used to uniquely identify a user when accessing the system.

[2162] A "password" is confidential information used for security authentication when accessing a system.

[2163] "Staff" refers to individuals who perform duties at a physical store.

[2164] "Emotional state" refers to data that indicates the current psychological and emotional condition of a user or staff member.

[2165] "Feedback" refers to information collected from users and customers, such as opinions and evaluations, to help improve the system.

[2166] "Real-time" means that data is updated and processed almost instantly.

[2167] A "dashboard" is a screen that visually displays important information within a user interface.

[2168] An "AI model" is a mathematical model that uses artificial intelligence technology to automate and optimize specific tasks.

[2169] An "abnormal" refers to data or a situation that deviates from established standards or normal patterns.

[2170] A "business task" is a specific work item that requires automation and monitoring in order to perform business efficiently.

[2171] This invention is a system that combines an AI model and emotion recognition technology, primarily to improve operational efficiency in physical stores. Embodiments of this system are described below.

[2172] overview

[2173] This system is designed to reduce the burden on store staff and streamline operations. Using smart glasses, the system allows staff to monitor and manage work processes in real time. It also enables rapid collection of customer feedback and its use in improving operations. Furthermore, by recognizing staff emotional states and providing appropriate support when stressed, it improves employee satisfaction and work efficiency.

[2174] Program Overview

[2175] 1. Account Management:

[2176] Staff members use smart glasses to log into their accounts and check shift information and other details.

[2177] 2. Automation of business tasks:

[2178] Automate business tasks such as inventory management, checking displayed merchandise, and scheduling cleaning.

[2179] 3. Monitoring process progress:

[2180] It monitors the progress of tasks in real time and sends alerts if any anomalies occur.

[2181] 4. Feedback collection and analysis:

[2182] We collect customer feedback, analyze it using AI and machine learning, and identify areas for improvement.

[2183] 5. Emotion recognition:

[2184] Recognize the emotional state of staff and provide support, such as suggesting breaks if they are feeling stressed.

[2185] Hardware and software to be used

[2186] Hardware: Smart glasses (e.g., Google Glass), servers, databases

[2187] Software: Emotion recognition library, task automation library, feedback collection library, dashboard display software

[2188] AI Models: AI / Machine Learning Models for Task Automation and Feedback Analysis

[2189] Processing flow

[2190] This system begins with staff logging into their accounts using smart glasses. Next, an AI model analyzes task information to automate work tasks and monitors progress in real time. Staff emotional states are analyzed by an emotion recognition library, and if stress is detected, an alert recommending a break is sent to the smart glasses. Customer feedback is collected in real time and analyzed by the AI ​​model. This results in specific operational improvement suggestions being presented on a dashboard.

[2191] Specific example

[2192] For example, a store employee arrives for work and logs in using smart glasses. When this employee enters an inventory check task into the system, the system automatically begins checking inventory. If the emotion recognition library detects the employee's emotional state as "stressed" during the process, the smart glasses suggest a break. Additionally, when a customer provides feedback about a product, the system analyzes the feedback in real time and displays suggestions for improving the display method on a dashboard.

[2193] Examples of prompts for generative AI models

[2194] Perform the following emotion recognition to detect the stress levels of your store staff:

[2195] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[2196] Step 1:

[2197] The user puts on smart glasses and enters their account registration information (username, password). The device sends this information to the server. The server retrieves the corresponding user information from the database and performs authentication. If authentication is successful, a login success message is sent back to the device, and the user can log in to the system.

[2198] Step 2:

[2199] The user inputs configuration information for a business task (e.g., inventory management) through smart glasses. The device sends this task information to a server. The server analyzes the received task information, selects and trains an appropriate AI model, and generates an automated process. Once this automated process is generated, its progress and results are sent from the server to the device and presented to the user visually.

[2200] Step 3:

[2201] The server monitors the progress of the generated automated processes in real time. Progress data is sent to the server, and an anomaly detection algorithm detects data that deviates from the standard values. When an anomaly is detected, the server generates an alert and sends it to the terminal. The terminal notifies the user of the alert information and displays the details of the anomaly and countermeasures on the dashboard.

[2202] Step 4:

[2203] The user sends feedback (e.g., customer opinions) through smart glasses. The device sends the feedback data to a server. The server collects this feedback data and stores it in a database. The stored data is analyzed using AI and machine learning models to extract areas for improvement and patterns. The analysis results are sent from the server to the device and provided to the user.

[2204] Step 5:

[2205] The server uses an emotion recognition library to analyze the user's emotional state. The device sends data acquired from the camera and microphone to the server, which analyzes this data to recognize the user's emotional state. If "stress" is detected, the server generates appropriate feedback (e.g., a suggestion to take a break) and sends it to the device. The device then notifies the user of this feedback.

[2206] Step 6:

[2207] Customers provide feedback in-store using smart glasses. The device sends this feedback information to a server. The server analyzes the feedback data and generates improvement suggestions. The generated suggestions are sent to the device and presented to the user visually. This enables rapid business improvements that reflect customer feedback.

[2208] As an example of using a generative AI model, consider the prompt: "Perform the following emotion recognition to detect the stress level of store staff:" Based on this prompt, the AI ​​model performs emotion analysis.

[2209] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[2210] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2211] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[2212] [Fourth Embodiment]

[2213] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[2214] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2215] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2216] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[2217] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[2218] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[2219] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[2220] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[2221] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[2222] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2223] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2224] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[2225] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2226] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, and feedback collection and analysis. The specific processing of each module and its implementation examples are described below.

[2227] Account Management Module

[2228] User registration and authentication

[2229] The user uses a device to access the system and enter their account registration information. The device sends the entered information, such as username, password, and email address, to the server. The server stores the received information in a database and sends a confirmation link to the registered email address. When the user clicks the confirmation link in the email, the device sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server, which then authenticates them by comparing it with the information in the database.

[2230] Business task automation module

[2231] Setting up and automating business tasks

[2232] The user enters the business tasks they want to automate (e.g., report creation or data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model. The server then trains the AI ​​model based on the task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[2233] Process monitoring and management module

[2234] Progress monitoring and anomaly detection

[2235] The server monitors the progress of automated business processes in real time and records logs. The database stores the monitoring data and works in conjunction with anomaly detection algorithms. The server detects data that deviates from configured parameters and baseline values, and generates an alert when an anomaly is found. The terminal notifies the user of this alert information, allowing them to check progress and problems on a real-time dashboard.

[2236] Feedback collection and analysis module

[2237] Data collection and analysis

[2238] The server continuously collects result data from automated processes and user feedback, storing it in a database. The server then analyzes this data using AI / machine learning models to extract areas for improvement and patterns. The analysis results are then provided to users via their terminals, allowing them to incorporate them into the design of new automated processes.

[2239] External sales compatible module

[2240] Service customization and delivery

[2241] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. The customized service package is released, and support is provided to users during its use. The terminal collects feedback from users and sends it to the server.

[2242] Metaverse / Digital Twin Integration Module

[2243] Data transformation and model generation

[2244] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. Based on the converted data, the server generates a metaverse or digital twin model, and the terminal provides the user with a preview of the model. The user then starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[2245] As described above, each module works in conjunction with others to enable efficient automation, management, analysis, and feedback collection and sharing of operations, providing an environment in which companies can operate with minimal resources.

[2246] The following describes the processing flow.

[2247] Account Management Module

[2248] User registration and authentication

[2249] Step 1:

[2250] The user enters their account registration information (username, password, email address).

[2251] Step 2:

[2252] The terminal sends the entered information to the server.

[2253] Step 3:

[2254] The server saves the received information to a database, generates a confirmation link, and sends it to the registered email address.

[2255] Step 4:

[2256] The user clicks the confirmation link sent to their email address.

[2257] Step 5:

[2258] The device sends a request to the server based on clicking the verification link.

[2259] Step 6:

[2260] The server receives the request and updates the authentication status. It then sends a notification to the user that they can log in.

[2261] Step 7:

[2262] The user enters their username and password on the login screen.

[2263] Step 8:

[2264] The terminal sends the entered information to the server.

[2265] Step 9:

[2266] The server verifies the information against the database and performs authentication. If authentication is successful, the user is provided with a dashboard screen.

[2267] Business task automation module

[2268] Setting up and automating business tasks

[2269] Step 1:

[2270] The user enters the business tasks they want to automate on the settings screen.

[2271] Step 2:

[2272] The terminal sends the configured work task information to the server.

[2273] Step 3:

[2274] The server analyzes the task information it receives and selects the appropriate AI model.

[2275] Step 4:

[2276] The server trains an AI model based on task information and generates an automated process.

[2277] Step 5:

[2278] The server sends the progress and results of the automated process it generates to the terminal.

[2279] Step 6:

[2280] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[2281] Process monitoring and management module

[2282] Progress monitoring and anomaly detection

[2283] Step 1:

[2284] The server monitors the progress of automated business processes in real time and records logs.

[2285] Step 2:

[2286] The database stores monitoring data and works in conjunction with anomaly detection algorithms.

[2287] Step 3:

[2288] The server detects data that deviates from the configured parameters and baseline values, and generates an alert when an anomaly is detected.

[2289] Step 4:

[2290] The terminal provides alert information to the user through an interface.

[2291] Step 5:

[2292] Users can check progress and issues in real time on the dashboard screen.

[2293] Feedback collection and analysis module

[2294] Data collection and analysis

[2295] Step 1:

[2296] The server continuously collects result data from automated processes and feedback from users.

[2297] Step 2:

[2298] The database systematically stores all collected data.

[2299] Step 3:

[2300] The server analyzes the accumulated data using AI / machine learning models to extract areas for improvement and patterns.

[2301] Step 4:

[2302] The terminal visualizes the analysis results through an interface that provides them to the user.

[2303] Step 5:

[2304] Users review the analysis results and incorporate them into the design of new automated processes.

[2305] External sales compatible module

[2306] Service customization and delivery

[2307] Step 1:

[2308] The server generates standard service packages for external sales based on accumulated know-how.

[2309] Step 2:

[2310] Users can customize the details of external sales services on the settings screen.

[2311] Step 3:

[2312] The device sends customized configuration information to the server.

[2313] Step 4:

[2314] The server reflects the configuration and generates a new service package.

[2315] Step 5:

[2316] The server releases customized service packages and supports users in use.

[2317] Step 6:

[2318] The device collects feedback from users while they are using it and sends it to the server.

[2319] Metaverse / Digital Twin Integration Module

[2320] Data transformation and model generation

[2321] Step 1:

[2322] The server converts business data into formats for the metaverse or digital twin.

[2323] Step 2:

[2324] The database stores the converted data.

[2325] Step 3:

[2326] The server generates metaverse and digital twin models based on the converted data.

[2327] Step 4:

[2328] The device provides the user with a preview of the model.

[2329] Step 5:

[2330] Users initiate simulations within the metaverse or digital twin to verify business processes.

[2331] Step 6:

[2332] The terminal displays and provides the user with simulation results in real time.

[2333] Step 7:

[2334] The server analyzes the simulation results and accumulates feedback.

[2335] (Example 1)

[2336] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[2337] Traditional systems relied on manual processes for automating, monitoring, and analyzing business processes, leading to decreased operational efficiency. Furthermore, effectively collecting and analyzing user feedback and incorporating it into new process design proved difficult. Additionally, the limited use of virtual spaces and digital replication hindered the easy simulation and verification of business processes.

[2338] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[2339] In this invention, the server includes means for receiving account registration information from a user and storing it in a database; means for sending a confirmation link to an email address based on the account registration information; means for receiving a click of the confirmation link from the user and updating the authentication status; means for converting business data into a virtual space or digital replica format and storing it on an analysis platform; means for generating a virtual space or digital replica model based on the converted data; and means for providing the user with a preview of the generated model and initiating a simulation. This enables automation and efficiency improvements of business operations, real-time monitoring and anomaly detection, and effective collection and analysis of user feedback.

[2340] A "user" refers to anyone who uses this system to perform operations such as registering an account, setting up work tasks, logging in, and providing feedback.

[2341] A "server" refers to a computer system that receives requests from users, stores information in a database, and performs overall system processing, including training and running AI models, generating automated processes, monitoring progress, detecting anomalies, and generating models for virtual spaces and digital replicas.

[2342] "Database" refers to a digital storage system for permanently storing various types of data required by this system, such as account registration information, work task information, progress data, and feedback data.

[2343] A "verification link" refers to a temporary URL sent to the user's email address, which the user clicks to authenticate their account.

[2344] "Authentication status" refers to status information indicating whether the user has clicked the verification link and completed account authentication.

[2345] "Business tasks" refer to business processes, such as report creation and data entry, that users set up in this system.

[2346] A "generative AI model" refers to an artificial intelligence model trained to generate appropriate automation processes based on the user's business task information.

[2347] A "prompt" refers to the text information or question sentences that are input into a generative AI model.

[2348] "Anomaly detection" refers to the process of monitoring the progress of automated business processes and identifying data that deviates from set parameters or baseline values.

[2349] A "virtual space" refers to a digital environment that digitally reproduces the actual physical space for conducting simulations and model verification.

[2350] "Digital replication" refers to a virtual model that reproduces actual business processes and environments as digital data and is used for simulation and verification.

[2351] "Visualized display methods" refer to display methods that use dashboards, graphs, and charts to allow users to intuitively understand the progress of a process and any anomaly information.

[2352] "Analysis techniques" refer to algorithms and methods for analyzing large amounts of data stored in a database and extracting patterns and areas for improvement.

[2353] The above are the key terms and their definitions included in the claims of this system.

[2354] This invention relates to a system that uses AI to efficiently automate business processes and operate with minimal employee resources. This system comprises multiple modules for user account management, automation of business tasks, process monitoring and management, feedback collection and analysis, external sales support, and metaverse / digital twin integration. The specific operation of each module is described in detail below.

[2355] Account Management Module

[2356] User registration and authentication

[2357] The user accesses the system using a device and enters account registration information such as username, password, and email address. The device sends this information to the server, which stores the received information in a database (e.g., MySQL). The server then sends a verification link to the registered email address (e.g., using an email service). When the user clicks the verification link, the device sends a request to the server based on that click. The server receives the request, updates the authentication status, and notifies the user that they have logged in. The user enters their username and password on the login screen, and the device sends this information to the server. The server authenticates the user by comparing it with the information in the database.

[2358] Business task automation module

[2359] Setting up and automating business tasks

[2360] The user enters the business tasks they want to automate (e.g., report creation, data entry) on a settings screen. The terminal sends this settings information to the server, which analyzes the received task information and selects an appropriate AI model (e.g., TensorFlow). The server then trains the AI ​​model based on this task information and generates the automated process. The terminal provides the user with a dashboard that visualizes the progress and results of the process, allowing them to easily check the status of their work.

[2361] Process monitoring and management module

[2362] Progress monitoring and anomaly detection

[2363] The server monitors the progress of automated business processes in real time and logs the data (e.g., using a monitoring tool). The database stores the monitoring data and works with anomaly detection algorithms (e.g., Scikit-learn). The server detects data that deviates from configured parameters or baseline values ​​and generates an alert when an anomaly is found. The terminal notifies the user of this alert information and allows them to check progress and problems on a real-time dashboard.

[2364] Feedback collection and analysis module

[2365] Data collection and analysis

[2366] The server continuously collects result data from automated processes and user feedback, storing it in a database (e.g., PostgreSQL). The server then analyzes the stored data using an AI / machine learning model (e.g., PyTorch) to extract areas for improvement and patterns. The analysis results are provided to the user via a terminal, allowing the user to incorporate them into the design of new automated processes.

[2367] External sales compatible module

[2368] Service customization and delivery

[2369] The server generates standard service packages for external sales based on accumulated know-how. Users customize the details of the external sales service on the settings screen, and the terminal sends this setting information to the server. The server reflects the customizations and generates a new service package. It releases the customized service package and supports users who are using it. The terminal collects feedback from users and sends it to the server.

[2370] Metaverse / Digital Twin Integration Module

[2371] Data transformation and model generation

[2372] The server converts business data into a format for the metaverse or digital twin and stores that data in a database. The server generates a model of the metaverse or digital twin based on the converted data (e.g., using Unity). The terminal provides the user with a preview of the model. The user starts a simulation within the metaverse or digital twin to verify the business process. The terminal displays the simulation results in real time, and the server analyzes the results and accumulates feedback.

[2373] Examples of specific cases and prompt statements

[2374] For example, a "mail service" can be used to send a confirmation link to a registered email address, and "MySQL" can be used as the database to store user information. Furthermore, "TensorFlow" or "PyTorch" can be used to train AI models. "Visualization tools" can be used to monitor the situation on the dashboard, and "Scikit-learn" can be utilized for anomaly detection. "3D modeling tools" can be used to generate models in virtual spaces or digital replicas.

[2375] Prompt example 1: "Send a confirmation email for new user registration and update the authentication status."

[2376] Prompt example 2: "Use a generative AI model to automate report generation tasks and visualize progress."

[2377] Prompt example 3: "Apply the anomaly detection algorithm and display an alert if an anomaly occurs."

[2378] Prompt example 4: "We will extract patterns through data analysis and identify areas for improvement."

[2379] As described above, each module works together through detailed and specific procedures, providing an environment where companies can efficiently automate their operations and operate with minimal resources.

[2380] The flow of the specific processing in Example 1 will be explained using Figure 11.

[2381] Account Management Module

[2382] User registration and authentication

[2383] Step 1:

[2384] The user accesses the system using a terminal and enters account registration information such as username, password, and email address.

[2385] Input: Username, Password, Email Address

[2386] Output: The request the terminal sends to the server with this information.

[2387] Step 2:

[2388] The device sends the entered account registration information to the server.

[2389] Input: Username, password, and email address entered by the user.

[2390] Output: Data packets to be sent to the server

[2391] Step 3:

[2392] The server parses the information it receives and saves it to a database (for example, MySQL).

[2393] Input: Account registration information sent from the device

[2394] Output: Saving information to the database using SQL INSERT statements

[2395] Specific operation: Execute SQL queries using the Python sqlite3 module or MySQL client.

[2396] Step 4:

[2397] The server will send a confirmation link to the registered email address.

[2398] Input: User's email address

[2399] Output: Email containing a confirmation link

[2400] Specific operation: Use an email service (e.g., SendGrid) to send emails via API.

[2401] Step 5:

[2402] The user clicks the verification link, and the device sends an authentication request to the server.

[2403] Input: Link access via user click

[2404] Output: Request to the server

[2405] Specific action: The device sends an HTTP GET request to the server.

[2406] Step 6:

[2407] The server receives the request and updates the authentication status.

[2408] Input: Authentication request to the server

[2409] Output: Authentication status update

[2410] Specific action: Update the status field of the corresponding user in the database.

[2411] Step 7:

[2412] The user enters their username and password on the login screen, and the device sends that information to the server.

[2413] Input: Username, Password

[2414] Output: Login request to the server

[2415] Specific action: Send data using the terminal's form submission function.

[2416] Step 8:

[2417] The server compares the information with that in the database and performs authentication.

[2418] Input: Username, Password

[2419] Output: Authentication success return code or error message

[2420] Specific operation: Database matching is performed using an SQL SELECT statement.

[2421] ---

[2422] Business task automation module

[2423] Setting up and automating business tasks

[2424] Step 1:

[2425] The user enters the business tasks they want to automate (e.g., report creation, data entry) on the settings screen.

[2426] Input: Detailed information about the task

[2427] Output: The request the terminal sends to the server with this information.

[2428] Step 2:

[2429] The device sends configuration information to the server.

[2430] Input: Detailed information about the task

[2431] Output: Request for configuration information from the server

[2432] Step 3:

[2433] The server analyzes the task information and selects an appropriate AI model (e.g., TensorFlow).

[2434] Input: Detailed information about the task

[2435] Output: Selected AI models

[2436] Specific action: Execute the AI ​​model selection algorithm.

[2437] Step 4:

[2438] The server trains the AI ​​model and generates automated processes.

[2439] Input: Detailed information on the business task, selected AI model

[2440] Output: Trained model, generated automation process

[2441] Specific operation: Train the model using the TensorFlow library.

[2442] Step 5:

[2443] The terminal provides users with a dashboard that visualizes the progress and results of the process.

[2444] Input: Progress data, result data

[2445] Output: Visualized dashboard

[2446] Specific actions: Use visualization tools (e.g., Grafana) to display data in graphs and charts.

[2447] ---

[2448] Process monitoring and management module

[2449] Progress monitoring and anomaly detection

[2450] Step 1:

[2451] The server monitors the pro...

Claims

1. A means of receiving account registration information from users and storing it in a database, A means of sending a verification link to an email address based on the account registration information, A means of receiving a confirmation link click from the user and updating the authentication status, A means of receiving the username and password entered by the user on the login screen, and authenticating them by comparing them with the information in the database, A system that includes this.

2. A means of receiving business task configuration information and generating an automated process using an AI model, A means for monitoring the progress of the generated automated process in real time and detecting anomalies, A means of providing users with a dashboard that notifies them of process progress and abnormal information, The system according to claim 1, including the following:

3. A means of collecting result data from automated business processes and user feedback, and storing it in a database, A method for analyzing accumulated data using AI / machine learning models to extract areas for improvement and patterns, A means of providing users with analysis results and incorporating them into the design of new automation processes, The system according to claim 1, including the following:

Images