System

Abstract

To provide a system.SOLUTION: A system includes: means for collecting an operation log of a user; means for mining a task from the collected operation log; means for generating an artificial intelligence model on the basis of the mined task; means for analyzing a feeling state of the user who is operating using a feeling engine to feed back an analysis result to the artificial intelligence model; and means for proposing a work procedure on the basis of the operation log and the feeling state.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Currently, many tasks that employees perform at work are linked to PC operations. While there are efforts to use task mining tools to analyze operation logs and use the results to improve work, there is a need for a more efficient way to navigate users' PC operations and tasks. [Means for solving the problem]

[0005] This invention collects users' PC operation logs and mines tasks from the collected PC operation logs. It then generates AI based on the mined tasks and provides a system that uses the generated AI to navigate users' PC operations and tasks. Furthermore, the AI ​​learns and predicts the user's PC operation patterns, determines the priority of the user's tasks, and navigates the tasks based on the priority. This makes it possible to improve the efficiency of users' PC operations and tasks. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12]FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Form Example 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] In one embodiment of the present invention, software is installed on the user's PC to collect the user's PC operation log, and the software records the user's operations in real time. This software collects information such as the type of operations performed by the user, the order of operations, and the time taken for each operation.

[0029] "Example 2"

[0030] As a means of mining tasks from collected PC operation logs, an algorithm is used to analyze the collected operation logs and find patterns of the same operations. For example, if a specific operation is repeated in a certain order, the combination of operations is recognized as a single task. "Example 3"

[0031] As a means of generating AI based on the mined tasks, we design an AI model that learns the mined task patterns. When a new operation log is input, this AI model predicts the next operation that is likely to be performed based on the task patterns it has learned in the past.

[0032] "Example 4"

[0033] As a means for the generated AI to navigate the user's PC operations and tasks, an interface is provided that presents the user with the operations predicted by the AI ​​model. For example, when the user starts a specific operation, the next operation predicted by the AI ​​model is displayed on the screen, allowing the user to select and execute that operation.

[0034] The processing flow of each embodiment will be described below.

[0035] "Example 1"

[0036] Step 1: Install software that runs on the user's PC and records user actions in real time.

[0037] Step 2: When a user operates the PC, the software records the operation. The recorded information includes the type of operation, the order of operations, and the time taken for the operation. Step 3: The recorded operation log is periodically collected and sent to the server.

[0038] "Example 2"

[0039] Step 1: The operation log sent to the server is analyzed by the task mining algorithm.

[0040] Step 2: The algorithm finds patterns of the same operations in the operation log. For example, if a specific operation is repeated in a certain order, it recognizes the combination of operations as a single task.

[0041] Step 3: The recognized tasks are stored in a database.

[0042] "Example 3"

[0043] Step 1: Use the tasks stored in the database as training data for the AI ​​model.

[0044] Step 2: The AI ​​model learns patterns for the task based on the training data.

[0045] Step 3: When a new operation log is input, the AI ​​model predicts the next likely operation based on the task patterns it has learned in the past.

[0046] "Example 4"

[0047] Step 1: Create an interface that presents the user with the operations predicted by the AI ​​model.

[0048] Step 2: When the user initiates a specific action, the AI ​​model displays the next action predicted by the user on the screen.

[0049] Step 3: The user can select from the displayed actions and perform the action.

[0050] Example 1

[0051] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0052] Conventional user operation log collection systems do not collect operation logs in real time, making it difficult to accurately grasp user operation patterns. Furthermore, analysis of collected data is often done manually, requiring efficient analysis. Furthermore, report generation based on analysis results is not automated, making it impossible to quickly provide useful information to users.

[0053] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0054] In this invention, the server includes means for installing software on the user's information processing device, means for the installed software to record user operations in real time, means for temporarily storing the recorded data in the information processing device, means for transmitting data to the server at regular time intervals, means for the server to analyze the received data, and means for the server to generate a report based on the analysis results. This makes it possible to collect user operation logs in real time, analyze them efficiently, and quickly provide useful reports.

[0055] "Information processing device" refers to devices such as computers and PCs operated by users.

[0056] "Software" refers to a program that is installed on an information processing device and records user operations.

[0057] "Real-time" means that the data is recorded immediately at the moment the user performs an action.

[0058] "Recorded data" refers to information collected as a user's operation log.

[0059] "Temporarily storing" refers to temporarily storing the recorded data in a storage device within the information processing device.

[0060] "Regular time intervals" means that the data is sent to the server periodically, for example every five minutes.

[0061] "Server" refers to a central computer system that analyzes the data it receives and generates a report of the analysis results.

[0062] "Analyzing" refers to processing collected data to identify user patterns and trends.

[0063] "Report" refers to documents or data generated based on analysis results that show user operation patterns and trends.

[0064] MODE FOR CARRYING OUT THE INVENTION

[0065] This invention is a system that installs software on a user's information processing device, records the user's operations in real time, temporarily stores the recorded data in the information processing device, transmits the data to a server at regular intervals, analyzes the data received by the server, and generates a report of the analysis results.

[0066] 1. Software Installation

[0067] Users use the provided installer to install the software on their information processing devices. This software is compatible with operating systems such as Windows and macOS (registered trademark).

[0068] 2. Real-time recording of operations

[0069] The software installed on the device runs in the background and records user actions in real time, specifically:

[0070] Keyboard input (type and order of keys)

[0071] Mouse operations (click, drag, location information)

[0072] A timestamp for each operation

[0073] 3. Temporary storage of data

[0074] The software temporarily saves the recorded operation data to the device's local storage in a general-purpose data format such as JSON or CSV.

[0075] 4. Data transmission

[0076] The device sends the recorded data to the server at regular intervals (for example, every 5 minutes). Data transmission is secure using the HTTPS protocol.

[0077] 5. Data Analysis

[0078] The server analyzes the received data using Python's Pandas library and machine learning models (e.g., scikit-learn and TENSORFLOW (registered trademark)). The purpose of the analysis is to identify user operation patterns.

[0079] 6. Generate reports

[0080] The server generates reports based on the analysis results, including information on user operation patterns, frequency, and the time it takes to perform specific operations, and the reports are displayed on a dashboard accessible to the user.

[0081] Specific examples

[0082] For example, if a user types a sentence in a text editor and then performs a search in a web browser, the specific behavior is as follows:

[0083] 09:00:01 - Start typing on the keyboard (text editor)

[0084] 09:00:15 - End of keyboard input

[0085] 09:00:20 - Mouse click (web browser icon)

[0086] 09:00:25 - Start typing on keyboard (search bar)

[0087] 09:00:30 - End of keyboard input

[0088] The recorded data is saved in local storage as operation log_20231010.json and sent to the server at 09:05:00. The server analyzes the received data and identifies patterns, such as "users tend to type text in a text editor at 9:00 AM and then perform searches in a web browser." A report is generated based on the analysis results and displayed as "Operation Patterns at 9:00 AM" on a dashboard accessible to users.

[0089] Prompt Sentence Examples

[0090] Develop software that collects users' PC operation logs and records the type, sequence, and time of operations. Include a function to send the collected data to a server and generate reports that analyze user operation patterns. Include a scenario in which input in a text editor and searches in a web browser are recorded as specific operation examples.

[0091] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0092] Step 1:

[0093] The user installs the software on their information processing device using the provided installer, and once the installation is complete, the software starts running in the background.

[0094] Input: Installer file

[0095] Output: Installed software

[0096] Step 2:

[0097] The software installed on the device records user actions in real time, including keyboard input, mouse movements, and a timestamp for each action.

[0098] Input: User actions (keyboard input, mouse operations)

[0099] Output: Operation log data (key type and sequence, click position, timestamp)

[0100] Step 3:

[0101] The software temporarily saves the recorded operation data to the device's local storage in a general-purpose data format such as JSON or CSV.

[0102] Input: Operation log data

[0103] Output: Data file saved in local storage (e.g., Operation Log_20231010.json)

[0104] Step 4:

[0105] The device sends the recorded data to the server at regular intervals (for example, every 5 minutes). Data transmission is secure using the HTTPS protocol.

[0106] Input: Data files stored in local storage

[0107] Output: Data sent to the server

[0108] Step 5:

[0109] The server analyzes the received data using Python's Pandas library and machine learning models (e.g., scikit-learn and TensorFlow). The goal of the analysis is to identify user behavior patterns.

[0110] Input: Data sent to the server

[0111] Output: Analysis results (user operation patterns and trends)

[0112] Step 6:

[0113] The server generates reports based on the analysis results, including information on user operation patterns, frequency, and the time it takes to perform specific operations, and the reports are displayed on a dashboard accessible to the user.

[0114] Input: Analysis results

[0115] Output: Generated report (displayed on the dashboard)

[0116] Specific examples

[0117] For example, if a user types a sentence in a text editor and then performs a search in a web browser, the specific behavior is as follows:

[0118] 09:00:01 - Start typing on the keyboard (text editor)

[0119] 09:00:15 - End of keyboard input

[0120] 09:00:20 - Mouse click (web browser icon)

[0121] 09:00:25 - Start typing on keyboard (search bar)

[0122] 09:00:30 - End of keyboard input

[0123] The recorded data is saved in local storage as operation log_20231010.json and sent to the server at 09:05:00. The server analyzes the received data and identifies patterns, such as "users tend to type text in a text editor at 9:00 AM and then perform searches in a web browser." A report is generated based on the analysis results and displayed as "Operation Patterns at 9:00 AM" on a dashboard accessible to users.

[0124] (Application example 1)

[0125] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0126] Conventional information processing device operation log collection systems were able to collect user operation logs, mine tasks, and generate artificial intelligence, but they lacked the functionality to collect industrial machine operation logs and provide data to optimize operation efficiency. As a result, data collection and analysis to improve the operational efficiency of industrial machines were not being carried out sufficiently.

[0127] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0128] In this invention, the server includes means for collecting user operation logs of information processing devices, means for mining tasks from the collected operation logs of information processing devices, means for generating artificial intelligence based on the mined tasks, means for navigating user operations of the information processing devices and tasks using the generated artificial intelligence, means for collecting operation logs of industrial machines and recording the type, sequence, and time of operations, and means for analyzing the recorded operation logs and providing data for optimizing operation efficiency, thereby making it possible to collect operation logs of information processing devices and optimize the operation efficiency of industrial machines.

[0129] An "information processing device" is an electronic device such as a computer or personal computer that is operated by a user.

[0130] An "operation log" is a record of the type, order, time, etc. of operations performed by a user on an information processing device.

[0131] A "task" is a series of operations or tasks that a user performs on an information processing device.

[0132] "Mining" is the process of extracting useful information and patterns from collected data.

[0133] "Artificial intelligence" refers to programs and systems that learn from collected data and make predictions and navigate.

[0134] "Navigating" means guiding and assisting users with operations and tasks.

[0135] "Industrial machinery" refers to robots and mechanical devices used in factories and manufacturing sites.

[0136] "Type of operation" refers to the specific movement or action that an industrial machine performs.

[0137] "Sequence of operations" is the order in which a series of operations are performed by industrial machinery.

[0138] "Time of operation" is the time it takes an industrial machine to perform a particular operation.

[0139] "Analysis" means examining collected data in detail to find meaning and patterns.

[0140] "Optimization" means making adjustments or improvements to maximize the efficiency of an operation.

[0141] A system for carrying out the present invention collects user operation logs of information processing devices and analyzes the operation logs of industrial machines to optimize operational efficiency. A specific embodiment of the system is described below.

[0142] System Configuration

[0143] The system consists of the following main components:

[0144] 1. User's information processing device: Electronic equipment such as a computer or personal computer operated by a user.

[0145] 2. Industrial machinery: Robots and mechanical devices used in factories and manufacturing sites.

[0146] 3. Server: A central processing unit that collects data, analyzes it, and generates artificial intelligence.

[0147] Program processing

[0148] The server processes the data in the following steps:

[0149] 1. Collecting operation logs of information processing devices:

[0150] Software installed on the user's information processing device collects operation logs in real time, recording data such as the type, order, and time of operations.

[0151] 2. Mining tasks:

[0152] The collected operation logs are mined to identify the tasks performed by the user, thereby extracting the user's operation patterns and task flow.

[0153] 3. Artificial Intelligence Generation:

[0154] Based on the mined tasks, an artificial intelligence is generated, which has the ability to learn and predict user operation patterns.

[0155] 4. Collecting industrial machine operation logs:

[0156] Sensors and software installed on industrial machines collect operation logs, including the type, sequence and time of operations.

[0157] 5. Analysis of operation logs:

[0158] Analyzes collected industrial machine operation logs and provides data to optimize operational efficiency. This analysis includes evaluation of operation patterns and efficiency.

[0159] Hardware and software used

[0160] Hardware: Information processing equipment (computers, personal computers), industrial machinery (robots, machinery), servers

[0161] Software: Log collection software, data mining software, artificial intelligence generation software, analysis software

[0162] Specific names: Python, JSON library

[0163] Specific examples

[0164] For example, when a user creates and saves a document on an information processing device, the operation log is collected. Next, when an industrial machine moves forward and lifts a box, the operation log is also collected. These logs are sent to the server and analyzed.

[0165] Prompt Sentence Examples

[0166] "Write a Python program to collect operation logs from a factory robot and record the type, order, and time of operations. Operation types include move, pick up, etc. Save the logs in JSON format."

[0167] In this way, the collection of operation logs of the information processing device and the optimization of the operational efficiency of the industrial machine are realized.

[0168] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0169] Step 1:

[0170] Software installed on the user's information processing device collects operation logs in real time. The input is user operations (e.g., clicks, keyboard input), and the output is log data that includes the type, order, and time of the operations. This log data is saved in JSON format.

[0171] Step 2:

[0172] The server receives the collected operation logs and stores them in a database. The input is the log data sent from the information processing device, and the output is the operation logs stored in the database. The database records the type, order, and time of operations.

[0173] Step 3:

[0174] The server mines tasks from the saved operation logs. The input is the operation logs stored in the database, and the output is mined task data. The task data includes the user's operation patterns and task flow.

[0175] Step 4:

[0176] The server generates an artificial intelligence model based on the mined task data. The task data is used as input, and an artificial intelligence model that learns and predicts user operation patterns is generated as output. This artificial intelligence model is built using a generative AI model.

[0177] Step 5:

[0178] Sensors and software installed on industrial machines collect operation logs. The input is the operation of the industrial machine (e.g., moving, picking up), and the output is log data that includes the type, sequence, and time of the operation. This log data is also saved in JSON format.

[0179] Step 6:

[0180] The server receives collected operation logs of industrial machines and stores them in a database. The input is the log data sent from the industrial machines, and the output is the operation logs stored in the database. The database records the type, sequence, and time of operations.

[0181] Step 7:

[0182] The server analyzes the saved operation logs of industrial machines. The input is the operation logs stored in the database, and the output is data generated to optimize the efficiency of the operation. This analysis includes evaluation of operation patterns and efficiency.

[0183] Step 8:

[0184] The server provides feedback based on the analysis results to optimize the operational efficiency of the industrial machinery. The analysis results are input, and optimized operating procedures and improvement suggestions are generated as output. This feedback is sent to the control system of the industrial machinery.

[0185] Through the above steps, the collection of operation logs of information processing devices and the optimization of the operational efficiency of industrial machines are realized.

[0186] Example 2

[0187] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0188] Conventional systems for analyzing information processing device operation logs have had difficulty efficiently detecting user operation patterns and defining them as tasks. Furthermore, they were unable to navigate user operations and tasks using AI models generated based on the detected tasks. This resulted in the issue of being unable to improve user work efficiency.

[0189] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0190] In this invention, the server includes means for collecting information processing device operation logs of users, means for preprocessing the collected information processing device operation logs, means for detecting the same operation patterns from the preprocessed operation logs, means for defining and saving tasks based on the detected operation patterns, means for visualizing the defined tasks and generating reports, means for creating a generative AI model based on the generated reports, and means for navigating the user's information processing device operations and tasks using the generated AI model. This makes it possible to efficiently detect the user's operation patterns and define them as tasks, and further to navigate the user's operations and tasks using the generated AI model, thereby improving the user's work efficiency.

[0191] The "information processing device operation log" is a record of operations performed by a user on an information processing device.

[0192] "Preprocessing" is the process of cleaning and organizing collected data to convert it into a form suitable for analysis.

[0193] An "operation pattern" is a sequence or combination of a series of operations that a user repeatedly performs on an information processing device.

[0194] A "task" is a set of operations with a specific purpose that is defined based on the detected operation pattern.

[0195] "Visualization" is the presentation of data or information in a visual format such as a graph or chart.

[0196] A "report" is a document or data that summarizes analysis results and statistical information.

[0197] A "generative AI model" is an artificial intelligence model created using machine learning algorithms based on collected data.

[0198] "Navigating" means guiding and assisting users with operations and tasks.

[0199] This invention is a system that collects and analyzes user operation logs of information processing devices to detect user operation patterns and define them as tasks. Furthermore, it can use the generated AI model to navigate user operations and tasks.

[0200] The server installs log collection software on the terminal to collect the user's information processing device operation log. This software records user operations (clicks, keystrokes, application startup / exit, etc.) in real time and periodically sends them to the server.

[0201] The server preprocesses the collected operation logs. Specifically, it uses data analysis software such as Python or R to clean the log data (removing noise and filling in missing data) and convert it into a format suitable for analysis. For example, it organizes the order of operations based on timestamps and classifies them by type of operation.

[0202] The server then applies a pattern recognition algorithm to the preprocessed operation log. Specifically, it uses a machine learning algorithm (e.g., k-means clustering or sequence mining algorithm) to detect patterns in which the same operations are repeated in a certain order, thereby identifying combinations of operations that users frequently perform.

[0203] The server defines tasks based on the detected operation patterns and stores them in a database. Specifically, it assigns a task name to each pattern and records details of the related operations (type, order, frequency, etc.). For example, a task called "check email and open files every day" might be defined.

[0204] The server also visualizes the defined tasks and generates reports, displaying statistical information such as task execution frequency and time period in graphs and charts for easy understanding by users and administrators. The generated reports are available via a web interface and in PDF format.

[0205] Based on the generated reports, the server creates a generative AI model that can learn and predict user operation patterns. For example, if a user tends to perform a specific operation at a specific time, the model can predict the operation in advance and notify the user.

[0206] For example, if a user checks email at a specific time every day, then opens a specific file, and then launches a specific piece of software, the server will recognize these operations as a single task, which is defined as "checking email and opening a file every day."

[0207] Examples of prompts to input to a generative AI model might include:

[0208] "Analyze the patterns of operations that users perform daily and identify them as tasks. For example, if operations such as checking email, opening files, and launching software are repeated, define them as one task."

[0209] In this way, the server analyzes the collected operation logs, and realizes a system that recognizes and visualizes the user's repeated operations as tasks, thereby improving the user's work efficiency.

[0210] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0211] Step 1:

[0212] Collecting operation logs

[0213] The server collects the user's operation logs for the information processing device. The log collection software installed on the terminal records the user's operations (clicks, keystrokes, application startup / exit, etc.) in real time and periodically sends them to the server.

[0214] Input: User operation data

[0215] Output: Collected operation log data

[0216] Step 2:

[0217] Preprocessing of operation logs

[0218] The server preprocesses the collected operation logs. Specifically, it uses data analysis software such as Python or R to clean the log data (removing noise and filling in missing data) and convert it into a format suitable for analysis. For example, it organizes the order of operations based on timestamps and classifies them by type of operation.

[0219] Input: Collected operation log data

[0220] Output: Preprocessed audit log data

[0221] Step 3:

[0222] Applying pattern recognition algorithms

[0223] The server applies a pattern recognition algorithm to the preprocessed operation log. Specifically, it uses a machine learning algorithm (e.g., k-means clustering or sequence mining algorithm) to detect patterns in which the same operations are repeated in a certain order. This allows it to identify combinations of operations that users frequently perform.

[0224] Input: Preprocessed operation log data

[0225] Output: Detected operation pattern

[0226] Step 4:

[0227] Define and save the task

[0228] The server defines tasks based on the detected operation patterns and stores them in a database. Specifically, it assigns a task name to each pattern and records details of the related operations (type, order, frequency, etc.). For example, a task called "check email and open files every day" might be defined.

[0229] Input: Detected operation pattern

[0230] Output: Defined task data

[0231] Step 5:

[0232] Task visualization and reporting

[0233] The server visualizes the defined tasks and generates reports. Specifically, it displays statistical information such as task execution frequency and time period in graphs and charts, making it easy for users and administrators to understand. The generated reports are available via a web interface or in PDF format.

[0234] Input: Defined task data

[0235] Output: Visualized report

[0236] Step 6:

[0237] Creating generative AI models

[0238] The server creates a generative AI model based on the generated report. This AI model can learn and predict user operation patterns. For example, if a user tends to perform a specific operation at a specific time, it can predict that operation in advance and notify the user.

[0239] Input: Visualized report

[0240] Output: The generated AI model

[0241] Step 7:

[0242] Navigate user actions and tasks

[0243] The server uses the generated AI model to navigate the user's operations and tasks. Specifically, it predicts the operations and tasks the user needs to perform in advance and provides notifications and guidance at the appropriate time, thereby improving the user's work efficiency.

[0244] Input: Generated AI model

[0245] Output: The operation or task that was navigated to

[0246] (Application example 2)

[0247] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0248] In conventional systems, it was difficult to find efficient task patterns simply by collecting user operation logs. Furthermore, there was a lack of means to automatically analyze operation patterns and recognize them as optimal tasks, making it impossible to improve user work efficiency. Furthermore, automated systems such as factory robots are also required to analyze operation logs to find efficient tasks, but current technology is not sufficient to address this need.

[0249] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0250] In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating artificial intelligence based on the mined tasks, means for navigating user operations and tasks using the generated artificial intelligence, means for extracting features from the operation logs and finding operation patterns using a clustering algorithm, and means for extracting the found operation patterns as task patterns. This makes it possible to efficiently analyze user operation logs and automatically find optimal task patterns.

[0251] "User" means the human or end user who operates the system.

[0252] An "operation log" is a record of a series of operations performed by a user on a system.

[0253] A "task" is a set of operations performed to achieve a specific goal.

[0254] "Mining" is the process of extracting useful information and patterns from data.

[0255] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[0256] "Navigating" means guiding a user through an operation or task.

[0257] A "feature" is an attribute or variable that is used to find specific patterns or trends in data.

[0258] A "clustering algorithm" is a computational method for dividing data into groups based on similarity.

[0259] An "operation pattern" is a tendency or regularity in the repetition of a series of operations performed by a user.

[0260] A "task pattern" is a set of operations required to achieve a specific goal.

[0261] The system for carrying out this invention collects and analyzes user operation logs to find and optimize efficient task patterns. A specific embodiment of this system is described below.

[0262] System Configuration

[0263] The system consists of the following main components:

[0264] 1. Operation log collection module: Collects user operation logs.

[0265] 2. Task mining module: Mines tasks from the collected operation logs.

[0266] 3. Artificial intelligence generation module: Generates artificial intelligence based on the mined tasks.

[0267] 4. Navigation module: The generated AI navigates the user through operations and tasks.

[0268] 5. Feature extraction module: Extracts features from the operation log.

[0269] 6. Clustering module: Uses a clustering algorithm to find operation patterns.

[0270] 7. Task pattern extraction module: Extracts the discovered operation patterns as task patterns.

[0271] Hardware and software used

[0272] Hardware: Factory robots, sensors for data collection

[0273] Software: Python, Pandas, Scikit-learn

[0274] Data processing and calculation

[0275] 1. Operation log collection: The operation log collection module records user operations in real time and stores them in a database.

[0276] 2. Feature Extraction: The feature extraction module extracts features such as operation type, duration, and parameters from the operation log.

[0277] 3. Clustering: The clustering module clusters the operation patterns using a clustering algorithm such as DBSCAN.

[0278] 4. Task Pattern Extraction: The task pattern extraction module extracts task patterns from the clustering results and recognizes them as efficient tasks.

[0279] 5. AI generation: The AI ​​generation module generates AI based on the extracted task patterns.

[0280] 6. Navigation: The navigation module uses the generated artificial intelligence to navigate the user through operations and tasks.

[0281] Specific examples

[0282] For example, in a part assembly operation in a factory, the operation log collection module collects the operation logs of a robot. The feature extraction module extracts the operation type, duration, and parameters from the operation logs, and the clustering module uses these features to cluster the operation patterns. The task pattern extraction module extracts efficient task patterns from the clustering results, and the artificial intelligence generation module generates artificial intelligence based on these. Finally, the navigation module uses the generated artificial intelligence to navigate the robot's operations and tasks.

[0283] Example prompt sentence:

[0284] Write a program to analyze the operation log data of a factory robot and find efficient task patterns. The operation log contains operation types, durations, and parameters. Use the DBSCAN clustering algorithm to cluster the operation patterns and extract the operations that belong to each cluster as task patterns.

[0285] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0286] Step 1:

[0287] Collecting operation logs

[0288] The server collects user operation logs in real time. Specifically, it records the operations performed by users on the system (clicks, keyboard input, scrolling, etc.) and stores them in a database. The input is the user operation data, and the output is the operation log stored in the database.

[0289] Step 2:

[0290] Feature extraction

[0291] The server extracts features from the collected operation logs. Specifically, it extracts features such as operation type, duration, and parameters for each entry in the operation log. The input is the operation log data, and the output is a dataset containing the features.

[0292] Step 3:

[0293] Clustering

[0294] The server clusters operation patterns using the features. Specifically, it uses a clustering algorithm such as DBSCAN to group similar operations. The input is a feature dataset, and the output is a dataset with cluster labels.

[0295] Step 4:

[0296] Task Pattern Extraction

[0297] The server extracts task patterns from the clustering results. Specifically, it analyzes the operations belonging to each cluster and recognizes them as efficient task patterns. The input is a dataset with cluster labels, and the output is task patterns.

[0298] Step 5:

[0299] Artificial Intelligence Generation

[0300] The server generates artificial intelligence based on the extracted task patterns. Specifically, it uses the task patterns as learning data to train a generative AI model. The input is the task patterns, and the output is the trained artificial intelligence model.

[0301] Step 6:

[0302] Navigation

[0303] The server uses the generated AI to navigate the user's operations and tasks. Specifically, it monitors the user's current operation status and guides the user to the next operation based on the optimal task pattern. The input is the user's current operation data and the AI ​​model, and the output is navigation instructions.

[0304] Example 3

[0305] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0306] In conventional systems, simply collecting user operation logs made it difficult to predict subsequent operations and provide appropriate navigation to users. Furthermore, the AI ​​model for predicting the next operation a user should perform was insufficiently trained, making it impossible to improve user operation efficiency. Furthermore, the system lacked the ability to determine the user's task priorities and navigate accordingly, resulting in a decrease in user work efficiency.

[0307] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0308] In this invention, the server includes means for collecting user operation logs, means for saving the collected operation logs, means for training an AI model using the saved operation logs, means for collecting new operation logs in real time, means for predicting the next operation, means for presenting the predicted results to the user, and means for the user to select and execute the presented predicted results. This makes it possible to predict the next operation with high accuracy based on the user's operation log and provide appropriate navigation to the user. Furthermore, by determining the priority of the user's tasks and navigating based on that, it is possible to significantly improve the user's work efficiency.

[0309] A "user operation log" is data that records a series of operations performed by a user in chronological order.

[0310] "Means of collection" refers to functions and devices for monitoring and recording user operation logs in real time.

[0311] The "storing means" refers to a function or device for storing the collected operation logs in a storage device such as a database.

[0312] An "AI model" is a model that uses machine learning algorithms to learn user operation patterns and predict the next operation.

[0313] "Training means" refers to functions or devices that train an AI model using saved operation logs.

[0314] "Means for collecting data in real time" refers to functions or devices that instantly record new operations performed by the user and send them to a server.

[0315] "Prediction means" refers to functions or devices that use an AI model to estimate the next operation that is likely to be performed.

[0316] "Presentation means" refers to a function or device for visually or audibly notifying the user of the prediction results.

[0317] The "means for selecting and executing" refers to a function or device that allows a user to select a presented prediction result and execute an operation based on that selection.

[0318] MODE FOR CARRYING OUT THE INVENTION

[0319] This invention is a system that collects user operation logs, predicts the next operation using an AI model, and presents it to the user. Specific embodiments of this system are described below.

[0320] System configuration

[0321] The system includes a terminal that collects user operation logs, a server that stores the collected operation logs and trains the AI ​​model, and an interface that presents prediction results to the user.

[0322] Hardware and software used

[0323] Hardware:

[0324] Device: PC or smartphone for recording user actions in real time

[0325] Server: A high-performance computer for storing operation logs and training AI models

[0326] software:

[0327] Operation log collection software: A program for recording user operations

[0328] Database: A database management system for storing operation logs (e.g., MySQL (registered trademark), PostgreSQL)

[0329] Machine learning framework: Software for training AI models (e.g., TensorFlow, PyTorch)

[0330] User interface: GUI for presenting prediction results to users (e.g., HTML, CSS, JavaScript (registered trademark))

[0331] System Operation

[0332] 1. Collecting and saving operation logs:

[0333] The terminal monitors the series of operations performed by the user in real time and records them as an operation log, which is then sent to a server and stored in a database.

[0334] 2. Training the AI ​​model:

[0335] The server uses the saved operation logs to train an AI model, specifically learning operation patterns using machine learning frameworks such as TensorFlow and PyTorch.

[0336] 3. Real-time operation log collection and prediction:

[0337] When a user initiates a new operation, the device sends a log of that operation in real time to the server, which then inputs the received operation log into the AI ​​model to predict the most likely next operation.

[0338] 4. Prediction and execution:

[0339] The server returns the prediction results to the device, which then presents them to the user. When the user selects a prediction result, the device executes the corresponding operation.

[0340] Specific examples

[0341] For example, suppose a user begins to enter data into Excel. The device records this operation as an operation log and sends it to the server. The server saves the received operation log in a database and predicts the next operation using an AI model. Based on past operation patterns, the AI ​​model predicts that the user is likely to graph the data next. The server sends this prediction result back to the device, and the device displays a message to the user asking, "Do you want to graph the data next?" If the user accepts this suggestion, the device automatically starts creating a graph in Excel.

[0342] Prompt Sentence Examples

[0343] "When a user starts entering data into Excel, please design an AI model that predicts the next operation that the user is likely to perform and suggests it to the user." The flow of the specific processing in Example 3 will be described with reference to Figure 15.

[0344] Step 1:

[0345] The terminal collects user operation logs. Specifically, it monitors a series of user operations (e.g., data entry in Excel, web browser operations, etc.) in real time and records them as an operation log. The input is the user's operation, and the output is the operation log data.

[0346] Step 2:

[0347] The terminal sends the collected operation logs to the server. The server stores the received operation logs in a database. The input is the operation log data, and the output is the operation log stored in the database.

[0348] Step 3:

[0349] The server uses the stored operation logs to train an AI model. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to learn operation patterns. The input is the operation logs stored in the database, and the output is the trained AI model.

[0350] Step 4:

[0351] When a user starts a new operation, the terminal sends the operation log to the server in real time. The input is the new operation log, and the output is the operation log sent to the server.

[0352] Step 5:

[0353] The server inputs the received operation log into the AI ​​model and predicts the next operation that is likely to be performed. Specifically, the AI ​​model makes predictions based on operation patterns it has learned in the past. The input is the new operation log and the trained AI model, and the output is the predicted next operation.

[0354] Step 6:

[0355] The server returns the prediction results to the terminal. The terminal presents the prediction results to the user. Specifically, it displays a message such as "Would you like to graph the data next?" on the screen. The input is the predicted next operation, and the output is the prediction result presented to the user.

[0356] Step 7:

[0357] When the user selects a predicted result, the terminal executes the operation. Specifically, when the user selects "Graph," the terminal automatically starts creating a graph in Excel. The input is the user's selection, and the output is the executed operation.

[0358] (Application example 3)

[0359] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0360] Conventional factory robots could collect operation logs but were unable to effectively utilize that data, making it difficult to predict the next operation. This resulted in problems such as reduced robot operating efficiency and difficulty in optimizing work. Furthermore, there were insufficient means for users to navigate through operations and tasks, which often led to operational errors and delays.

[0361] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating AI based on the mined tasks, means for navigating user operations and tasks using the generated AI, means for collecting robot operation logs and learning operation patterns, means for predicting the next operation to be performed when a new operation log is input, and means for displaying the predicted operation on the robot's interface so that the robot can select and execute the operation. This improves the operating efficiency of the robot and enables work optimization.

[0362] A "user operation log" is a record of a series of operations performed by a user.

[0363] "Task mining" means extracting specific tasks and procedures from collected operation logs.

[0364] "Generating AI" means building an artificial intelligence model based on collected data.

[0365] "Navigating user operations and tasks" means presenting the next operation or task predicted by the AI ​​to the user and assisting them with their work.

[0366] A "robot operation log" is a record of a series of operations performed by a robot.

[0367] "Learning operation patterns" means that the AI ​​learns common operating procedures and trends from collected operation logs.

[0368] "Predicting the next operation" means that when a new operation log is input, the AI ​​guesses what operation should be performed next.

[0369] A "robot interface" is a point of contact through which a robot exchanges information with users and other systems.

[0370] "Selecting and executing an operation" means that the robot selects a predicted operation and actually performs that operation.

[0371] A system for carrying out this invention includes means for collecting user operation logs, mining tasks from the collected operation logs, and generating AI based on the mined tasks. It also includes means for navigating user operations and tasks using the generated AI. It also includes means for collecting robot operation logs and learning operation patterns, means for predicting the next operation to be performed when a new operation log is input, and means for displaying the predicted operation on the robot's interface so that the robot can select and execute the operation.

[0372] Hardware and software used

[0373] Hardware: Factory robots (e.g., general-purpose robotic arms), high-performance GPUs (e.g., NVIDIA Tesla)

[0374] Software: TensorFlow (used to build and train AI models), Python (programming language)

[0375] Data processing and calculation

[0376] Data collection

[0377] The server collects user operation logs and robot operation logs, which include a series of operations performed by the user and a series of operations performed by the robot.

[0378] Task Mining

[0379] The server extracts specific tasks and procedures from the collected operation logs, which makes task patterns clear.

[0380] Generating AI models

[0381] The server generates an AI model based on the mined tasks, which is built using TensorFlow and learns patterns from the operation logs.

[0382] Operation Prediction

[0383] When a new operation log is input, the server uses the AI ​​model to predict the next operation to be performed, and the prediction results are displayed on the robot's interface.

[0384] Executing an operation

[0385] The robot selects the predicted operation from the interface and executes it, improving the robot's operational efficiency and optimizing the work.

[0386] Specific examples

[0387] For example, when a factory robot assembles parts, it can predict the next part or tool to be installed based on past operation logs and provide this information to the robot, improving work efficiency and reducing the occurrence of errors.

[0388] Prompt Sentence Examples

[0389] "Design an AI model that predicts the next operation that a factory robot should perform based on past operation logs when assembling parts, and presents this to the robot."

[0390] In this way, a smart navigation system can be built to optimize the operation of factory robots.

[0391] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0392] Step 1:

[0393] The server collects user operation logs and robot operation logs. The input includes a series of operations performed by the user and a series of operations performed by the robot. These operation logs are stored in a database. The output is the collected operation logs.

[0394] Step 2:

[0395] The server extracts specific tasks and procedures from the collected operation logs. The operation logs collected in step 1 are used as input. Data mining techniques are used to extract task patterns from the operation logs. The output is the mined tasks.

[0396] Step 3:

[0397] The server generates an AI model based on the mined tasks. The task patterns obtained in step 2 are used as input. TensorFlow is used to build an AI model that learns patterns in the operation logs. The generated AI model is obtained as output.

[0398] Step 4:

[0399] When a new operation log is input, the server uses the AI ​​model to predict the next operation to be performed. The new operation log and the AI ​​model generated in step 3 are used as input. The AI ​​model predicts the next operation based on the pattern in the operation log. The predicted operation is obtained as output.

[0400] Step 5:

[0401] The server displays the predicted operation on the robot's interface. As input, the predicted operation obtained in step 4 is used. The predicted operation is displayed on the robot's interface, allowing the robot to select and execute the operation. As output, the operation displayed on the robot's interface is obtained.

[0402] Step 6:

[0403] The robot selects a predicted operation from the interface and executes it. As input, it uses the operation displayed in step 5. The robot executes the selected operation and proceeds with its work. As output, it obtains the executed operation.

[0404] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0405] "Example 1"

[0406] In one embodiment of the present invention, an emotion engine is incorporated into a system that includes means for collecting a user's PC operation log, means for mining tasks from the collected PC operation log, means for generating an AI based on the mined tasks, and means for navigating the user's PC operations and tasks using the generated AI. This emotion engine analyzes the user's emotional state while using the PC in real time and feeds the results back to the AI.

[0407] "Example 2"

[0408] Specifically, the emotion engine estimates a user's emotional state, such as stress level and concentration level, based on their keyboard typing speed, mouse movements, and PC usage time. For example, if the keyboard typing speed is fast and the mouse movements are large, it determines that the user may be feeling stressed. Also, if the PC usage time is long, it determines that the user may be concentrating on their work.

[0409] "Example 3"

[0410] Based on feedback from the emotion engine, the AI ​​provides task navigation that adapts to the user's emotional state. For example, if the AI ​​determines that the user is feeling stressed, it will prioritize tasks to reduce stress (such as displaying a message encouraging them to take a break). If the AI ​​determines that the user is concentrating on a task, it will prioritize tasks related to that task.

[0411] The processing flow of each embodiment will be described below.

[0412] "Example 1"

[0413] Step 1: Collect user PC operation logs.

[0414] Step 2: Mine tasks from the collected PC operation logs.

[0415] Step 3: Generate AI based on the mined tasks.

[0416] Step 4: The generated AI guides the user through PC operations and tasks.

[0417] Step 5: Analyze the user's emotional state in real time while using the PC and provide feedback to the AI.

[0418] "Example 2"

[0419] Step 1: Estimate the user's emotional state, such as stress level and concentration level, based on the user's keyboard typing speed, mouse movements, and PC usage time.

[0420] Step 2: Feed back the estimated emotional state to the AI.

[0421] "Example 3"

[0422] Step 1: The AI ​​provides task navigation adapted to the user's emotional state based on feedback from the emotion engine.

[0423] Step 2: If the AI ​​determines that the user is stressed, it prioritizes tasks to reduce stress (for example, displaying a message encouraging the user to take a break).

[0424] Step 3: If the AI ​​determines that the user is concentrating on a task, it will prioritize navigating tasks related to that task.

[0425] Example 1

[0426] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0427] Conventional information processing device operation support systems were able to collect user operation logs, mine tasks, and generate AI, but they did not provide support that took into account the user's emotional state. As a result, there were operations that users found stressful or that they could not perform efficiently, and the user experience was not sufficiently improved. In addition, the system was limited in its ability to learn and predict user operation patterns and determine task priorities. This led to problems such as reduced user operation efficiency and satisfaction.

[0428] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0429] In this invention, the server includes means for collecting a user's information processing device operation log, means for mining tasks from the collected information processing device operation log, means for generating an artificial intelligence based on the mined tasks, means for navigating the user's information processing device operations and tasks using the generated artificial intelligence, means for analyzing the user's emotional state during operation in real time, and means for feeding back the analysis results to the artificial intelligence. This makes it possible to learn and predict the user's operation patterns and further enable support that takes the user's emotional state into consideration. This can improve the user's operation efficiency and satisfaction.

[0430] A "user" is an individual or group that operates an information processing device.

[0431] An "information processing device" is an electronic device such as a computer or tablet that is operated by a user.

[0432] An "operation log" is a record of operations performed by a user on an information processing device.

[0433] A "task" is a series of operations or tasks that a user performs on an information processing device.

[0434] "Mining" is the process of extracting useful information and patterns from collected operation logs.

[0435] "Artificial intelligence" refers to algorithms and models that learn user operation patterns and make predictions and provide assistance.

[0436] "Navigation" means providing instructions and suggestions to assist users in their operations and tasks, and to make them more efficient.

[0437] "Emotional state" refers to the psychological state, such as stress or joy, that a user feels while operating the device.

[0438] "Real-time analysis" is the process of instantly assessing a user's emotional state while they are interacting.

[0439] "Feedback" is the process of reflecting the results of analysis into artificial intelligence.

[0440] MODE FOR CARRYING OUT THE INVENTION

[0441] This invention is a system that collects user operation logs of information processing devices, mines tasks from the collected operation logs, generates artificial intelligence based on the mined tasks, and uses the generated artificial intelligence to navigate the user's operations and tasks. It also aims to improve user operation efficiency and satisfaction by analyzing the user's emotional state during operation in real time and feeding the results back to the artificial intelligence.

[0442] 1. Collection of user operation logs for information processing devices

[0443] Users install dedicated software on their information processing devices. This software runs in the background and records user operations in real time. Specifically, it collects information such as the type of operation (e.g., clicks, keyboard input, window switching, etc.), the order of operations, and the time taken for the operation. For example, if a user opens a browser, visits a specific website, and then sends an email, the entire series of operations is recorded in detail.

[0444] 2. Task Mining

[0445] The device periodically sends the recorded operation log to the server. The server receives it and stores it in a database. The server analyzes the received operation log and extracts tasks from the series of operations performed by the user. For example, the user's daily email check and reply operation is recognized as one task.

[0446] 3. Generating an AI model

[0447] The server uses the mined work data to generate an AI model that learns the user's operation patterns. This AI model predicts the user's operations and makes suggestions for improving efficiency. The server then distributes the generated AI model to the device. The device then uses this AI model to navigate the user's operations.

[0448] 4. Incorporating an Emotional Engine

[0449] An emotion engine built into the device analyzes the user's emotional state in real time while they are operating the device. For example, it can detect emotions such as stress or joy from the user's facial expressions and voice. The device then sends the analyzed emotional data to a server, which receives it and stores it in a database. The server then feeds the received emotional data back to the AI ​​model, allowing the AI ​​model to navigate operations while taking the user's emotional state into account. For example, if the user is feeling stressed, the AI ​​will suggest ways to simplify operations or provide assistance.

[0450] Examples of specific examples and prompts

[0451] Specific examples

[0452] Let's say a user checks their email at 9 a.m. every morning and replies to important emails. This operation log is collected and mined as work. Next, an AI model is generated that automatically opens the email client at the time the user checks email and navigates to prioritize important emails. If the user feels stressed while checking email, the emotion engine feeds that information back to the AI, which then suggests ways to simplify the operation.

[0453] Prompt Sentence Examples

[0454] "Please record the email check and reply operations that a user performs every morning at 9am, and generate an AI model to automate this operation. Also, please use an emotion engine to analyze the stress the user feels while performing this operation, and feed the results back to the AI ​​to suggest ways to simplify the operation."

[0455] In this way, by collecting the user's information processing device operation logs, mining the work, generating an artificial intelligence model, and incorporating an emotion engine, a system is realized that navigates the user's operations and improves efficiency.

[0456] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0457] Step 1:

[0458] The user installs the dedicated software into the information processing device.

[0459] Input: Dedicated software installer

[0460] Output: Installed software

[0461] How it works: The user downloads the dedicated software installer and installs it on their information processing device by following the installation wizard. Once the installation is complete, the software starts running in the background.

[0462] Step 2:

[0463] The device records user actions in real time.

[0464] Input: User actions (clicks, keyboard input, window switching, etc.)

[0465] Output: Operation log

[0466] Specific actions: Software installed on the device monitors user actions in real time and records information such as the type, sequence, and time of actions. For example, a user opening a browser, visiting a specific website, and then sending an email is recorded.

[0467] Step 3:

[0468] The operation log recorded by the terminal is sent to the server.

[0469] Input: Operation log

[0470] Output: Operation log sent to the server

[0471] Specific operation: The terminal sends operation logs to the server at regular intervals, and the server stores the received operation logs in a database.

[0472] Step 4:

[0473] The server analyzes the operation log and mines the work.

[0474] Input: Operation log

[0475] Output: Mined work

[0476] Specific operation: The server analyzes the received operation log and extracts tasks from the series of operations performed by the user. For example, the user's daily email check and reply operation is recognized as one task.

[0477] Step 5:

[0478] The server generates an artificial intelligence model based on the mined work.

[0479] Input: Mined work

[0480] Output: Artificial Intelligence Model

[0481] How it works: The server uses the mined task data to generate an AI model that learns the user's operation patterns. This AI model predicts the user's operations and makes suggestions to improve efficiency.

[0482] Step 6:

[0483] The server distributes the generated artificial intelligence model to the terminal.

[0484] Input: Artificial Intelligence Model

[0485] Output: AI model delivered to the device

[0486] Specific operation: The server delivers the generated AI model to the terminal, which then uses the AI ​​model to navigate the user's operations.

[0487] Step 7:

[0488] The device analyzes the user's emotional state in real time while operating the device.

[0489] Input: Data during user operation (facial expressions, voice, etc.)

[0490] Output: Emotional state data

[0491] Specific operation: The emotion engine built into the device analyzes the user's facial expressions and voice in real time while operating the device, detecting emotional states such as stress or joy.

[0492] Step 8:

[0493] The terminal transmits the emotional state data to the server.

[0494] Input: Emotional state data

[0495] Output: Emotional state data sent to the server

[0496] Specific operation: The device sends the analyzed emotional state data to the server, which receives it and stores it in a database.

[0497] Step 9:

[0498] The server feeds emotional state data back to the artificial intelligence model.

[0499] Input: Emotional state data

[0500] Output: fed back artificial intelligence model

[0501] Specific operation: The server feeds the received emotional state data back to the AI ​​model, allowing the AI ​​model to perform operation navigation that takes the user's emotional state into account. For example, if the user is feeling stressed, the AI ​​will suggest ways to simplify operations or provide assistance.

[0502] (Application example 1)

[0503] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0504] Conventional factory robots lack navigation systems to improve work efficiency and accuracy, and in particular, they are not optimized to take into account the emotional state of the robot during operation. This results in issues such as reduced robot operation efficiency and insufficient work accuracy. Furthermore, the lack of a system for mining tasks based on operation logs or providing feedback on emotional states makes it difficult to optimize robot operations in real time.

[0505] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0506] In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating artificial intelligence based on the mined tasks, means for navigating user operations and tasks using the generated artificial intelligence, means for analyzing emotional states during operations in real time using an emotion engine and feeding the results back to the artificial intelligence, and means for optimizing work efficiency based on the operation logs and emotional states, thereby optimizing the operating efficiency of factory robots and improving work accuracy.

[0507] An "operation log" is a record of the type, order, and time of operations performed by a user or robot.

[0508] "Task mining" is a technique for extracting and analyzing specific tasks and processes from collected operation logs.

[0509] "Artificial intelligence" is a computer program that learns from collected data and makes predictions and navigates.

[0510] "Means of navigating" refers to technology that instructs users and robots on optimal operations and task progression.

[0511] The "emotion engine" is a system that analyzes the emotional state during operation in real time and provides feedback based on the results.

[0512] "Means for optimizing work efficiency" is a technology for maximizing work efficiency based on operation logs and emotional states.

[0513] A system for implementing the present invention includes an application called "Smart Factory Navigator" that is installed on a factory robot. A specific embodiment of this system will be described below.

[0514] System configuration

[0515] The system consists of the following main components:

[0516] 1. Operation log collection method: Collect operation logs of factory robots in real time.

[0517] 2. Task mining method: Tasks are mined from the collected operation logs.

[0518] 3. AI generation means: Generate AI based on the mined tasks.

[0519] 4. Navigation: The generated artificial intelligence navigates the robot through operations and tasks.

[0520] 5. Emotion engine: Analyzes the emotional state during operation in real time and feeds the results back to the artificial intelligence.

[0521] 6. Work efficiency optimization method: Optimize work efficiency based on operation logs and emotional state.

[0522] Hardware and software used

[0523] Hardware: Factory robots (e.g., general-purpose robotic arms), emotion sensors (e.g., biosignal sensors)

[0524] Software: Python, EmotionEngine library, TaskMining library, AI Navigator library

[0525] Data processing and calculation

[0526] 1. Operation log collection: Collect factory robot operation logs (type of operation, sequence, and time) in real time. This includes data from the robot's motion sensors and control system.

[0527] 2. Emotional state analysis: The emotion engine analyzes the emotional state of the robot in real time while it is operating. It analyzes bio-signal data obtained from emotion sensors and evaluates emotional states such as stress and concentration.

[0528] 3. Task mining: Extract and analyze specific tasks and processes from collected operation logs, thereby identifying the priority and efficient procedures for tasks that robots should perform.

[0529] 4. AI generation and navigation: AI is generated based on the mined tasks to navigate the robot's operations and tasks. The AI ​​proposes optimal work procedures based on the operation log and emotional state.

[0530] Specific examples

[0531] For example, if a robot performing parts assembly work in a factory collects operation logs and detects stress using its emotion engine, the AI ​​will adjust the pace of the work and navigate to reduce stress, optimizing the robot's operational efficiency and improving its work accuracy.

[0532] Prompt Sentence Examples

[0533] "How can we collect the operation logs of factory robots, mine their tasks, generate artificial intelligence, and adjust the pace of work if the emotion engine detects a state of stress?"

[0534] In this way, the smart factory navigator can optimize the operating efficiency of factory robots and improve work accuracy.

[0535] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0536] Step 1:

[0537] The server collects the operation logs of factory robots in real time. The operation logs include the type, sequence, and time of operations. This accumulates the robot's operation data. The input is data from the robot's operation sensors and control system, and the output is a database of operation logs.

[0538] Step 2:

[0539] The server collects biosignal data obtained from the emotion sensor in real time. Using the emotion engine, the server analyzes this data and evaluates the emotional state, such as stress and concentration. The input is the biosignal data from the emotion sensor, and the output is the evaluation result of the emotional state.

[0540] Step 3:

[0541] The server mines tasks from the collected operation logs. Specifically, it analyzes the operation logs and extracts specific tasks and processes. The input is a database of operation logs, and the output is a list of mined tasks.

[0542] Step 4:

[0543] The server generates an AI based on the mined tasks. The generated AI proposes optimal work procedures based on the operation log and emotional state. The input is the list of mined tasks and the evaluation result of the emotional state, and the output is the generated AI model.

[0544] Step 5:

[0545] The server uses the generated AI to navigate the factory robot's operations and tasks. Specifically, the AI ​​instructs the robot on optimal work procedures and adjusts the pace of work. The input is the generated AI model, and the output is navigation instructions for the robot.

[0546] Step 6:

[0547] The server optimizes work efficiency based on the operation log and emotional state. Specifically, AI provides real-time feedback on the emotional state and adjusts the pace and procedures of work. The input is the operation log and the evaluation result of the emotional state, and the output is an optimized work procedure.

[0548] In this way, the server can optimize the operating efficiency of the factory robots and improve the accuracy of their work.

[0549] Example 2

[0550] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0551] Conventional systems for collecting and analyzing information processing device operation logs have had difficulty identifying user operation patterns and mining tasks. It has also been difficult to estimate a user's emotional state and provide appropriate navigation based on that. This has led to issues such as insufficient work efficiency and stress management for users.

[0552] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0553] In this invention, the server includes means for collecting a user's information processing device operation log, means for mining tasks from the collected information processing device operation log, means for generating artificial intelligence based on the mined tasks, means for navigating the user's information processing device operations and tasks using the generated artificial intelligence, means for analyzing the user's operation log data and identifying operation patterns, means for estimating the user's emotional state from the operation log data, and means for providing the analysis results to the user. This makes it possible to identify the user's operation patterns and efficiently mine tasks. Furthermore, by estimating the user's emotional state and providing appropriate navigation based on that, it is possible to improve the user's work efficiency and stress management.

[0554] "User" refers to an individual or group that operates an information processing device.

[0555] "Information processing device" refers to electronic devices such as computers, tablets, and smartphones.

[0556] The term "operation log" refers to data that records input data and operation history when a user operates an information processing device.

[0557] A "task" refers to a series of operations or tasks that a user performs on an information processing device.

[0558] "Mining" refers to the process of extracting useful information and patterns from collected data.

[0559] "Artificial intelligence" refers to a system that uses machine learning and data analysis to predict and navigate user operations and tasks.

[0560] "Navigating" refers to assisting and guiding users in their operations and tasks.

[0561] "Emotional state" refers to a user's psychological state, such as stress level or concentration.

[0562] "Analysis" refers to the process of processing collected data and extracting meaningful information.

[0563] A "clustering algorithm" refers to a method for grouping data and grouping together data that has the same patterns.

[0564] "Report" refers to documents or data summarizing the results of analysis.

[0565] MODE FOR CARRYING OUT THE INVENTION

[0566] The present invention provides a system that collects and analyzes a user's operation log of an information processing device to identify the user's operation pattern and emotional state and provide appropriate navigation. Specific embodiments of this system are described below.

[0567] 1. Program Generation

[0568] The program in this system is designed to collect and analyze user operation logs of information processing devices. Specifically, it includes an algorithm for mining tasks from the operation logs and an emotion engine for estimating the user's emotional state.

[0569] 2. Program Processing

[0570] The server collects the user's operation log of the information processing device. The collected data includes keyboard typing speed, mouse movement, and the amount of time the information processing device is used. This data is analyzed on the server.

[0571] For the analysis, we use Python's Pandas library to preprocess the data and then use Scikit-learn to build a machine learning model. Specifically, we use a clustering algorithm (e.g., K-means) to find the same operation patterns from the operation logs.

[0572] For example, if a specific operation is performed repeatedly in a certain order, the combination of operations is recognized as a single task, which allows us to identify which tasks the user frequently performs.

[0573] The emotion engine estimates the user's emotional state, such as stress level and concentration level, based on the user's keyboard typing speed, mouse movements, and the amount of time the information processing device is used. For example, if the keyboard typing speed is fast and the mouse movements are large, it determines that the user may be feeling stressed. Also, if the information processing device is used for a long time, it determines that the user may be concentrating on their work.

[0574] 3. Examples and prompts

[0575] As a concrete example, let's say the following operation log is collected:

[0576] Keyboard typing speed: 120 keystrokes / min

[0577] Mouse movement: 500 pixels / minute

[0578] Time spent using information processing equipment: 4 hours

[0579] The server analyzes this data and estimates that the user is under high stress. If a specific operation pattern is repeated, the server recognizes the operation as a single task.

[0580] Example prompts to input to the generative AI model:

[0581] Analyze the user activity log data and provide the following information:

[0582] Keyboard typing speed

[0583] Mouse movement

[0584] Information processing device usage time

[0585] Use this data to estimate the user's stress level and concentration, and identify recurring patterns of operation and recognize them as a single task.

[0586] Using this prompt, the generative AI model can analyze the user's operation log data and provide the necessary information.

[0587] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0588] Step 1:

[0589] Users install and launch dedicated client software on their information processing devices. This software runs in the background and collects real-time operation logs of users' keyboard typing speed, mouse movements, and the amount of time they use the information processing device. The collected data is then sent to a server at regular intervals.

[0590] Input: User operation log data (keyboard typing speed, mouse movement, time spent using information processing devices)

[0591] Output: Operation log data sent to the server

[0592] Step 2:

[0593] The server receives the received operation log data and performs preprocessing on the data. Specifically, it cleans the data using the Python Pandas library, fills in missing values, removes outliers, and converts the data into a format that is easy to analyze.

[0594] Input: Received operation log data

[0595] Output: Preprocessed audit log data

[0596] Step 3:

[0597] The server uses the preprocessed data to analyze operation patterns. Specifically, it uses Scikit-learn's clustering algorithm (e.g., K-means) to group together operations with the same pattern. For example, if a specific operation is repeated in a certain order, the combination of operations is recognized as a single task.

[0598] Input: Preprocessed operation log data

[0599] Output: Identified operation patterns and tasks

[0600] Step 4:

[0601] The server estimates the user's emotional state from the operation log data. Specifically, it estimates the user's stress level and concentration level based on data such as keyboard typing speed, mouse movements, and the amount of time the information processing device is used. For example, if the keyboard typing speed is fast and the mouse movements are large, it is determined that the user may be feeling stressed. Also, if the information processing device is used for a long time, it is determined that the user may be concentrating on their work.

[0602] Input: Operation log data (keyboard typing speed, mouse movement, time spent using information processing devices)

[0603] Output: Estimated user's emotional state (stress level, concentration)

[0604] Step 5:

[0605] The server provides the analysis results to the user. Specifically, it generates a report on the user's operation patterns and emotional state and sends it to the user's device. The user can use this report to improve their work efficiency and stress management.

[0606] Input: Identified operation patterns and tasks, estimated user emotional state

[0607] Output: Report provided to the user (operation patterns, emotional state)

[0608] (Application example 2)

[0609] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0610] While conventional systems were able to collect user operation logs and mine tasks, they were unable to analyze robot operation logs to propose efficient work procedures or estimate the need for maintenance based on the robot's operating speed and operating time. This made it difficult to efficiently operate and optimize maintenance of robots in factories and other workplaces.

[0611] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0612] In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating artificial intelligence based on the mined tasks, means for navigating user operations and tasks using the generated artificial intelligence, means for collecting and analyzing robot operation logs, means for finding similar operation patterns and proposing efficient work procedures, and means for estimating the need for maintenance from the robot's operating speed and operating time. This makes it possible to analyze both the user operation logs and the robot operation logs, propose efficient work procedures, and optimize maintenance.

[0613] A "user operation log" is a series of records generated when a user operates a computer or device.

[0614] "Task mining" is a method of analyzing collected operation logs, finding the same operation patterns, and recognizing them as a single task.

[0615] "Artificial intelligence" is a computer program that learns from collected data and makes predictions and judgments.

[0616] "Means of navigation" refers to a method of instructing users or robots on efficient operations and work procedures.

[0617] A "robot operation log" is a series of records generated when a robot performs a task.

[0618] "Means of analysis" are ways of finding patterns and trends based on collected data.

[0619] "Means for proposing efficient work procedures" refers to a method for finding and proposing optimal work procedures based on the analysis results.

[0620] "Operation speed" refers to the speed at which the robot performs a task.

[0621] "Operating time" is the total amount of time that the robot is actually performing work.

[0622] "Means for estimating the need for maintenance" refers to a method for determining whether maintenance is necessary based on data such as the robot's operating speed and operating time.

[0623] A system for carrying out the present invention includes a program for collecting and analyzing user operation logs and robot operation logs. Specific embodiments of the system will be described below.

[0624] Hardware and software used

[0625] Hardware: computers, robots, sensors for collecting log data, servers for data analysis

[0626] Software: Python, Pandas (library for data analysis), Scikit-learn (library for machine learning)

[0627] Data processing and calculation

[0628] 1. Collecting user operation logs

[0629] The server collects a series of records generated when users interact with their computers or devices, including keyboard typing speed, mouse movements, and usage time.

[0630] 2. Collecting robot operation logs

[0631] The server collects a series of records generated by the robot as it performs its tasks, including its speed, duration, and type of operation.

[0632] 3. Analysis of operation logs

[0633] The server analyzes the collected operation logs and finds the same operation patterns, which makes it possible to propose efficient work procedures. Specifically, it clusters the operation logs using KMeans clustering to find patterns.

[0634] 4. Estimating maintenance needs

[0635] The server estimates the need for maintenance based on the robot's movement speed and operating time. For example, if the robot's movement speed drops or its operating time exceeds a certain threshold, the server determines that maintenance is necessary.

[0636] Specific examples

[0637] For example, if a robot used in a factory repeats a specific operation, the server will detect that operation pattern and suggest an efficient work procedure. If the robot's operating speed slows down, the server will notify the robot that maintenance is required.

[0638] Prompt Sentence Examples

[0639] "Please develop an application that analyzes the operation logs of factory robots and proposes efficient work procedures. Also, please add a function that estimates the need for maintenance based on the robot's operating speed and operating time."

[0640] In this way, both the user's operation log and the robot's operation log can be analyzed, making it possible to propose efficient work procedures and optimize maintenance.

[0641] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0642] Step 1:

[0643] The server collects user operation logs. Specifically, it collects data such as keyboard typing speed, mouse movement, and usage time generated when a user operates a computer or device. The input is the user operation data, and the output is the collected operation log.

[0644] Step 2:

[0645] The server collects the robot's operation logs. Specifically, it collects data such as the robot's operating speed, operating time, and type of operation generated when the robot performs a task. The input is the robot's operation data, and the output is the collected operation log.

[0646] Step 3:

[0647] The server analyzes the collected user operation logs. Specifically, it clusters the data using KMeans clustering to find the same operation patterns from the operation logs. The input is the collected operation logs, and the output is the clustered operation patterns.

[0648] Step 4:

[0649] The server analyzes the collected robot operation logs. Specifically, it clusters the data using KMeans clustering to find the same operation patterns from the operation logs. The input is the collected operation logs, and the output is the clustered operation patterns.

[0650] Step 5:

[0651] The server proposes efficient work procedures based on the analysis results. Specifically, it finds the optimal work procedure based on the clustered operation patterns and proposes it to the user or robot. The input is the clustered operation patterns, and the output is the proposed work procedure.

[0652] Step 6:

[0653] The server estimates the need for maintenance based on the robot's operating speed and operating time. Specifically, it determines that maintenance is necessary if the operating speed drops or if the operating time exceeds a certain threshold. The input is the robot's operating speed and operating time, and the output is a notification regarding the need for maintenance.

[0654] Step 7:

[0655] The server notifies users and robots of efficient work procedures and the need for maintenance. Specifically, it provides information on proposed work procedures and maintenance needs to users and robots. The input is the notification of the proposed work procedures and the need for maintenance, and the output is the notification to the user or robot.

[0656] Example 3

[0657] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0658] Conventional systems only collected user operation logs, but lacked the means to effectively utilize that data. Furthermore, task navigation based on the user's emotional state was not provided, resulting in insufficient work efficiency and stress management for users. Furthermore, the system lacked the functionality to predict and present the next operation to the user, which often made user operations cumbersome.

[0659] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0660] In this invention, the server includes a means for collecting user operation logs, a means for storing the collected operation logs in a database, a means for training an AI model using the stored operation logs, a means for predicting the next operation when a new operation log is input, a means for presenting the prediction result to the user, and a means for adjusting task navigation based on feedback from the emotion engine. This makes it possible to effectively utilize the user's operation logs to predict and present the next operation to be performed. Furthermore, by providing task navigation according to the user's emotional state, it is possible to improve the user's work efficiency and manage stress.

[0661] A "user operation log" is data that records a series of operations performed by a user in chronological order.

[0662] A "database" is a system for storing and managing collected operation logs.

[0663] An "AI model" is a model trained using a machine learning algorithm that learns a user's operation patterns and predicts the next operation.

[0664] "Training" is the process of teaching an AI model using collected operation logs.

[0665] "Prediction" means that the AI ​​model estimates the most likely next operation based on new operation logs.

[0666] The "emotion engine" is a system that analyzes the user's emotional state from their facial expressions and voice and provides feedback based on the results.

[0667] "Task navigation" refers to guiding users to the next action or task they should perform.

[0668] "Feedback" refers to providing information about the user's emotional state obtained from the emotion engine to the AI ​​model.

[0669] This invention is a system that collects user operation logs, uses the data to train an AI model, and predicts the next action to be taken. It also has the ability to adjust task navigation based on the user's emotional state.

[0670] The server uses dedicated software to collect user operation logs. This software monitors the series of operations performed by the user in real time and records them as an operation log. The operation log includes information such as the buttons the user clicked, the text they entered, and the applications they opened.

[0671] The collected operation logs are stored in a database, using a relational database such as MySQL or PostgreSQL. The operation logs are stored in a format that includes information such as the user ID, operation details, and timestamp.

[0672] The server uses the saved operation logs to train the AI ​​model. For training, it uses machine learning frameworks such as TensorFlow and PyTorch. Specifically, the operation logs are used as input data and the next operation is used as output data to train the model.

[0673] When a user performs a new operation, the operation log is sent to the server in real time. The server inputs this new operation log into the AI ​​model and predicts the next operation that is likely to be performed. The prediction result is output as the type of operation and its probability.

[0674] The device presents the prediction results received from the server to the user. Specifically, it displays the predicted operation on the screen and allows the user to select and execute the operation. For example, a message such as "Do you want to save the data next?" is displayed.

[0675] Furthermore, the server receives feedback from the emotion engine and adjusts task navigation based on the user's emotional state. The emotion engine analyzes the user's emotional state from their facial expressions and voice and sends the results to the server. For example, if the server determines that the user is feeling stressed, it will send a message to the device encouraging them to take a break.

[0676] As a concrete example, consider the case where a user is entering data into Excel. Based on past operation logs, the AI ​​model predicts that the next most likely operation will be "saving data" or "creating a graph." These operations are displayed on the user's screen, and the user can select and execute them.

[0677] An example of a prompt sentence might be:

[0678] "When a user is entering data in Excel, predict what they're likely to do next."

[0679] Using this prompt sentence, the AI ​​model predicts the next operation to be performed and presents it to the user. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0680] Step 1:

[0681] The server collects user operation logs. Specifically, it uses dedicated software to monitor the series of operations performed by the user in real time and records them as an operation log. Input includes information such as the buttons the user clicked, the text they entered, and the applications they opened. The output is a log of these operations.

[0682] Step 2:

[0683] The server saves the collected operation logs in a database. A relational database such as MySQL or PostgreSQL is used as the database. The input includes the operation logs collected in step 1. The output is the operation logs saved in the database. Specifically, they are saved in a format that includes information such as the user ID, operation details, and timestamp.

[0684] Step 3:

[0685] The server uses the stored operation logs to train the AI ​​model. Machine learning frameworks such as TensorFlow and PyTorch are used for training. The input includes the operation logs stored in the database. The output is a trained AI model. Specifically, the model learns using the operation logs as input data and the next operation as output data.

[0686] Step 4:

[0687] When a user performs a new operation, the operation log is sent to the server in real time. The input includes the new operation log performed by the user. The output is sent to the server. Specifically, it includes information such as the button the user clicked and the text they entered.

[0688] Step 5:

[0689] The server uses the AI ​​model to predict the next operation based on the new operation log. The input includes the new operation log sent in step 4. The output is a prediction of the most likely next operation. Specifically, the type of operation and its probability are output.

[0690] Step 6:

[0691] The device presents the prediction results received from the server to the user. The input includes the prediction results generated in step 5. The output displays the predicted operation on the screen. Specifically, a message such as "Do you want to save the data next?" is displayed, allowing the user to select and execute the operation.

[0692] Step 7:

[0693] The server receives feedback from the emotion engine and adjusts the task navigation based on the user's emotional state. The input includes feedback information from the emotion engine. The output generates an adjusted task navigation. Specifically, if the server determines that the user is feeling stressed, it sends a message to the device encouraging the user to take a break.

[0694] (Application example 3)

[0695] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0696] Conventional factory robots were unable to predict the next task based on operation logs, making efficient work navigation difficult. Furthermore, they lacked the ability to determine the user's emotional state based on data obtained from the robot's sensors and prioritize appropriate tasks, potentially reducing work efficiency and safety.

[0697] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating AI based on the mined tasks, means for navigating user operations and tasks using the generated AI, means for determining the user's emotional state based on data obtained from the robot's sensors, and means for preferentially navigating tasks according to the emotional state. This makes it possible to predict the next task to be performed based on the operation logs, enabling efficient and safe task navigation.

[0698] An "operation log" is data that records the history of operations performed by users and robots.

[0699] A "task" is a series of operations or tasks performed to achieve a specific purpose.

[0700] "Mining" is the process of extracting useful information and patterns from large amounts of data.

[0701] "AI" is an abbreviation for artificial intelligence, a technology that allows computers to imitate human intelligence to learn and reason.

[0702] "Navigating" means instructing a user or robot on the next operation or task to perform.

[0703] A "sensor" is a device that detects physical environmental information and outputs it as data.

[0704] "Emotional state" refers to the current psychological and physiological state of the user or robot.

[0705] "Priority" is a criterion for determining which of multiple tasks or operations should be performed first.

[0706] A system for implementing this invention collects user operation logs, mines tasks from the collected operation logs, and generates AI based on the mined tasks. The generated AI navigates the user's operations and tasks, determines the user's emotional state based on data obtained from the robot's sensors, and prioritizes navigating tasks according to the user's emotional state.

[0707] Hardware and software used

[0708] Hardware: Factory robots, sensors (temperature, humidity, vibration, etc.)

[0709] Software: TensorFlow (AI model design and training), Python (data preprocessing and program execution)

[0710] Data processing and calculation

[0711] The server first collects user operation logs. These logs are data that record the history of operations performed by users and robots, and tasks are mined from them. AI is then generated based on the mined tasks, and the generated AI navigates the user's operations and tasks.

[0712] Furthermore, the robot determines the user's emotional state based on data obtained from its sensors. Emotional state refers to the current psychological and physiological state of the user or robot, and prioritizes task navigation accordingly. For example, if it determines that the user is feeling stressed, it will display a message encouraging them to take a break. Similarly, if it determines that the user is concentrating on a task, it will prioritize tasks related to that task.

[0713] Specific examples

[0714] For example, imagine a robot assembling parts in a factory. In this case, the system uses operation log data to predict the next task to be performed and issues instructions to the robot. If the robot's sensor detects high temperatures, the emotion engine determines that the robot is feeling "stressed" and displays a message encouraging the robot to take a break.

[0715] Prompt Sentence Examples

[0716] When a factory robot is assembling parts, use the operation log data to predict the next task to be performed and issue instructions to the robot. Also, if the robot's sensor detects a high temperature, the emotion engine will determine that the robot is feeling "stressed" and display a message encouraging it to take a break.

[0717] In this way, the next task to be performed can be predicted based on the operation log, enabling efficient and safe task navigation.

[0718] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0719] Step 1:

[0720] The server collects user operation logs. Operation logs are data that records the history of operations performed by users and robots, and subsequent processing is based on this data. The input is the operation data of users and robots, and the output is the collected operation logs.

[0721] Step 2:

[0722] The server mines tasks from the collected operation logs. A task is a series of operations or tasks performed to achieve a specific purpose. The input is the collected operation logs, and the output is the mined tasks. Specifically, the server analyzes the operation log data and extracts frequently occurring operation patterns.

[0723] Step 3:

[0724] The server generates AI based on the mined tasks. AI stands for artificial intelligence, a technology in which computers mimic human intelligence to learn and infer. The input is the mined tasks, and the output is the generated AI model. Specifically, the server designs an AI model to learn task patterns and trains the model using learning data.

[0725] Step 4:

[0726] The server uses the generated AI to navigate the user's operations and tasks. Navigation means instructing the user or robot on the next operation or task to be performed. The input is the current operation log and the generated AI model, and the output is instructions on the next operation or task to be performed. Specifically, the current operation log is input into the AI ​​model, and the next operation to be performed is predicted.

[0727] Step 5:

[0728] The server determines the user's emotional state based on data obtained from the robot's sensors. The emotional state refers to the current psychological and physiological state of the user or robot. The input is data from the sensors, and the output is the determined emotional state. Specifically, the server analyzes the sensor data and estimates the emotional state, such as stress and concentration.

[0729] Step 6:

[0730] The server prioritizes navigation to tasks according to the user's emotional state. The input is the determined emotional state and the current task, and the output is the task to be prioritized. Specific operations include reevaluating task priorities based on the user's emotional state and selecting and navigating to an appropriate task. For example, if the server determines that the user is feeling stressed, it displays a message encouraging the user to take a break. Furthermore, if the server determines that the user is concentrating on a task, it prioritizes navigation to tasks related to that task.

[0731] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0732] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0733] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[0734] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0735] [Second embodiment]

[0736] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0737] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0738] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0739] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0740] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0741] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0742] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0743] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0744] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0745] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0746] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0747] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0748] "Example 1"

[0749] In one embodiment of the present invention, software is installed on the user's PC to collect the user's PC operation log, and the software records the user's operations in real time. This software collects information such as the type of operations performed by the user, the order of operations, and the time taken for each operation.

[0750] "Example 2"

[0751] As a means of mining tasks from collected PC operation logs, an algorithm is used to analyze the collected operation logs and find patterns of the same operations. For example, if a specific operation is repeated in a certain order, the combination of operations is recognized as a single task. "Example 3"

[0752] As a means of generating AI based on the mined tasks, we design an AI model that learns the mined task patterns. When a new operation log is input, this AI model predicts the next operation that is likely to be performed based on the task patterns it has learned in the past.

[0753] "Example 4"

[0754] As a means for the generated AI to navigate the user's PC operations and tasks, an interface is provided that presents the user with the operations predicted by the AI ​​model. For example, when the user starts a specific operation, the next operation predicted by the AI ​​model is displayed on the screen, allowing the user to select and execute that operation.

[0755] The processing flow of each embodiment will be described below.

[0756] "Example 1"

[0757] Step 1: Install software that runs on the user's PC and records user actions in real time.

[0758] Step 2: When a user operates the PC, the software records the operation. The recorded information includes the type of operation, the order of operations, and the time taken for the operation. Step 3: The recorded operation log is periodically collected and sent to the server.

[0759] "Example 2"

[0760] Step 1: The operation log sent to the server is analyzed by the task mining algorithm.

[0761] Step 2: The algorithm finds patterns of the same operations in the operation log. For example, if a specific operation is repeated in a certain order, it recognizes the combination of operations as a single task.

[0762] Step 3: The recognized tasks are stored in a database.

[0763] "Example 3"

[0764] Step 1: Use the tasks stored in the database as training data for the AI ​​model.

[0765] Step 2: The AI ​​model learns patterns for the task based on the training data.

[0766] Step 3: When a new operation log is input, the AI ​​model predicts the next likely operation based on the task patterns it has learned in the past.

[0767] "Example 4"

[0768] Step 1: Create an interface that presents the user with the operations predicted by the AI ​​model.

[0769] Step 2: When the user initiates a specific action, the AI ​​model displays the next action predicted by the user on the screen.

[0770] Step 3: The user can select from the displayed actions and perform the action.

[0771] Example 1

[0772] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0773] Conventional user operation log collection systems do not collect operation logs in real time, making it difficult to accurately grasp user operation patterns. Furthermore, analysis of collected data is often done manually, requiring efficient analysis. Furthermore, report generation based on analysis results is not automated, making it impossible to quickly provide useful information to users.

[0774] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0775] In this invention, the server includes means for installing software on the user's information processing device, means for the installed software to record user operations in real time, means for temporarily storing the recorded data in the information processing device, means for transmitting data to the server at regular time intervals, means for the server to analyze the received data, and means for the server to generate a report based on the analysis results. This makes it possible to collect user operation logs in real time, analyze them efficiently, and quickly provide useful reports.

[0776] "Information processing device" refers to devices such as computers and PCs operated by users.

[0777] "Software" refers to a program that is installed on an information processing device and records user operations.

[0778] "Real-time" means that the data is recorded immediately at the moment the user performs an action.

[0779] "Recorded data" refers to information collected as a user's operation log.

[0780] "Temporarily storing" refers to temporarily storing the recorded data in a storage device within the information processing device.

[0781] "Regular time intervals" means that the data is sent to the server periodically, for example every five minutes.

[0782] "Server" refers to a central computer system that analyzes the data it receives and generates a report of the analysis results.

[0783] "Analyzing" refers to processing collected data to identify user patterns and trends.

[0784] "Report" refers to documents or data generated based on analysis results that show user operation patterns and trends.

[0785] MODE FOR CARRYING OUT THE INVENTION

[0786] This invention is a system that installs software on a user's information processing device, records the user's operations in real time, temporarily stores the recorded data in the information processing device, sends the data to a server at regular intervals, analyzes the data received by the server, and generates a report of the analysis results.

[0787] 1. Software Installation

[0788] Users use the provided installer to install the software on their information processing devices. This software is compatible with operating systems such as Windows and macOS.

[0789] 2. Real-time recording of operations

[0790] The software installed on the device runs in the background and records user actions in real time, specifically:

[0791] Keyboard input (type and order of keys)

[0792] Mouse operations (click, drag, location information)

[0793] A timestamp for each operation

[0794] 3. Temporary storage of data

[0795] The software temporarily saves the recorded operation data to the device's local storage in a general-purpose data format such as JSON or CSV.

[0796] 4. Data transmission

[0797] The device sends the recorded data to the server at regular intervals (for example, every 5 minutes). Data transmission is secure using the HTTPS protocol.

[0798] 5. Data Analysis

[0799] The server analyzes the received data using Python's Pandas library and machine learning models (e.g., scikit-learn and TensorFlow). The goal of the analysis is to identify user behavior patterns.

[0800] 6. Generate reports

[0801] The server generates reports based on the analysis results, including information on user operation patterns, frequency, and the time it takes to perform specific operations, and the reports are displayed on a dashboard accessible to the user.

[0802] Specific examples

[0803] For example, if a user types a sentence in a text editor and then performs a search in a web browser, the specific behavior is as follows:

[0804] 09:00:01 - Start typing on the keyboard (text editor)

[0805] 09:00:15 - End of keyboard input

[0806] 09:00:20 - Mouse click (web browser icon)

[0807] 09:00:25 - Start typing on keyboard (search bar)

[0808] 09:00:30 - End of keyboard input

[0809] The recorded data is saved in local storage as operation log_20231010.json and sent to the server at 09:05:00. The server analyzes the received data and identifies patterns, such as "users tend to type text in a text editor at 9:00 AM and then perform searches in a web browser." A report is generated based on the analysis results and displayed as "Operation Patterns at 9:00 AM" on a dashboard accessible to users.

[0810] Prompt Sentence Examples

[0811] Develop software that collects users' PC operation logs and records the type, sequence, and time of operations. Include a function to send the collected data to a server and generate reports that analyze user operation patterns. Include a scenario in which input in a text editor and searches in a web browser are recorded as specific operation examples.

[0812] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0813] Step 1:

[0814] The user installs the software on their information processing device using the provided installer, and once the installation is complete, the software starts running in the background.

[0815] Input: Installer file

[0816] Output: Installed software

[0817] Step 2:

[0818] The software installed on the device records user actions in real time, including keyboard input, mouse movements, and a timestamp for each action.

[0819] Input: User actions (keyboard input, mouse operations)

[0820] Output: Operation log data (key type and sequence, click position, timestamp)

[0821] Step 3:

[0822] The software temporarily saves the recorded operation data to the device's local storage in a general-purpose data format such as JSON or CSV.

[0823] Input: Operation log data

[0824] Output: Data file saved in local storage (e.g., Operation Log_20231010.json)

[0825] Step 4:

[0826] The device sends the recorded data to the server at regular intervals (for example, every 5 minutes). Data transmission is secure using the HTTPS protocol.

[0827] Input: Data files stored in local storage

[0828] Output: Data sent to the server

[0829] Step 5:

[0830] The server analyzes the received data using Python's Pandas library and machine learning models (e.g., scikit-learn and TensorFlow). The goal of the analysis is to identify user behavior patterns.

[0831] Input: Data sent to the server

[0832] Output: Analysis results (user operation patterns and trends)

[0833] Step 6:

[0834] The server generates reports based on the analysis results, including information on user operation patterns, frequency, and the time it takes to perform specific operations, and the reports are displayed on a dashboard accessible to the user.

[0835] Input: Analysis results

[0836] Output: Generated report (displayed on the dashboard)

[0837] Specific examples

[0838] For example, if a user types a sentence in a text editor and then performs a search in a web browser, the specific behavior is as follows:

[0839] 09:00:01 - Start typing on the keyboard (text editor)

[0840] 09:00:15 - End of keyboard input

[0841] 09:00:20 - Mouse click (web browser icon)

[0842] 09:00:25 - Start typing on keyboard (search bar)

[0843] 09:00:30 - End of keyboard input

[0844] The recorded data is saved in local storage as operation log_20231010.json and sent to the server at 09:05:00. The server analyzes the received data and identifies patterns, such as "users tend to type text in a text editor at 9:00 AM and then perform searches in a web browser." A report is generated based on the analysis results and displayed as "Operation Patterns at 9:00 AM" on a dashboard accessible to users.

[0845] (Application example 1)

[0846] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0847] Conventional information processing device operation log collection systems were able to collect user operation logs, mine tasks, and generate artificial intelligence, but they lacked the functionality to collect industrial machine operation logs and provide data to optimize operation efficiency. As a result, data collection and analysis to improve the operational efficiency of industrial machines were not being carried out sufficiently.

[0848] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0849] In this invention, the server includes means for collecting user operation logs of information processing devices, means for mining tasks from the collected operation logs of information processing devices, means for generating artificial intelligence based on the mined tasks, means for navigating user operations of the information processing devices and tasks using the generated artificial intelligence, means for collecting operation logs of industrial machines and recording the type, sequence, and time of operations, and means for analyzing the recorded operation logs and providing data for optimizing operation efficiency, thereby making it possible to collect operation logs of information processing devices and optimize the operation efficiency of industrial machines.

[0850] An "information processing device" is an electronic device such as a computer or personal computer that is operated by a user.

[0851] An "operation log" is a record of the type, order, time, etc. of operations performed by a user on an information processing device.

[0852] A "task" is a series of operations or tasks that a user performs on an information processing device.

[0853] "Mining" is the process of extracting useful information and patterns from collected data.

[0854] "Artificial intelligence" refers to programs and systems that learn from collected data and make predictions and navigate.

[0855] "Navigating" means guiding and assisting users with operations and tasks.

[0856] "Industrial machinery" refers to robots and mechanical devices used in factories and manufacturing sites.

[0857] "Type of operation" refers to the specific movement or action that an industrial machine performs.

[0858] "Sequence of operations" is the order in which a series of operations are performed by industrial machinery.

[0859] "Time of operation" is the time it takes an industrial machine to perform a particular operation.

[0860] "Analysis" means examining collected data in detail to find meaning and patterns.

[0861] "Optimization" means making adjustments or improvements to maximize the efficiency of an operation.

[0862] A system for carrying out the present invention collects user operation logs of information processing devices and analyzes the operation logs of industrial machines to optimize operational efficiency. A specific embodiment of the system is described below.

[0863] System Configuration

[0864] The system consists of the following main components:

[0865] 1. User's information processing device: Electronic equipment such as a computer or personal computer operated by a user.

[0866] 2. Industrial machinery: Robots and mechanical devices used in factories and manufacturing sites.

[0867] 3. Server: A central processing unit that collects data, analyzes it, and generates artificial intelligence.

[0868] Program processing

[0869] The server processes the data in the following steps:

[0870] 1. Collecting operation logs of information processing devices:

[0871] Software installed on the user's information processing device collects operation logs in real time, recording data such as the type, order, and time of operations.

[0872] 2. Mining tasks:

[0873] The collected operation logs are mined to identify the tasks performed by the user, thereby extracting the user's operation patterns and task flow.

[0874] 3. Artificial Intelligence Generation:

[0875] Based on the mined tasks, an artificial intelligence is generated, which has the ability to learn and predict user operation patterns.

[0876] 4. Collecting industrial machine operation logs:

[0877] Sensors and software installed on industrial machines collect operation logs, including the type, sequence and time of operations.

[0878] 5. Analysis of operation logs:

[0879] Analyzes collected industrial machine operation logs and provides data to optimize operational efficiency. This analysis includes evaluation of operation patterns and efficiency.

[0880] Hardware and software used

[0881] Hardware: Information processing equipment (computers, personal computers), industrial machinery (robots, machinery), servers

[0882] Software: Log collection software, data mining software, artificial intelligence generation software, analysis software

[0883] Specific names: Python, JSON library

[0884] Specific examples

[0885] For example, when a user creates and saves a document on an information processing device, the operation log is collected. Next, when an industrial machine moves forward and lifts a box, the operation log is also collected. These logs are sent to the server and analyzed.

[0886] Prompt Sentence Examples

[0887] "Write a Python program to collect operation logs from a factory robot and record the type, order, and time of operations. Operation types include move, pick up, etc. Save the logs in JSON format."

[0888] In this way, the collection of operation logs of the information processing device and the optimization of the operational efficiency of the industrial machine are realized.

[0889] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0890] Step 1:

[0891] Software installed on the user's information processing device collects operation logs in real time. The input is user operations (e.g., clicks, keyboard input), and the output is log data that includes the type, order, and time of the operations. This log data is saved in JSON format.

[0892] Step 2:

[0893] The server receives the collected operation logs and stores them in a database. The input is the log data sent from the information processing device, and the output is the operation logs stored in the database. The database records the type, order, and time of operations.

[0894] Step 3:

[0895] The server mines tasks from the saved operation logs. The input is the operation logs stored in the database, and the output is mined task data. The task data includes the user's operation patterns and task flow.

[0896] Step 4:

[0897] The server generates an artificial intelligence model based on the mined task data. The task data is used as input, and an artificial intelligence model that learns and predicts user operation patterns is generated as output. This artificial intelligence model is built using a generative AI model.

[0898] Step 5:

[0899] Sensors and software installed on industrial machines collect operation logs. The input is the operation of the industrial machine (e.g., moving, picking up), and the output is log data that includes the type, sequence, and time of the operation. This log data is also saved in JSON format.

[0900] Step 6:

[0901] The server receives collected operation logs of industrial machines and stores them in a database. The input is the log data sent from the industrial machines, and the output is the operation logs stored in the database. The database records the type, sequence, and time of operations.

[0902] Step 7:

[0903] The server analyzes the saved operation logs of industrial machines. The input is the operation logs stored in the database, and the output is data generated to optimize the efficiency of the operation. This analysis includes evaluation of operation patterns and efficiency.

[0904] Step 8:

[0905] The server provides feedback based on the analysis results to optimize the operational efficiency of the industrial machinery. The analysis results are input, and optimized operating procedures and improvement suggestions are generated as output. This feedback is sent to the control system of the industrial machinery.

[0906] Through the above steps, the collection of operation logs of information processing devices and the optimization of the operational efficiency of industrial machines are realized.

[0907] Example 2

[0908] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0909] Conventional systems for analyzing information processing device operation logs have had difficulty efficiently detecting user operation patterns and defining them as tasks. Furthermore, they were unable to navigate user operations and tasks using AI models generated based on the detected tasks. This resulted in the issue of being unable to improve user work efficiency.

[0910] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0911] In this invention, the server includes means for collecting information processing device operation logs of users, means for preprocessing the collected information processing device operation logs, means for detecting the same operation patterns from the preprocessed operation logs, means for defining and saving tasks based on the detected operation patterns, means for visualizing the defined tasks and generating reports, means for creating a generative AI model based on the generated reports, and means for navigating the user's information processing device operations and tasks using the generated AI model. This makes it possible to efficiently detect the user's operation patterns and define them as tasks, and further to navigate the user's operations and tasks using the generated AI model, thereby improving the user's work efficiency.

[0912] The "information processing device operation log" is a record of operations performed by a user on an information processing device.

[0913] "Preprocessing" is the process of cleaning and organizing collected data to convert it into a form suitable for analysis.

[0914] An "operation pattern" is a sequence or combination of a series of operations that a user repeatedly performs on an information processing device.

[0915] A "task" is a set of operations with a specific purpose that is defined based on the detected operation pattern.

[0916] "Visualization" is the presentation of data or information in a visual format such as a graph or chart.

[0917] A "report" is a document or data that summarizes analysis results and statistical information.

[0918] A "generative AI model" is an artificial intelligence model created using machine learning algorithms based on collected data.

[0919] "Navigating" means guiding and assisting users with operations and tasks.

[0920] This invention is a system that collects and analyzes user operation logs of information processing devices to detect user operation patterns and define them as tasks. Furthermore, it can use the generated AI model to navigate user operations and tasks.

[0921] The server installs log collection software on the terminal to collect the user's information processing device operation log. This software records user operations (clicks, keystrokes, application startup / exit, etc.) in real time and periodically sends them to the server.

[0922] The server preprocesses the collected operation logs. Specifically, it uses data analysis software such as Python or R to clean the log data (removing noise and filling in missing data) and convert it into a format suitable for analysis. For example, it organizes the order of operations based on timestamps and classifies them by type of operation.

[0923] The server then applies a pattern recognition algorithm to the preprocessed operation log. Specifically, it uses a machine learning algorithm (e.g., k-means clustering or sequence mining algorithm) to detect patterns in which the same operations are repeated in a certain order, thereby identifying combinations of operations that users frequently perform.

[0924] The server defines tasks based on the detected operation patterns and stores them in a database. Specifically, it assigns a task name to each pattern and records details of the related operations (type, order, frequency, etc.). For example, a task called "check email and open files every day" might be defined.

[0925] The server also visualizes the defined tasks and generates reports, displaying statistical information such as task execution frequency and time period in graphs and charts for easy understanding by users and administrators. The generated reports are available via a web interface and in PDF format.

[0926] Based on the generated reports, the server creates a generative AI model that can learn and predict user operation patterns. For example, if a user tends to perform a specific operation at a specific time, the model can predict the operation in advance and notify the user.

[0927] For example, if a user checks email at a specific time every day, then opens a specific file, and then launches a specific piece of software, the server will recognize these operations as a single task, which is defined as "checking email and opening a file every day."

[0928] Examples of prompts to input to a generative AI model might include:

[0929] "Analyze the patterns of operations that users perform daily and identify them as tasks. For example, if operations such as checking email, opening files, and launching software are repeated, define them as one task."

[0930] In this way, the server analyzes the collected operation logs, and realizes a system that recognizes and visualizes the user's repeated operations as tasks, thereby improving the user's work efficiency.

[0931] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0932] Step 1:

[0933] Collecting operation logs

[0934] The server collects the user's operation logs for the information processing device. The log collection software installed on the terminal records the user's operations (clicks, keystrokes, application startup / exit, etc.) in real time and periodically sends them to the server.

[0935] Input: User operation data

[0936] Output: Collected operation log data

[0937] Step 2:

[0938] Preprocessing of operation logs

[0939] The server preprocesses the collected operation logs. Specifically, it uses data analysis software such as Python or R to clean the log data (removing noise and filling in missing data) and convert it into a format suitable for analysis. For example, it organizes the order of operations based on timestamps and classifies them by type of operation.

[0940] Input: Collected operation log data

[0941] Output: Preprocessed audit log data

[0942] Step 3:

[0943] Applying pattern recognition algorithms

[0944] The server applies a pattern recognition algorithm to the preprocessed operation log. Specifically, it uses a machine learning algorithm (e.g., k-means clustering or sequence mining algorithm) to detect patterns in which the same operations are repeated in a certain order. This allows it to identify combinations of operations that users frequently perform.

[0945] Input: Preprocessed operation log data

[0946] Output: Detected operation pattern

[0947] Step 4:

[0948] Define and save the task

[0949] The server defines tasks based on the detected operation patterns and stores them in a database. Specifically, it assigns a task name to each pattern and records details of the related operations (type, order, frequency, etc.). For example, a task called "check email and open files every day" might be defined.

[0950] Input: Detected operation pattern

[0951] Output: Defined task data

[0952] Step 5:

[0953] Task visualization and reporting

[0954] The server visualizes the defined tasks and generates reports. Specifically, it displays statistical information such as task execution frequency and time period in graphs and charts, making it easy for users and administrators to understand. The generated reports are available via a web interface or in PDF format.

[0955] Input: Defined task data

[0956] Output: Visualized report

[0957] Step 6:

[0958] Creating generative AI models

[0959] The server creates a generative AI model based on the generated report. This AI model can learn and predict user operation patterns. For example, if a user tends to perform a specific operation at a specific time, it can predict that operation in advance and notify the user.

[0960] Input: Visualized report

[0961] Output: The generated AI model

[0962] Step 7:

[0963] Navigate user actions and tasks

[0964] The server uses the generated AI model to navigate the user's operations and tasks. Specifically, it predicts the operations and tasks the user needs to perform in advance and provides notifications and guidance at the appropriate time, thereby improving the user's work efficiency.

[0965] Input: Generated AI model

[0966] Output: The operation or task that was navigated to

[0967] (Application example 2)

[0968] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0969] In conventional systems, it was difficult to find efficient task patterns simply by collecting user operation logs. Furthermore, there was a lack of means to automatically analyze operation patterns and recognize them as optimal tasks, making it impossible to improve user work efficiency. Furthermore, automated systems such as factory robots are also required to analyze operation logs to find efficient tasks, but current technology is not sufficient to address this need.

[0970] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0971] In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating artificial intelligence based on the mined tasks, means for navigating user operations and tasks using the generated artificial intelligence, means for extracting features from the operation logs and finding operation patterns using a clustering algorithm, and means for extracting the found operation patterns as task patterns. This makes it possible to efficiently analyze user operation logs and automatically find optimal task patterns.

[0972] "User" means the human or end user who operates the system.

[0973] An "operation log" is a record of a series of operations performed by a user on a system.

[0974] A "task" is a set of operations performed to achieve a specific goal.

[0975] "Mining" is the process of extracting useful information and patterns from data.

[0976] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[0977] "Navigating" means guiding a user through an operation or task.

[0978] A "feature" is an attribute or variable that is used to find specific patterns or trends in data.

[0979] A "clustering algorithm" is a computational method for dividing data into groups based on similarity.

[0980] An "operation pattern" is a tendency or regularity in the repetition of a series of operations performed by a user.

[0981] A "task pattern" is a set of operations required to achieve a specific goal.

[0982] The system for carrying out this invention collects and analyzes user operation logs to find and optimize efficient task patterns. A specific embodiment of this system is described below.

[0983] System Configuration

[0984] The system consists of the following main components:

[0985] 1. Operation log collection module: Collects user operation logs.

[0986] 2. Task mining module: Mines tasks from the collected operation logs.

[0987] 3. Artificial intelligence generation module: Generates artificial intelligence based on the mined tasks.

[0988] 4. Navigation module: The generated AI navigates the user through operations and tasks.

[0989] 5. Feature extraction module: Extracts features from the operation log.

[0990] 6. Clustering module: Uses a clustering algorithm to find operation patterns.

[0991] 7. Task pattern extraction module: Extracts the discovered operation patterns as task patterns.

[0992] Hardware and software used

[0993] Hardware: Factory robots, sensors for data collection

[0994] Software: Python, Pandas, Scikit-learn

[0995] Data processing and calculation

[0996] 1. Operation log collection: The operation log collection module records user operations in real time and stores them in a database.

[0997] 2. Feature Extraction: The feature extraction module extracts features such as operation type, duration, and parameters from the operation log.

[0998] 3. Clustering: The clustering module clusters the operation patterns using a clustering algorithm such as DBSCAN.

[0999] 4. Task Pattern Extraction: The task pattern extraction module extracts task patterns from the clustering results and recognizes them as efficient tasks.

[1000] 5. AI generation: The AI ​​generation module generates AI based on the extracted task patterns.

[1001] 6. Navigation: The navigation module uses the generated artificial intelligence to navigate the user through operations and tasks.

[1002] Specific examples

[1003] For example, in a part assembly operation in a factory, the operation log collection module collects the operation logs of a robot. The feature extraction module extracts the operation type, duration, and parameters from the operation logs, and the clustering module uses these features to cluster the operation patterns. The task pattern extraction module extracts efficient task patterns from the clustering results, and the artificial intelligence generation module generates artificial intelligence based on these. Finally, the navigation module uses the generated artificial intelligence to navigate the robot's operations and tasks.

[1004] Example prompt sentence:

[1005] Write a program to analyze the operation log data of a factory robot and find efficient task patterns. The operation log contains operation types, durations, and parameters. Use the DBSCAN clustering algorithm to cluster the operation patterns and extract the operations that belong to each cluster as task patterns.

[1006] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1007] Step 1:

[1008] Collecting operation logs

[1009] The server collects user operation logs in real time. Specifically, it records the operations performed by users on the system (clicks, keyboard input, scrolling, etc.) and stores them in a database. The input is the user operation data, and the output is the operation log stored in the database.

[1010] Step 2:

[1011] Feature extraction

[1012] The server extracts features from the collected operation logs. Specifically, it extracts features such as operation type, duration, and parameters for each entry in the operation log. The input is the operation log data, and the output is a dataset containing the features.

[1013] Step 3:

[1014] Clustering

[1015] The server clusters operation patterns using the features. Specifically, it uses a clustering algorithm such as DBSCAN to group similar operations. The input is a feature dataset, and the output is a dataset with cluster labels.

[1016] Step 4:

[1017] Task Pattern Extraction

[1018] The server extracts task patterns from the clustering results. Specifically, it analyzes the operations belonging to each cluster and recognizes them as efficient task patterns. The input is a dataset with cluster labels, and the output is task patterns.

[1019] Step 5:

[1020] Artificial Intelligence Generation

[1021] The server generates artificial intelligence based on the extracted task patterns. Specifically, it uses the task patterns as learning data to train a generative AI model. The input is the task patterns, and the output is the trained artificial intelligence model.

[1022] Step 6:

[1023] Navigation

[1024] The server uses the generated AI to navigate the user's operations and tasks. Specifically, it monitors the user's current operation status and guides the user to the next operation based on the optimal task pattern. The input is the user's current operation data and the AI ​​model, and the output is navigation instructions.

[1025] Example 3

[1026] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1027] In conventional systems, simply collecting user operation logs made it difficult to predict subsequent operations and provide appropriate navigation to users. Furthermore, the AI ​​model for predicting the next operation a user should perform was insufficiently trained, making it impossible to improve user operation efficiency. Furthermore, the system lacked the ability to determine the user's task priorities and navigate accordingly, resulting in a decrease in user work efficiency.

[1028] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1029] In this invention, the server includes means for collecting user operation logs, means for saving the collected operation logs, means for training an AI model using the saved operation logs, means for collecting new operation logs in real time, means for predicting the next operation, means for presenting the predicted results to the user, and means for the user to select and execute the presented predicted results. This makes it possible to predict the next operation with high accuracy based on the user's operation log and provide appropriate navigation to the user. Furthermore, by determining the priority of the user's tasks and navigating based on that, it is possible to significantly improve the user's work efficiency.

[1030] A "user operation log" is data that records a series of operations performed by a user in chronological order.

[1031] "Means of collection" refers to functions and devices for monitoring and recording user operation logs in real time.

[1032] The "storing means" refers to a function or device for storing the collected operation logs in a storage device such as a database.

[1033] An "AI model" is a model that uses machine learning algorithms to learn user operation patterns and predict the next operation.

[1034] "Training means" refers to functions or devices that train an AI model using saved operation logs.

[1035] "Means for collecting data in real time" refers to functions or devices that instantly record new operations performed by the user and send them to a server.

[1036] "Prediction means" refers to functions or devices that use an AI model to estimate the next operation that is likely to be performed.

[1037] "Presentation means" refers to a function or device for visually or audibly notifying the user of the prediction results.

[1038] The "means for selecting and executing" refers to a function or device that allows a user to select a presented prediction result and execute an operation based on that selection.

[1039] MODE FOR CARRYING OUT THE INVENTION

[1040] This invention is a system that collects user operation logs, predicts the next operation using an AI model, and presents it to the user. A specific embodiment of this system will be described below.

[1041] System configuration

[1042] The system includes a terminal that collects user operation logs, a server that stores the collected operation logs and trains the AI ​​model, and an interface that presents prediction results to the user.

[1043] Hardware and software used

[1044] Hardware:

[1045] Device: PC or smartphone for recording user actions in real time

[1046] Server: A high-performance computer for storing operation logs and training AI models

[1047] software:

[1048] Operation log collection software: A program for recording user operations

[1049] Database: A database management system for storing operation logs (e.g., MySQL, PostgreSQL)

[1050] Machine learning framework: Software for training AI models (e.g., TensorFlow, PyTorch)

[1051] User Interface: GUI for presenting prediction results to users (e.g. HTML, CSS, JavaScript)

[1052] System Operation

[1053] 1. Collecting and saving operation logs:

[1054] The terminal monitors the series of operations performed by the user in real time and records them as an operation log, which is then sent to a server and stored in a database.

[1055] 2. Training the AI ​​model:

[1056] The server uses the saved operation logs to train an AI model, specifically learning operation patterns using machine learning frameworks such as TensorFlow and PyTorch.

[1057] 3. Real-time operation log collection and prediction:

[1058] When a user initiates a new operation, the device sends a log of that operation in real time to the server, which then inputs the received operation log into the AI ​​model to predict the most likely next operation.

[1059] 4. Prediction and execution:

[1060] The server returns the prediction results to the device, which then presents them to the user. When the user selects a prediction result, the device executes the corresponding operation.

[1061] Specific examples

[1062] For example, suppose a user begins to enter data into Excel. The device records this operation as an operation log and sends it to the server. The server saves the received operation log in a database and predicts the next operation using an AI model. Based on past operation patterns, the AI ​​model predicts that the user is likely to graph the data next. The server sends this prediction result back to the device, and the device displays a message to the user asking, "Do you want to graph the data next?" If the user accepts this suggestion, the device automatically starts creating a graph in Excel.

[1063] Prompt Sentence Examples

[1064] "When a user starts entering data into Excel, please design an AI model that predicts the next operation that the user is likely to perform and suggests it to the user." The flow of the specific processing in Example 3 will be described with reference to Figure 15.

[1065] Step 1:

[1066] The terminal collects user operation logs. Specifically, it monitors a series of user operations (e.g., data entry in Excel, web browser operations, etc.) in real time and records them as an operation log. The input is the user's operation, and the output is the operation log data.

[1067] Step 2:

[1068] The terminal sends the collected operation logs to the server. The server stores the received operation logs in a database. The input is the operation log data, and the output is the operation log stored in the database.

[1069] Step 3:

[1070] The server uses the stored operation logs to train an AI model. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to learn operation patterns. The input is the operation logs stored in the database, and the output is the trained AI model.

[1071] Step 4:

[1072] When a user starts a new operation, the terminal sends the operation log to the server in real time. The input is the new operation log, and the output is the operation log sent to the server.

[1073] Step 5:

[1074] The server inputs the received operation log into the AI ​​model and predicts the next operation that is likely to be performed. Specifically, the AI ​​model makes predictions based on operation patterns it has learned in the past. The input is the new operation log and the trained AI model, and the output is the predicted next operation.

[1075] Step 6:

[1076] The server returns the prediction results to the terminal. The terminal presents the prediction results to the user. Specifically, it displays a message such as "Would you like to graph the data next?" on the screen. The input is the predicted next operation, and the output is the prediction result presented to the user.

[1077] Step 7:

[1078] When the user selects a predicted result, the terminal executes the operation. Specifically, when the user selects "Graph," the terminal automatically starts creating a graph in Excel. The input is the user's selection, and the output is the executed operation.

[1079] (Application example 3)

[1080] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1081] Conventional factory robots could collect operation logs but were unable to effectively utilize that data, making it difficult to predict the next operation. This resulted in problems such as reduced robot operating efficiency and difficulty in optimizing work. Furthermore, there were insufficient means for users to navigate through operations and tasks, which often led to operational errors and delays.

[1082] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating AI based on the mined tasks, means for navigating user operations and tasks using the generated AI, means for collecting robot operation logs and learning operation patterns, means for predicting the next operation to be performed when a new operation log is input, and means for displaying the predicted operation on the robot's interface so that the robot can select and execute the operation. This improves the operating efficiency of the robot and enables work optimization.

[1083] A "user operation log" is a record of a series of operations performed by a user.

[1084] "Task mining" means extracting specific tasks and procedures from collected operation logs.

[1085] "Generating AI" means building an artificial intelligence model based on collected data.

[1086] "Navigating user operations and tasks" means presenting the next operation or task predicted by the AI ​​to the user and assisting them with their work.

[1087] A "robot operation log" is a record of a series of operations performed by a robot.

[1088] "Learning operation patterns" means that the AI ​​learns common operating procedures and trends from collected operation logs.

[1089] "Predicting the next operation" means that when a new operation log is input, the AI ​​guesses what operation should be performed next.

[1090] A "robot interface" is a point of contact through which a robot exchanges information with users and other systems.

[1091] "Selecting and executing an operation" means that the robot selects a predicted operation and actually performs that operation.

[1092] A system for carrying out this invention includes means for collecting user operation logs, mining tasks from the collected operation logs, and generating AI based on the mined tasks. It also includes means for navigating user operations and tasks using the generated AI. It also includes means for collecting robot operation logs and learning operation patterns, means for predicting the next operation to be performed when a new operation log is input, and means for displaying the predicted operation on the robot's interface so that the robot can select and execute the operation.

[1093] Hardware and software used

[1094] Hardware: Factory robots (e.g., general-purpose robotic arms), high-performance GPUs (e.g., NVIDIA Tesla)

[1095] Software: TensorFlow (used to build and train AI models), Python (programming language)

[1096] Data processing and calculation

[1097] Data collection

[1098] The server collects user operation logs and robot operation logs, which include a series of operations performed by the user and a series of operations performed by the robot.

[1099] Task Mining

[1100] The server extracts specific tasks and procedures from the collected operation logs, which makes task patterns clear.

[1101] Generating AI models

[1102] The server generates an AI model based on the mined tasks, which is built using TensorFlow and learns patterns from the operation logs.

[1103] Operation Prediction

[1104] When a new operation log is input, the server uses the AI ​​model to predict the next operation to be performed, and the prediction results are displayed on the robot's interface.

[1105] Executing an operation

[1106] The robot selects the predicted operation from the interface and executes it, improving the robot's operational efficiency and optimizing the work.

[1107] Specific examples

[1108] For example, when a factory robot assembles parts, it can predict the next part or tool to be installed based on past operation logs and provide this information to the robot, thereby improving work efficiency and reducing the occurrence of errors.

[1109] Prompt Sentence Examples

[1110] "Design an AI model that predicts the next operation that a factory robot should perform based on past operation logs when assembling parts, and presents this to the robot."

[1111] In this way, a smart navigation system can be built to optimize the operation of factory robots.

[1112] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1113] Step 1:

[1114] The server collects user operation logs and robot operation logs. The input includes a series of operations performed by the user and a series of operations performed by the robot. These operation logs are stored in a database. The output is the collected operation logs.

[1115] Step 2:

[1116] The server extracts specific tasks and procedures from the collected operation logs. The operation logs collected in step 1 are used as input. Data mining techniques are used to extract task patterns from the operation logs. The output is the mined tasks.

[1117] Step 3:

[1118] The server generates an AI model based on the mined tasks. The task patterns obtained in step 2 are used as input. TensorFlow is used to build an AI model that learns patterns in the operation logs. The generated AI model is obtained as output.

[1119] Step 4:

[1120] When a new operation log is input, the server uses the AI ​​model to predict the next operation to be performed. The new operation log and the AI ​​model generated in step 3 are used as input. The AI ​​model predicts the next operation based on the pattern in the operation log. The predicted operation is obtained as output.

[1121] Step 5:

[1122] The server displays the predicted operation on the robot's interface. As input, the predicted operation obtained in step 4 is used. The predicted operation is displayed on the robot's interface, allowing the robot to select and execute the operation. As output, the operation displayed on the robot's interface is obtained.

[1123] Step 6:

[1124] The robot selects a predicted operation from the interface and executes it. As input, it uses the operation displayed in step 5. The robot executes the selected operation and proceeds with its work. As output, it obtains the executed operation.

[1125] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1126] "Example 1"

[1127] In one embodiment of the present invention, an emotion engine is incorporated into a system that includes means for collecting a user's PC operation log, means for mining tasks from the collected PC operation log, means for generating an AI based on the mined tasks, and means for navigating the user's PC operations and tasks using the generated AI. This emotion engine analyzes the user's emotional state while using the PC in real time and feeds the results back to the AI.

[1128] "Example 2"

[1129] Specifically, the emotion engine estimates a user's emotional state, such as stress level and concentration level, based on their keyboard typing speed, mouse movements, and PC usage time. For example, if the keyboard typing speed is fast and the mouse movements are large, it determines that the user may be feeling stressed. Also, if the PC usage time is long, it determines that the user may be concentrating on their work.

[1130] "Example 3"

[1131] Based on feedback from the emotion engine, the AI ​​provides task navigation that adapts to the user's emotional state. For example, if the AI ​​determines that the user is feeling stressed, it will prioritize tasks to reduce stress (such as displaying a message encouraging them to take a break). If the AI ​​determines that the user is concentrating on a task, it will prioritize tasks related to that task.

[1132] The processing flow of each embodiment will be described below.

[1133] "Example 1"

[1134] Step 1: Collect user PC operation logs.

[1135] Step 2: Mine tasks from the collected PC operation logs.

[1136] Step 3: Generate AI based on the mined tasks.

[1137] Step 4: The generated AI guides the user through PC operations and tasks.

[1138] Step 5: Analyze the user's emotional state in real time while using the PC and provide feedback to the AI.

[1139] "Example 2"

[1140] Step 1: Estimate the user's emotional state, such as stress level and concentration level, based on the user's keyboard typing speed, mouse movements, and PC usage time.

[1141] Step 2: Feed back the estimated emotional state to the AI.

[1142] "Example 3"

[1143] Step 1: The AI ​​provides task navigation adapted to the user's emotional state based on feedback from the emotion engine.

[1144] Step 2: If the AI ​​determines that the user is stressed, it prioritizes tasks to reduce stress (for example, displaying a message encouraging the user to take a break).

[1145] Step 3: If the AI ​​determines that the user is concentrating on a task, it will prioritize navigating tasks related to that task.

[1146] Example 1

[1147] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1148] Conventional information processing device operation support systems were able to collect user operation logs, mine tasks, and generate AI, but they did not provide support that took into account the user's emotional state. As a result, there were operations that users found stressful or that they could not perform efficiently, and the user experience was not sufficiently improved. In addition, the system was limited in its ability to learn and predict user operation patterns and determine task priorities. This led to problems such as reduced user operation efficiency and satisfaction.

[1149] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1150] In this invention, the server includes means for collecting a user's information processing device operation log, means for mining tasks from the collected information processing device operation log, means for generating an artificial intelligence based on the mined tasks, means for navigating the user's information processing device operations and tasks using the generated artificial intelligence, means for analyzing the user's emotional state during operation in real time, and means for feeding back the analysis results to the artificial intelligence. This makes it possible to learn and predict the user's operation patterns and further enable support that takes the user's emotional state into consideration. This can improve the user's operation efficiency and satisfaction.

[1151] A "user" is an individual or group that operates an information processing device.

[1152] An "information processing device" is an electronic device such as a computer or tablet that is operated by a user.

[1153] An "operation log" is a record of operations performed by a user on an information processing device.

[1154] A "task" is a series of operations or tasks that a user performs on an information processing device.

[1155] "Mining" is the process of extracting useful information and patterns from collected operation logs.

[1156] "Artificial intelligence" refers to algorithms and models that learn user operation patterns and make predictions and provide assistance.

[1157] "Navigation" means providing instructions and suggestions to assist users in their operations and tasks, and to make them more efficient.

[1158] "Emotional state" refers to the psychological state, such as stress or joy, that a user feels while operating the device.

[1159] "Real-time analysis" is the process of instantly assessing a user's emotional state while they are interacting.

[1160] "Feedback" is the process of reflecting the results of analysis into artificial intelligence.

[1161] MODE FOR CARRYING OUT THE INVENTION

[1162] This invention is a system that collects user operation logs of information processing devices, mines tasks from the collected operation logs, generates artificial intelligence based on the mined tasks, and uses the generated artificial intelligence to navigate the user's operations and tasks. It also aims to improve user operation efficiency and satisfaction by analyzing the user's emotional state during operation in real time and feeding the results back to the artificial intelligence.

[1163] 1. Collection of user operation logs for information processing devices

[1164] Users install dedicated software on their information processing devices. This software runs in the background and records user operations in real time. Specifically, it collects information such as the type of operation (e.g., clicks, keyboard input, window switching, etc.), the order of operations, and the time taken for the operation. For example, if a user opens a browser, visits a specific website, and then sends an email, the entire series of operations is recorded in detail.

[1165] 2. Task Mining

[1166] The device periodically sends the recorded operation log to the server. The server receives it and stores it in a database. The server analyzes the received operation log and extracts tasks from the series of operations performed by the user. For example, the user's daily email check and reply operation is recognized as one task.

[1167] 3. Generating an AI model

[1168] The server uses the mined work data to generate an AI model that learns the user's operation patterns. This AI model predicts the user's operations and makes suggestions for improving efficiency. The server then distributes the generated AI model to the device. The device then uses this AI model to navigate the user's operations.

[1169] 4. Incorporating an Emotional Engine

[1170] An emotion engine built into the device analyzes the user's emotional state in real time while they are operating the device. For example, it can detect emotions such as stress or joy from the user's facial expressions and voice. The device then sends the analyzed emotional data to a server, which receives it and stores it in a database. The server then feeds the received emotional data back to the AI ​​model, allowing the AI ​​model to navigate operations while taking the user's emotional state into account. For example, if the user is feeling stressed, the AI ​​will suggest ways to simplify operations or provide assistance.

[1171] Examples of concrete examples and prompts

[1172] Specific examples

[1173] Let's say a user checks their email at 9 a.m. every morning and replies to important emails. This operation log is collected and mined as work. Next, an AI model is generated that automatically opens the email client at the time the user checks email and navigates to prioritize important emails. If the user feels stressed while checking email, the emotion engine feeds that information back to the AI, which then suggests ways to simplify the operation.

[1174] Prompt Sentence Examples

[1175] "Please record the email check and reply operations that a user performs every morning at 9am, and generate an AI model to automate this operation. Also, please use an emotion engine to analyze the stress the user feels while performing this operation, and feed the results back to the AI ​​to suggest ways to simplify the operation."

[1176] In this way, by collecting the user's information processing device operation logs, mining the work, generating an artificial intelligence model, and incorporating an emotion engine, a system is realized that navigates the user's operations and improves efficiency.

[1177] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1178] Step 1:

[1179] The user installs the dedicated software into the information processing device.

[1180] Input: Dedicated software installer

[1181] Output: Installed software

[1182] How it works: The user downloads the dedicated software installer and installs it on their information processing device by following the installation wizard. Once the installation is complete, the software starts running in the background.

[1183] Step 2:

[1184] The device records user actions in real time.

[1185] Input: User actions (clicks, keyboard input, window switching, etc.)

[1186] Output: Operation log

[1187] Specific actions: Software installed on the device monitors user actions in real time and records information such as the type, sequence, and time of actions. For example, a user opening a browser, visiting a specific website, and then sending an email is recorded.

[1188] Step 3:

[1189] The operation log recorded by the terminal is sent to the server.

[1190] Input: Operation log

[1191] Output: Operation log sent to the server

[1192] Specific operation: The terminal sends operation logs to the server at regular intervals, and the server stores the received operation logs in a database.

[1193] Step 4:

[1194] The server analyzes the operation log and mines the work.

[1195] Input: Operation log

[1196] Output: Mined work

[1197] Specific operation: The server analyzes the received operation log and extracts tasks from the series of operations performed by the user. For example, the user's daily morning email check and reply operation is recognized as one task.

[1198] Step 5:

[1199] The server generates an artificial intelligence model based on the mined work.

[1200] Input: Mined work

[1201] Output: Artificial Intelligence Model

[1202] Specific operation: Based on the mined operation data, the server generates an AI model that learns the user's operation patterns. This AI model predicts the user's operations and makes suggestions to improve efficiency.

[1203] Step 6:

[1204] The server distributes the generated artificial intelligence model to the terminal.

[1205] Input: Artificial Intelligence Model

[1206] Output: AI model delivered to the device

[1207] Specific operation: The server delivers the generated AI model to the terminal, which then uses the AI ​​model to navigate the user's operations.

[1208] Step 7:

[1209] The device analyzes the user's emotional state in real time while operating the device.

[1210] Input: Data during user operation (facial expressions, voice, etc.)

[1211] Output: Emotional state data

[1212] Specific operation: The emotion engine built into the device analyzes the user's facial expressions and voice in real time while operating the device, detecting emotional states such as stress or joy.

[1213] Step 8:

[1214] The terminal transmits the emotional state data to the server.

[1215] Input: Emotional state data

[1216] Output: Emotional state data sent to the server

[1217] Specific operation: The device sends the analyzed emotional state data to the server, which receives it and stores it in a database.

[1218] Step 9:

[1219] The server feeds emotional state data back to the artificial intelligence model.

[1220] Input: Emotional state data

[1221] Output: fed back artificial intelligence model

[1222] Specific operation: The server feeds the received emotional state data back to the AI ​​model, allowing the AI ​​model to perform operation navigation that takes the user's emotional state into account. For example, if the user is feeling stressed, the AI ​​will suggest ways to simplify operations or provide assistance.

[1223] (Application example 1)

[1224] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1225] Conventional factory robots lack navigation systems to improve work efficiency and accuracy, and in particular, they are not optimized to take into account the emotional state of the robot during operation. This results in issues such as reduced robot operation efficiency and insufficient work accuracy. Furthermore, the lack of a system for mining tasks based on operation logs or providing feedback on emotional states makes it difficult to optimize robot operations in real time.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1227] In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating artificial intelligence based on the mined tasks, means for navigating user operations and tasks using the generated artificial intelligence, means for analyzing emotional states during operations in real time using an emotion engine and feeding the results back to the artificial intelligence, and means for optimizing work efficiency based on the operation logs and emotional states, thereby optimizing the operating efficiency of factory robots and improving work accuracy.

[1228] An "operation log" is a record of the type, order, and time of operations performed by a user or robot.

[1229] "Task mining" is a technique for extracting and analyzing specific tasks and processes from collected operation logs.

[1230] "Artificial intelligence" is a computer program that learns from collected data and makes predictions and navigates.

[1231] "Means of navigating" refers to technology that instructs users and robots on optimal operations and task progression.

[1232] The "emotion engine" is a system that analyzes the emotional state during operation in real time and provides feedback based on the results.

[1233] "Means for optimizing work efficiency" is a technology for maximizing work efficiency based on operation logs and emotional states.

[1234] A system for implementing the present invention includes an application called "Smart Factory Navigator" that is installed on a factory robot. A specific embodiment of this system will be described below.

[1235] System configuration

[1236] The system consists of the following main components:

[1237] 1. Operation log collection method: Collect operation logs of factory robots in real time.

[1238] 2. Task mining method: Tasks are mined from the collected operation logs.

[1239] 3. AI generation means: Generate AI based on the mined tasks.

[1240] 4. Navigation: The generated artificial intelligence navigates the robot through operations and tasks.

[1241] 5. Emotion engine: Analyzes the emotional state during operation in real time and feeds the results back to the artificial intelligence.

[1242] 6. Work efficiency optimization method: Optimize work efficiency based on operation logs and emotional state.

[1243] Hardware and software used

[1244] Hardware: Factory robots (e.g., general-purpose robotic arms), emotion sensors (e.g., biosignal sensors)

[1245] Software: Python, EmotionEngine library, TaskMining library, AI Navigator library

[1246] Data processing and calculation

[1247] 1. Operation log collection: Collect factory robot operation logs (type of operation, sequence, and time) in real time. This includes data from the robot's motion sensors and control system.

[1248] 2. Emotional state analysis: The emotion engine analyzes the emotional state of the robot in real time while it is operating. It analyzes bio-signal data obtained from emotion sensors and evaluates emotional states such as stress and concentration.

[1249] 3. Task mining: Extract and analyze specific tasks and processes from collected operation logs, thereby identifying the priority and efficient procedures for tasks that robots should perform.

[1250] 4. AI generation and navigation: AI is generated based on the mined tasks to navigate the robot's operations and tasks. The AI ​​proposes optimal work procedures based on the operation log and emotional state.

[1251] Specific examples

[1252] For example, if a robot performing parts assembly work in a factory collects operation logs and detects stress using its emotion engine, the AI ​​will adjust the pace of the work and navigate to reduce stress, optimizing the robot's operational efficiency and improving its work accuracy.

[1253] Prompt Sentence Examples

[1254] "How can we collect the operation logs of factory robots, mine their tasks, generate artificial intelligence, and adjust the pace of work if the emotion engine detects a state of stress?"

[1255] In this way, the smart factory navigator can optimize the operating efficiency of factory robots and improve work accuracy.

[1256] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1257] Step 1:

[1258] The server collects the operation logs of factory robots in real time. The operation logs include the type, sequence, and time of operations. This accumulates the robot's operation data. The input is data from the robot's operation sensors and control system, and the output is a database of operation logs.

[1259] Step 2:

[1260] The server collects biosignal data obtained from the emotion sensor in real time. The emotion engine analyzes this data and evaluates the user's emotional state, such as stress and concentration. The input is the biosignal data from the emotion sensor, and the output is the evaluation result of the user's emotional state.

[1261] Step 3:

[1262] The server mines tasks from the collected operation logs. Specifically, it analyzes the operation logs and extracts specific tasks and processes. The input is a database of operation logs, and the output is a list of mined tasks.

[1263] Step 4:

[1264] The server generates an AI based on the mined tasks. The generated AI proposes optimal work procedures based on the operation log and emotional state. The input is the list of mined tasks and the evaluation result of the emotional state, and the output is the generated AI model.

[1265] Step 5:

[1266] The server uses the generated AI to navigate the factory robot's operations and tasks. Specifically, the AI ​​instructs the robot on optimal work procedures and adjusts the pace of work. The input is the generated AI model, and the output is navigation instructions for the robot.

[1267] Step 6:

[1268] The server optimizes work efficiency based on the operation log and the user's emotional state. Specifically, the AI ​​provides real-time feedback on the user's emotional state and adjusts the pace and procedures of the work. The input is the operation log and the evaluation results of the user's emotional state, and the output is an optimized work procedure.

[1269] In this way, the server can optimize the operating efficiency of the factory robots and improve the accuracy of their work.

[1270] Example 2

[1271] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1272] Conventional systems for collecting and analyzing information processing device operation logs have had difficulty identifying user operation patterns and mining tasks. It has also been difficult to estimate a user's emotional state and provide appropriate navigation based on that. This has led to issues such as insufficient work efficiency and stress management for users.

[1273] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1274] In this invention, the server includes means for collecting a user's information processing device operation log, means for mining tasks from the collected information processing device operation log, means for generating artificial intelligence based on the mined tasks, means for navigating the user's information processing device operations and tasks using the generated artificial intelligence, means for analyzing the user's operation log data and identifying operation patterns, means for estimating the user's emotional state from the operation log data, and means for providing the analysis results to the user. This makes it possible to identify the user's operation patterns and efficiently mine tasks. Furthermore, by estimating the user's emotional state and providing appropriate navigation based on that, it is possible to improve the user's work efficiency and stress management.

[1275] "User" refers to an individual or group that operates an information processing device.

[1276] "Information processing device" refers to electronic devices such as computers, tablets, and smartphones.

[1277] The term "operation log" refers to data that records input data and operation history when a user operates an information processing device.

[1278] A "task" refers to a series of operations or tasks that a user performs on an information processing device.

[1279] "Mining" refers to the process of extracting useful information and patterns from collected data.

[1280] "Artificial intelligence" refers to a system that uses machine learning and data analysis to predict and navigate user operations and tasks.

[1281] "Navigating" refers to assisting and guiding users in their operations and tasks.

[1282] "Emotional state" refers to a user's psychological state, such as stress level or concentration.

[1283] "Analysis" refers to the process of processing collected data and extracting meaningful information.

[1284] A "clustering algorithm" refers to a method for grouping data and grouping together data that has the same patterns.

[1285] "Report" refers to documents or data summarizing the results of analysis.

[1286] MODE FOR CARRYING OUT THE INVENTION

[1287] The present invention provides a system that collects and analyzes a user's operation log of an information processing device to identify the user's operation pattern and emotional state and provide appropriate navigation. Specific embodiments of this system are described below.

[1288] 1. Program Generation

[1289] The program in this system is designed to collect and analyze user operation logs of information processing devices. Specifically, it includes an algorithm for mining tasks from the operation logs and an emotion engine for estimating the user's emotional state.

[1290] 2. Program Processing

[1291] The server collects the user's operation log of the information processing device. The collected data includes keyboard typing speed, mouse movement, and the amount of time the information processing device is used. This data is analyzed on the server.

[1292] For the analysis, we use Python's Pandas library to preprocess the data and then use Scikit-learn to build a machine learning model. Specifically, we use a clustering algorithm (e.g., K-means) to find the same operation patterns from the operation logs.

[1293] For example, if a specific operation is performed repeatedly in a certain order, the combination of operations is recognized as a single task, which allows us to identify which tasks the user frequently performs.

[1294] The emotion engine estimates the user's emotional state, such as stress level and concentration level, based on the user's keyboard typing speed, mouse movements, and the amount of time the information processing device is used. For example, if the keyboard typing speed is fast and the mouse movements are large, it determines that the user may be feeling stressed. Also, if the information processing device is used for a long time, it determines that the user may be concentrating on their work.

[1295] 3. Examples and prompts

[1296] As a concrete example, let's say the following operation log is collected:

[1297] Keyboard typing speed: 120 keystrokes / min

[1298] Mouse movement: 500 pixels / minute

[1299] Time spent using information processing equipment: 4 hours

[1300] The server analyzes this data and estimates that the user is under high stress. If a specific operation pattern is repeated, the server recognizes the operation as a single task.

[1301] Example prompts to input to a generative AI model:

[1302] Analyze the user activity log data and provide the following information:

[1303] Keyboard typing speed

[1304] Mouse movement

[1305] Information processing device usage time

[1306] Use this data to estimate the user's stress level and concentration, and identify recurring patterns of operation and recognize them as a single task.

[1307] Using this prompt, the generative AI model can analyze the user's operation log data and provide the necessary information.

[1308] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1309] Step 1:

[1310] Users install and launch dedicated client software on their information processing devices. This software runs in the background and collects real-time operation logs of users' keyboard typing speed, mouse movements, and the amount of time they use the information processing device. The collected data is then sent to a server at regular intervals.

[1311] Input: User operation log data (keyboard typing speed, mouse movement, time spent using information processing devices)

[1312] Output: Operation log data sent to the server

[1313] Step 2:

[1314] The server receives the received operation log data and performs preprocessing on the data. Specifically, it cleans the data using the Python Pandas library, fills in missing values, removes outliers, and converts the data into a format that is easy to analyze.

[1315] Input: Received operation log data

[1316] Output: Preprocessed audit log data

[1317] Step 3:

[1318] The server uses the preprocessed data to analyze operation patterns. Specifically, it uses Scikit-learn's clustering algorithm (e.g., K-means) to group together operations with the same pattern. For example, if a specific operation is repeated in a certain order, the combination of operations is recognized as a single task.

[1319] Input: Preprocessed operation log data

[1320] Output: Identified operation patterns and tasks

[1321] Step 4:

[1322] The server estimates the user's emotional state from the operation log data. Specifically, it estimates the user's stress level and concentration level based on data such as keyboard typing speed, mouse movements, and the amount of time the information processing device is used. For example, if the keyboard typing speed is fast and the mouse movements are large, it is determined that the user may be feeling stressed. Also, if the information processing device is used for a long time, it is determined that the user may be concentrating on their work.

[1323] Input: Operation log data (keyboard typing speed, mouse movement, time spent using information processing devices)

[1324] Output: Estimated user's emotional state (stress level, concentration)

[1325] Step 5:

[1326] The server provides the analysis results to the user. Specifically, it generates a report on the user's operation patterns and emotional state and sends it to the user's device. The user can use this report to improve their work efficiency and stress management.

[1327] Input: Identified operation patterns and tasks, estimated user emotional state

[1328] Output: Report provided to the user (operation patterns, emotional state)

[1329] (Application example 2)

[1330] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1331] While conventional systems were able to collect user operation logs and mine tasks, they were unable to analyze robot operation logs to propose efficient work procedures or estimate the need for maintenance from the robot's operating speed or operating time, making it difficult to efficiently operate and optimize maintenance of robots in factories and other workplaces.

[1332] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1333] In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating artificial intelligence based on the mined tasks, means for navigating user operations and tasks using the generated artificial intelligence, means for collecting and analyzing robot operation logs, means for finding similar operation patterns and proposing efficient work procedures, and means for estimating the need for maintenance from the robot's operating speed and operating time. This makes it possible to analyze both the user operation logs and the robot operation logs, propose efficient work procedures, and optimize maintenance.

[1334] A "user operation log" is a series of records generated when a user operates a computer or device.

[1335] "Task mining" is a method of analyzing collected operation logs, finding the same operation patterns, and recognizing them as a single task.

[1336] "Artificial intelligence" is a computer program that learns from collected data and makes predictions and judgments.

[1337] "Means of navigation" refers to a method of instructing users or robots on efficient operations and work procedures.

[1338] A "robot operation log" is a series of records generated when a robot performs a task.

[1339] "Means of analysis" are ways of finding patterns and trends based on collected data.

[1340] "Means for proposing efficient work procedures" refers to a method for finding and proposing optimal work procedures based on the analysis results.

[1341] "Operation speed" refers to the speed at which the robot performs a task.

[1342] "Operating time" is the total amount of time that the robot is actually performing work.

[1343] "Means for estimating the need for maintenance" refers to a method for determining whether maintenance is necessary based on data such as the robot's operating speed and operating time.

[1344] A system for carrying out the present invention includes a program for collecting and analyzing user operation logs and robot operation logs. Specific embodiments of the system will be described below.

[1345] Hardware and software used

[1346] Hardware: computers, robots, sensors for collecting log data, servers for data analysis

[1347] Software: Python, Pandas (library for data analysis), Scikit-learn (library for machine learning)

[1348] Data processing and calculation

[1349] 1. Collecting user operation logs

[1350] The server collects a series of records generated when users interact with their computers or devices, including keyboard typing speed, mouse movements, and usage time.

[1351] 2. Collecting robot operation logs

[1352] The server collects a series of records generated by the robot as it performs its tasks, including its speed, duration, and type of operation.

[1353] 3. Analysis of operation logs

[1354] The server analyzes the collected operation logs and finds the same operation patterns, which makes it possible to propose efficient work procedures. Specifically, it clusters the operation logs using KMeans clustering to find patterns.

[1355] 4. Estimating maintenance needs

[1356] The server estimates the need for maintenance based on the robot's movement speed and operating time. For example, if the robot's movement speed drops or its operating time exceeds a certain threshold, the server determines that maintenance is necessary.

[1357] Specific examples

[1358] For example, if a robot used in a factory repeats a specific operation, the server will detect that operation pattern and suggest an efficient work procedure. If the robot's operating speed slows down, the server will notify the robot that maintenance is required.

[1359] Prompt Sentence Examples

[1360] "Please develop an application that analyzes the operation logs of factory robots and proposes efficient work procedures. Also, please add a function that estimates the need for maintenance based on the robot's operating speed and operating time."

[1361] In this way, both the user's operation log and the robot's operation log can be analyzed, making it possible to propose efficient work procedures and optimize maintenance.

[1362] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1363] Step 1:

[1364] The server collects user operation logs. Specifically, it collects data such as keyboard typing speed, mouse movement, and usage time generated when a user operates a computer or device. The input is the user operation data, and the output is the collected operation log.

[1365] Step 2:

[1366] The server collects the robot's operation logs. Specifically, it collects data such as the robot's operating speed, operating time, and type of operation generated when the robot performs a task. The input is the robot's operation data, and the output is the collected operation log.

[1367] Step 3:

[1368] The server analyzes the collected user operation logs. Specifically, it clusters the data using KMeans clustering to find the same operation patterns from the operation logs. The input is the collected operation logs, and the output is the clustered operation patterns.

[1369] Step 4:

[1370] The server analyzes the collected robot operation logs. Specifically, it clusters the data using KMeans clustering to find the same operation patterns from the operation logs. The input is the collected operation logs, and the output is the clustered operation patterns.

[1371] Step 5:

[1372] The server proposes efficient work procedures based on the analysis results. Specifically, it finds the optimal work procedure based on the clustered operation patterns and proposes it to the user or robot. The input is the clustered operation patterns, and the output is the proposed work procedure.

[1373] Step 6:

[1374] The server estimates the need for maintenance based on the robot's operating speed and operating time. Specifically, it determines that maintenance is necessary if the operating speed drops or if the operating time exceeds a certain threshold. The input is the robot's operating speed and operating time, and the output is a notification regarding the need for maintenance.

[1375] Step 7:

[1376] The server notifies users and robots of efficient work procedures and the need for maintenance. Specifically, it provides information on proposed work procedures and maintenance needs to users and robots. The input is the notification of the proposed work procedures and the need for maintenance, and the output is the notification to the user or robot.

[1377] Example 3

[1378] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1379] Conventional systems only collected user operation logs, but lacked the means to effectively utilize that data. Furthermore, task navigation based on the user's emotional state was not provided, resulting in insufficient work efficiency and stress management for users. Furthermore, the system lacked the functionality to predict and present the next operation to the user, which often made user operations cumbersome.

[1380] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1381] In this invention, the server includes a means for collecting user operation logs, a means for storing the collected operation logs in a database, a means for training an AI model using the stored operation logs, a means for predicting the next operation when a new operation log is input, a means for presenting the prediction result to the user, and a means for adjusting task navigation based on feedback from the emotion engine. This makes it possible to effectively utilize the user's operation logs to predict and present the next operation to be performed. Furthermore, by providing task navigation according to the user's emotional state, it is possible to improve the user's work efficiency and manage stress.

[1382] A "user operation log" is data that records a series of operations performed by a user in chronological order.

[1383] A "database" is a system for storing and managing collected operation logs.

[1384] An "AI model" is a model trained using a machine learning algorithm that learns a user's operation patterns and predicts the next operation.

[1385] "Training" is the process of teaching an AI model using collected operation logs.

[1386] "Prediction" means that the AI ​​model estimates the most likely next operation based on new operation logs.

[1387] The "emotion engine" is a system that analyzes the user's emotional state from their facial expressions and voice and provides feedback based on the results.

[1388] "Task navigation" refers to guiding users to the next action or task they should perform.

[1389] "Feedback" refers to providing information about the user's emotional state obtained from the emotion engine to the AI ​​model.

[1390] This invention is a system that collects user operation logs, uses the data to train an AI model, and predicts the next action to be taken. It also has the ability to adjust task navigation based on the user's emotional state.

[1391] The server uses dedicated software to collect user operation logs. This software monitors the series of operations performed by the user in real time and records them as an operation log. The operation log includes information such as the buttons the user clicked, the text they entered, and the applications they opened.

[1392] The collected operation logs are stored in a database, using a relational database such as MySQL or PostgreSQL. The operation logs are stored in a format that includes information such as the user ID, operation details, and timestamp.

[1393] The server uses the saved operation logs to train the AI ​​model. For training, it uses machine learning frameworks such as TensorFlow and PyTorch. Specifically, the operation logs are used as input data and the next operation is used as output data to train the model.

[1394] When a user performs a new operation, the operation log is sent to the server in real time. The server inputs this new operation log into the AI ​​model and predicts the next operation that is likely to be performed. The prediction result is output as the type of operation and its probability.

[1395] The device presents the prediction results received from the server to the user. Specifically, it displays the predicted operation on the screen and allows the user to select and execute the operation. For example, it displays a message such as "Do you want to save the data next?"

[1396] Furthermore, the server receives feedback from the emotion engine and adjusts task navigation based on the user's emotional state. The emotion engine analyzes the user's emotional state from their facial expressions and voice and sends the results to the server. For example, if the server determines that the user is feeling stressed, it will send a message to the device encouraging them to take a break.

[1397] As a concrete example, consider the case where a user is entering data into Excel. Based on past operation logs, the AI ​​model predicts that the next most likely operation will be "saving data" or "creating a graph." These operations are displayed on the user's screen, and the user can select and execute them.

[1398] An example of a prompt sentence might be:

[1399] "When a user is entering data in Excel, predict what they're likely to do next."

[1400] Using this prompt sentence, the AI ​​model predicts the next operation to be performed and presents it to the user. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1401] Step 1:

[1402] The server collects user operation logs. Specifically, it uses dedicated software to monitor the series of operations performed by the user in real time and records them as an operation log. Input includes information such as the buttons the user clicked, the text they entered, and the applications they opened. The output is a log of these operations.

[1403] Step 2:

[1404] The server saves the collected operation logs in a database. A relational database such as MySQL or PostgreSQL is used as the database. The input includes the operation logs collected in step 1. The output is the operation logs saved in the database. Specifically, they are saved in a format that includes information such as the user ID, operation details, and timestamp.

[1405] Step 3:

[1406] The server uses the stored operation logs to train the AI ​​model. Machine learning frameworks such as TensorFlow and PyTorch are used for training. The input includes the operation logs stored in the database. The output is a trained AI model. Specifically, the model learns using the operation logs as input data and the next operation as output data.

[1407] Step 4:

[1408] When a user performs a new operation, the operation log is sent to the server in real time. The input contains the new operation log performed by the user. The output is sent to the server. Specifically, it contains information such as the button the user clicked and the text they entered.

[1409] Step 5:

[1410] The server uses the AI ​​model to predict the next operation based on the new operation log. The input includes the new operation log sent in step 4. The output is a prediction of the most likely next operation. Specifically, the type of operation and its probability are output.

[1411] Step 6:

[1412] The device presents the prediction results received from the server to the user. The input includes the prediction results generated in step 5. The output displays the predicted operation on the screen. Specifically, a message such as "Do you want to save the data next?" is displayed, allowing the user to select and execute the operation.

[1413] Step 7:

[1414] The server receives feedback from the emotion engine and adjusts the task navigation based on the user's emotional state. The input includes feedback information from the emotion engine. The output generates an adjusted task navigation. Specifically, if the server determines that the user is feeling stressed, it sends a message to the device encouraging the user to take a break.

[1415] (Application example 3)

[1416] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1417] Conventional factory robots were unable to predict the next task based on operation logs, making efficient work navigation difficult. Furthermore, they lacked the ability to determine the user's emotional state based on data obtained from the robot's sensors and prioritize appropriate tasks, potentially reducing work efficiency and safety.

[1418] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating AI based on the mined tasks, means for navigating user operations and tasks using the generated AI, means for determining the user's emotional state based on data obtained from the robot's sensors, and means for preferentially navigating tasks according to the emotional state. This makes it possible to predict the next task to be performed based on the operation logs, enabling efficient and safe task navigation.

[1419] An "operation log" is data that records the history of operations performed by users and robots.

[1420] A "task" is a series of operations or tasks performed to achieve a specific purpose.

[1421] "Mining" is the process of extracting useful information and patterns from large amounts of data.

[1422] "AI" is an abbreviation for artificial intelligence, a technology that allows computers to imitate human intelligence to learn and reason.

[1423] "Navigating" means instructing a user or robot on the next operation or task to perform.

[1424] A "sensor" is a device that detects physical environmental information and outputs it as data.

[1425] "Emotional state" refers to the current psychological and physiological state of the user or robot.

[1426] "Priority" is a criterion for determining which of multiple tasks or operations should be performed first.

[1427] A system for implementing this invention collects user operation logs, mines tasks from the collected operation logs, and generates AI based on the mined tasks. The generated AI navigates the user's operations and tasks, determines the user's emotional state based on data obtained from the robot's sensors, and prioritizes navigating tasks according to the user's emotional state.

[1428] Hardware and software used

[1429] Hardware: Factory robots, sensors (temperature, humidity, vibration, etc.)

[1430] Software: TensorFlow (AI model design and training), Python (data preprocessing and program execution)

[1431] Data processing and calculation

[1432] The server first collects user operation logs. These logs are data that record the history of operations performed by users and robots, and tasks are mined from them. AI is then generated based on the mined tasks, and the generated AI navigates the user's operations and tasks.

[1433] Furthermore, the robot determines the user's emotional state based on data obtained from its sensors. Emotional state refers to the current psychological and physiological state of the user or robot, and prioritizes task navigation accordingly. For example, if it determines that the user is feeling stressed, it will display a message encouraging them to take a break. Similarly, if it determines that the user is concentrating on a task, it will prioritize tasks related to that task.

[1434] Specific examples

[1435] For example, imagine a robot assembling parts in a factory. In this case, the system uses operation log data to predict the next task to be performed and issues instructions to the robot. If the robot's sensor detects high temperatures, the emotion engine determines that the robot is feeling "stressed" and displays a message encouraging the robot to take a break.

[1436] Prompt Sentence Examples

[1437] When a factory robot is assembling parts, use the operation log data to predict the next task to be performed and issue instructions to the robot. Also, if the robot's sensor detects a high temperature, the emotion engine will determine that the robot is feeling "stressed" and display a message encouraging it to take a break.

[1438] In this way, the next task to be performed can be predicted based on the operation log, enabling efficient and safe task navigation.

[1439] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1440] Step 1:

[1441] The server collects user operation logs. Operation logs are data that records the history of operations performed by users and robots, and subsequent processing is based on this data. The input is the operation data of users and robots, and the output is the collected operation logs.

[1442] Step 2:

[1443] The server mines tasks from the collected operation logs. A task is a series of operations or tasks performed to achieve a specific purpose. The input is the collected operation logs, and the output is the mined tasks. Specifically, the server analyzes the operation log data and extracts frequently occurring operation patterns.

[1444] Step 3:

[1445] The server generates AI based on the mined tasks. AI stands for artificial intelligence, a technology in which computers mimic human intelligence to learn and infer. The input is the mined tasks, and the output is the generated AI model. Specifically, the server designs an AI model to learn task patterns and trains the model using learning data.

[1446] Step 4:

[1447] The server uses the generated AI to navigate the user's operations and tasks. Navigation means instructing the user or robot on the next operation or task to be performed. The input is the current operation log and the generated AI model, and the output is instructions on the next operation or task to be performed. Specifically, the current operation log is input into the AI ​​model, and the next operation to be performed is predicted.

[1448] Step 5:

[1449] The server determines the user's emotional state based on data obtained from the robot's sensors. The emotional state refers to the current psychological and physiological state of the user or robot. The input is data from the sensors, and the output is the determined emotional state. Specifically, the server analyzes the sensor data and estimates the emotional state, such as stress and concentration.

[1450] Step 6:

[1451] The server prioritizes navigation to tasks according to the user's emotional state. The input is the determined emotional state and the current task, and the output is the task to be prioritized. Specific operations include reevaluating task priorities based on the user's emotional state and selecting and navigating to an appropriate task. For example, if the server determines that the user is feeling stressed, it displays a message encouraging the user to take a break. Furthermore, if the server determines that the user is concentrating on a task, it prioritizes navigation to tasks related to that task.

[1452] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1453] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1454] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are mentioned.

[1455] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1456] [Third embodiment]

[1457] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1458] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1459] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1460] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1461] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1462] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1463] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1464] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1465] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1466] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1467] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1468] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1469] "Example 1"

[1470] In one embodiment of the present invention, software is installed on the user's PC to collect the user's PC operation log, and the software records the user's operations in real time. This software collects information such as the type of operations performed by the user, the order of operations, and the time taken for each operation.

[1471] "Example 2"

[1472] As a means of mining tasks from collected PC operation logs, an algorithm is used to analyze the collected operation logs and find patterns of the same operations. For example, if a specific operation is repeated in a certain order, the combination of operations is recognized as a single task. "Example 3"

[1473] As a means of generating AI based on the mined tasks, we design an AI model that learns the mined task patterns. When a new operation log is input, this AI model predicts the next operation that is likely to be performed based on the task patterns it has learned in the past.

[1474] "Example 4"

[1475] As a means for the generated AI to navigate the user's PC operations and tasks, an interface is provided that presents the user with the operations predicted by the AI ​​model. For example, when the user starts a specific operation, the next operation predicted by the AI ​​model is displayed on the screen, allowing the user to select and execute that operation.

[1476] The processing flow of each embodiment will be described below.

[1477] "Example 1"

[1478] Step 1: Install software that runs on the user's PC and records user actions in real time.

[1479] Step 2: When a user operates the PC, the software records the operation. The recorded information includes the type of operation, the order of operations, and the time taken for the operation. Step 3: The recorded operation log is periodically collected and sent to the server.

[1480] "Example 2"

[1481] Step 1: The operation log sent to the server is analyzed by the task mining algorithm.

[1482] Step 2: The algorithm finds patterns of the same operations in the operation log. For example, if a specific operation is repeated in a certain order, it recognizes the combination of operations as a single task.

[1483] Step 3: The recognized tasks are stored in a database.

[1484] "Example 3"

[1485] Step 1: Use the tasks stored in the database as training data for the AI ​​model.

[1486] Step 2: The AI ​​model learns patterns for the task based on the training data.

[1487] Step 3: When a new operation log is input, the AI ​​model predicts the next likely operation based on the task patterns it has learned in the past.

[1488] "Example 4"

[1489] Step 1: Create an interface that presents the user with the operations predicted by the AI ​​model.

[1490] Step 2: When the user initiates a specific action, the AI ​​model displays the next action predicted by the user on the screen.

[1491] Step 3: The user can select from the displayed actions and perform the action.

[1492] Example 1

[1493] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1494] Conventional user operation log collection systems do not collect operation logs in real time, making it difficult to accurately grasp user operation patterns. Furthermore, analysis of collected data is often done manually, requiring efficient analysis. Furthermore, report generation based on analysis results is not automated, making it impossible to quickly provide useful information to users.

[1495] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1496] In this invention, the server includes means for installing software on the user's information processing device, means for the installed software to record user operations in real time, means for temporarily storing the recorded data in the information processing device, means for transmitting data to the server at regular time intervals, means for the server to analyze the received data, and means for the server to generate a report based on the analysis results. This makes it possible to collect user operation logs in real time, analyze them efficiently, and quickly provide useful reports.

[1497] "Information processing device" refers to devices such as computers and PCs operated by users.

[1498] "Software" refers to a program that is installed on an information processing device and records user operations.

[1499] "Real-time" means that the data is recorded immediately at the moment the user performs an action.

[1500] "Recorded data" refers to information collected as a user's operation log.

[1501] "Temporarily storing" refers to temporarily storing the recorded data in a storage device within the information processing device.

[1502] "Regular time intervals" means that the data is sent to the server periodically, for example every five minutes.

[1503] "Server" refers to a central computer system that analyzes the data it receives and generates a report of the analysis results.

[1504] "Analyzing" refers to processing collected data to identify user patterns and trends.

[1505] "Report" refers to documents or data generated based on analysis results that show user operation patterns and trends.

[1506] MODE FOR CARRYING OUT THE INVENTION

[1507] This invention is a system that installs software on a user's information processing device, records the user's operations in real time, temporarily stores the recorded data in the information processing device, sends the data to a server at regular intervals, analyzes the data received by the server, and generates a report of the analysis results.

[1508] 1. Software Installation

[1509] Users use the provided installer to install the software on their information processing devices. This software is compatible with operating systems such as Windows and macOS.

[1510] 2. Real-time recording of operations

[1511] The software installed on the device runs in the background and records user actions in real time, specifically:

[1512] Keyboard input (type and order of keys)

[1513] Mouse operations (click, drag, location information)

[1514] A timestamp for each operation

[1515] 3. Temporary storage of data

[1516] The software temporarily saves the recorded operation data to the device's local storage in a general-purpose data format such as JSON or CSV.

[1517] 4. Data transmission

[1518] The device sends the recorded data to the server at regular intervals (for example, every 5 minutes). Data transmission is secure using the HTTPS protocol.

[1519] 5. Data Analysis

[1520] The server analyzes the received data using Python's Pandas library and machine learning models (e.g., scikit-learn and TensorFlow). The goal of the analysis is to identify user behavior patterns.

[1521] 6. Generate reports

[1522] The server generates reports based on the analysis results, including information on user operation patterns, frequency, and the time it takes to perform specific operations, and the reports are displayed on a dashboard accessible to the user.

[1523] Specific examples

[1524] For example, if a user types a sentence in a text editor and then performs a search in a web browser, the specific behavior is as follows:

[1525] 09:00:01 - Start typing on the keyboard (text editor)

[1526] 09:00:15 - End of keyboard input

[1527] 09:00:20 - Mouse click (web browser icon)

[1528] 09:00:25 - Start typing on keyboard (search bar)

[1529] 09:00:30 - End of keyboard input

[1530] The recorded data is saved in local storage as operation log_20231010.json and sent to the server at 09:05:00. The server analyzes the received data and identifies patterns, such as "users tend to type text in a text editor at 9:00 AM and then perform searches in a web browser." A report is generated based on the analysis results and displayed as "Operation Patterns at 9:00 AM" on a dashboard accessible to users.

[1531] Prompt Sentence Examples

[1532] Develop software that collects users' PC operation logs and records the type, sequence, and time of operations. Include a function to send the collected data to a server and generate reports that analyze user operation patterns. Include a scenario in which input in a text editor and searches in a web browser are recorded as specific operation examples.

[1533] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1534] Step 1:

[1535] The user installs the software on their information processing device using the provided installer, and once the installation is complete, the software starts running in the background.

[1536] Input: Installer file

[1537] Output: Installed software

[1538] Step 2:

[1539] The software installed on the device records user actions in real time, including keyboard input, mouse movements, and a timestamp for each action.

[1540] Input: User actions (keyboard input, mouse operations)

[1541] Output: Operation log data (key type and sequence, click position, timestamp)

[1542] Step 3:

[1543] The software temporarily saves the recorded operation data to the device's local storage in a general-purpose data format such as JSON or CSV.

[1544] Input: Operation log data

[1545] Output: Data file saved in local storage (e.g., Operation Log_20231010.json)

[1546] Step 4:

[1547] The device sends the recorded data to the server at regular intervals (for example, every 5 minutes). Data transmission is secure using the HTTPS protocol.

[1548] Input: Data files stored in local storage

[1549] Output: Data sent to the server

[1550] Step 5:

[1551] The server analyzes the received data using Python's Pandas library and machine learning models (e.g., scikit-learn and TensorFlow). The goal of the analysis is to identify user behavior patterns.

[1552] Input: Data sent to the server

[1553] Output: Analysis results (user operation patterns and trends)

[1554] Step 6:

[1555] The server generates reports based on the analysis results, including information on user operation patterns, frequency, and the time it takes to perform specific operations, and the reports are displayed on a dashboard accessible to the user.

[1556] Input: Analysis results

[1557] Output: Generated report (displayed on the dashboard)

[1558] Specific examples

[1559] For example, if a user types a sentence in a text editor and then performs a search in a web browser, the specific behavior is as follows:

[1560] 09:00:01 - Start typing on the keyboard (text editor)

[1561] 09:00:15 - End of keyboard input

[1562] 09:00:20 - Mouse click (web browser icon)

[1563] 09:00:25 - Start typing on keyboard (search bar)

[1564] 09:00:30 - End of keyboard input

[1565] The recorded data is saved in local storage as operation log_20231010.json and sent to the server at 09:05:00. The server analyzes the received data and identifies patterns, such as "users tend to type text in a text editor at 9:00 AM and then perform searches in a web browser." A report is generated based on the analysis results and displayed as "Operation Patterns at 9:00 AM" on a dashboard accessible to users.

[1566] (Application example 1)

[1567] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1568] Conventional information processing device operation log collection systems were able to collect user operation logs, mine tasks, and generate artificial intelligence, but they lacked the functionality to collect industrial machine operation logs and provide data to optimize operation efficiency. As a result, data collection and analysis to improve the operational efficiency of industrial machines were not being carried out sufficiently.

[1569] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1570] In this invention, the server includes means for collecting user operation logs of information processing devices, means for mining tasks from the collected operation logs of information processing devices, means for generating artificial intelligence based on the mined tasks, means for navigating user operations of the information processing devices and tasks using the generated artificial intelligence, means for collecting operation logs of industrial machines and recording the type, sequence, and time of operations, and means for analyzing the recorded operation logs and providing data for optimizing operation efficiency, thereby making it possible to collect operation logs of information processing devices and optimize the operation efficiency of industrial machines.

[1571] An "information processing device" is an electronic device such as a computer or personal computer that is operated by a user.

[1572] An "operation log" is a record of the type, order, time, etc. of operations performed by a user on an information processing device.

[1573] A "task" is a series of operations or tasks that a user performs on an information processing device.

[1574] "Mining" is the process of extracting useful information and patterns from collected data.

[1575] "Artificial intelligence" refers to programs and systems that learn from collected data and make predictions and navigate.

[1576] "Navigating" means guiding and assisting users with operations and tasks.

[1577] "Industrial machinery" refers to robots and mechanical devices used in factories and manufacturing sites.

[1578] "Type of operation" refers to the specific movement or action that an industrial machine performs.

[1579] "Sequence of operations" is the order in which a series of operations are performed by industrial machinery.

[1580] "Time of operation" is the time it takes an industrial machine to perform a particular operation.

[1581] "Analysis" means examining collected data in detail to find meaning and patterns.

[1582] "Optimization" means making adjustments or improvements to maximize the efficiency of an operation.

[1583] A system for carrying out the present invention collects user operation logs of information processing devices and analyzes the operation logs of industrial machines to optimize operational efficiency. A specific embodiment of the system is described below.

[1584] System Configuration

[1585] The system consists of the following main components:

[1586] 1. User's information processing device: Electronic equipment such as a computer or personal computer operated by a user.

[1587] 2. Industrial machinery: Robots and mechanical devices used in factories and manufacturing sites.

[1588] 3. Server: A central processing unit that collects data, analyzes it, and generates artificial intelligence.

[1589] Program processing

[1590] The server processes the data in the following steps:

[1591] 1. Collecting operation logs of information processing devices:

[1592] Software installed on the user's information processing device collects operation logs in real time, recording data such as the type, order, and time of operations.

[1593] 2. Mining tasks:

[1594] The collected operation logs are mined to identify the tasks performed by the user, thereby extracting the user's operation patterns and task flow.

[1595] 3. Artificial Intelligence Generation:

[1596] Based on the mined tasks, an artificial intelligence is generated, which has the ability to learn and predict user operation patterns.

[1597] 4. Collecting industrial machine operation logs:

[1598] Sensors and software installed on industrial machines collect operation logs, including the type, sequence and time of operations.

[1599] 5. Analysis of operation logs:

[1600] Analyzes collected industrial machine operation logs and provides data to optimize operational efficiency. This analysis includes evaluation of operation patterns and efficiency.

[1601] Hardware and software used

[1602] Hardware: Information processing equipment (computers, personal computers), industrial machinery (robots, machinery), servers

[1603] Software: Log collection software, data mining software, artificial intelligence generation software, analysis software

[1604] Specific names: Python, JSON library

[1605] Specific examples

[1606] For example, when a user creates and saves a document on an information processing device, the operation log is collected. Next, when an industrial machine moves forward and lifts a box, the operation log is also collected. These logs are sent to the server and analyzed.

[1607] Prompt Sentence Examples

[1608] "Write a Python program to collect operation logs from a factory robot and record the type, order, and time of operations. Operation types include move, pick up, etc. Save the logs in JSON format."

[1609] In this way, the collection of operation logs of the information processing device and the optimization of the operational efficiency of the industrial machine are realized.

[1610] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1611] Step 1:

[1612] Software installed on the user's information processing device collects operation logs in real time. The input is user operations (e.g., clicks, keyboard input), and the output is log data that includes the type, order, and time of the operations. This log data is saved in JSON format.

[1613] Step 2:

[1614] The server receives the collected operation logs and stores them in a database. The input is the log data sent from the information processing device, and the output is the operation logs stored in the database. The database records the type, order, and time of operations.

[1615] Step 3:

[1616] The server mines tasks from the saved operation logs. The input is the operation logs stored in the database, and the output is mined task data. The task data includes the user's operation patterns and task flow.

[1617] Step 4:

[1618] The server generates an artificial intelligence model based on the mined task data. The task data is used as input, and an artificial intelligence model that learns and predicts user operation patterns is generated as output. This artificial intelligence model is built using a generative AI model.

[1619] Step 5:

[1620] Sensors and software installed on industrial machines collect operation logs. The input is the operation of the industrial machine (e.g., moving, picking up), and the output is log data that includes the type, sequence, and time of the operation. This log data is also saved in JSON format.

[1621] Step 6:

[1622] The server receives collected operation logs of industrial machines and stores them in a database. The input is the log data sent from the industrial machines, and the output is the operation logs stored in the database. The database records the type, sequence, and time of operations.

[1623] Step 7:

[1624] The server analyzes the saved operation logs of industrial machines. The input is the operation logs stored in the database, and the output is data generated to optimize the efficiency of the operation. This analysis includes evaluation of operation patterns and efficiency.

[1625] Step 8:

[1626] The server provides feedback based on the analysis results to optimize the operational efficiency of the industrial machinery. The analysis results are input, and optimized operating procedures and improvement suggestions are generated as output. This feedback is sent to the control system of the industrial machinery.

[1627] Through the above steps, the collection of operation logs of information processing devices and the optimization of the operational efficiency of industrial machines are realized.

[1628] Example 2

[1629] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1630] Conventional systems for analyzing information processing device operation logs have had difficulty efficiently detecting user operation patterns and defining them as tasks. Furthermore, they were unable to navigate user operations and tasks using AI models generated based on the detected tasks. This resulted in the issue of being unable to improve user work efficiency.

[1631] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1632] In this invention, the server includes means for collecting information processing device operation logs of users, means for preprocessing the collected information processing device operation logs, means for detecting the same operation patterns from the preprocessed operation logs, means for defining and saving tasks based on the detected operation patterns, means for visualizing the defined tasks and generating reports, means for creating a generative AI model based on the generated reports, and means for navigating the user's information processing device operations and tasks using the generated AI model. This makes it possible to efficiently detect the user's operation patterns and define them as tasks, and further to navigate the user's operations and tasks using the generated AI model, thereby improving the user's work efficiency.

[1633] The "information processing device operation log" is a record of operations performed by a user on an information processing device.

[1634] "Preprocessing" is the process of cleaning and organizing collected data to convert it into a form suitable for analysis.

[1635] An "operation pattern" is a sequence or combination of a series of operations that a user repeatedly performs on an information processing device.

[1636] A "task" is a set of operations with a specific purpose that is defined based on the detected operation pattern.

[1637] "Visualization" is the presentation of data or information in a visual format such as a graph or chart.

[1638] A "report" is a document or data that summarizes analysis results and statistical information.

[1639] A "generative AI model" is an artificial intelligence model created using machine learning algorithms based on collected data.

[1640] "Navigating" means guiding and assisting users with operations and tasks.

[1641] This invention is a system that collects and analyzes user operation logs of information processing devices to detect user operation patterns and define them as tasks. Furthermore, it can use the generated AI model to navigate user operations and tasks.

[1642] The server installs log collection software on the terminal to collect the user's information processing device operation log. This software records user operations (clicks, keystrokes, application startup / exit, etc.) in real time and periodically sends them to the server.

[1643] The server preprocesses the collected operation logs. Specifically, it uses data analysis software such as Python or R to clean the log data (removing noise and filling in missing data) and convert it into a format suitable for analysis. For example, it organizes the order of operations based on timestamps and classifies them by type of operation.

[1644] The server then applies a pattern recognition algorithm to the preprocessed operation log. Specifically, it uses a machine learning algorithm (e.g., k-means clustering or sequence mining algorithm) to detect patterns in which the same operations are repeated in a certain order, thereby identifying combinations of operations that users frequently perform.

[1645] The server defines tasks based on the detected operation patterns and stores them in a database. Specifically, it assigns a task name to each pattern and records details of the related operations (type, order, frequency, etc.). For example, a task called "check email and open files every day" might be defined.

[1646] The server also visualizes the defined tasks and generates reports, displaying statistical information such as task execution frequency and time period in graphs and charts for easy understanding by users and administrators. The generated reports are available via a web interface and in PDF format.

[1647] Based on the generated reports, the server creates a generative AI model that can learn and predict user operation patterns. For example, if a user tends to perform a specific operation at a specific time, the model can predict the operation in advance and notify the user.

[1648] For example, if a user checks email at a specific time every day, then opens a specific file, and then launches a specific piece of software, the server will recognize these operations as a single task, which is defined as "checking email and opening a file every day."

[1649] Examples of prompts to input to a generative AI model might include:

[1650] "Analyze the patterns of operations that users perform every day and identify them as tasks. For example, if operations such as checking email, opening files, and launching software are repeated, define them as one task."

[1651] In this way, the server analyzes the collected operation logs, and realizes a system that recognizes and visualizes the user's repeated operations as tasks, thereby improving the user's work efficiency.

[1652] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1653] Step 1:

[1654] Collecting operation logs

[1655] The server collects the user's operation logs for the information processing device. The log collection software installed on the terminal records the user's operations (clicks, keystrokes, application startup / exit, etc.) in real time and periodically sends them to the server.

[1656] Input: User operation data

[1657] Output: Collected operation log data

[1658] Step 2:

[1659] Preprocessing of operation logs

[1660] The server preprocesses the collected operation logs. Specifically, it uses data analysis software such as Python or R to clean the log data (removing noise and filling in missing data) and convert it into a format suitable for analysis. For example, it organizes the order of operations based on timestamps and classifies them by type of operation.

[1661] Input: Collected operation log data

[1662] Output: Preprocessed audit log data

[1663] Step 3:

[1664] Applying pattern recognition algorithms

[1665] The server applies a pattern recognition algorithm to the preprocessed operation log. Specifically, it uses a machine learning algorithm (e.g., k-means clustering or sequence mining algorithm) to detect patterns in which the same operations are repeated in a certain order. This allows it to identify combinations of operations that users frequently perform.

[1666] Input: Preprocessed operation log data

[1667] Output: Detected operation pattern

[1668] Step 4:

[1669] Define and save the task

[1670] The server defines tasks based on the detected operation patterns and stores them in a database. Specifically, it assigns a task name to each pattern and records details of the related operations (type, order, frequency, etc.). For example, a task called "check email and open files every day" might be defined.

[1671] Input: Detected operation pattern

[1672] Output: Defined task data

[1673] Step 5:

[1674] Task visualization and reporting

[1675] The server visualizes the defined tasks and generates reports. Specifically, it displays statistical information such as task execution frequency and time period in graphs and charts, making it easy for users and administrators to understand. The generated reports are available via a web interface or in PDF format.

[1676] Input: Defined task data

[1677] Output: Visualized report

[1678] Step 6:

[1679] Creating generative AI models

[1680] The server creates a generative AI model based on the generated report. This AI model can learn and predict user operation patterns. For example, if a user tends to perform a specific operation at a specific time, it can predict that operation in advance and notify the user.

[1681] Input: Visualized report

[1682] Output: The generated AI model

[1683] Step 7:

[1684] Navigate user actions and tasks

[1685] The server uses the generated AI model to navigate the user's operations and tasks. Specifically, it predicts the operations and tasks the user needs to perform in advance and provides notifications and guidance at the appropriate time, thereby improving the user's work efficiency.

[1686] Input: Generated AI model

[1687] Output: The operation or task that was navigated to

[1688] (Application example 2)

[1689] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1690] In conventional systems, it was difficult to find efficient task patterns simply by collecting user operation logs. Furthermore, there was a lack of means to automatically analyze operation patterns and recognize them as optimal tasks, making it impossible to improve user work efficiency. Furthermore, automated systems such as factory robots are also required to analyze operation logs to find efficient tasks, but current technology is not sufficient to address this need.

[1691] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1692] In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating artificial intelligence based on the mined tasks, means for navigating user operations and tasks using the generated artificial intelligence, means for extracting features from the operation logs and finding operation patterns using a clustering algorithm, and means for extracting the found operation patterns as task patterns. This makes it possible to efficiently analyze user operation logs and automatically find optimal task patterns.

[1693] "User" means the human or end user who operates the system.

[1694] An "operation log" is a record of a series of operations performed by a user on a system.

[1695] A "task" is a set of operations performed to achieve a specific goal.

[1696] "Mining" is the process of extracting useful information and patterns from data.

[1697] "Artificial intelligence" is a technology that allows computers to learn and reason by imitating human intelligence.

[1698] "Navigating" means guiding a user through an operation or task.

[1699] A "feature" is an attribute or variable that is used to find specific patterns or trends in data.

[1700] A "clustering algorithm" is a computational method for dividing data into groups based on similarity.

[1701] An "operation pattern" is a tendency or regularity in the repetition of a series of operations performed by a user.

[1702] A "task pattern" is a set of operations required to achieve a specific goal.

[1703] The system for carrying out this invention collects and analyzes user operation logs to find and optimize efficient task patterns. A specific embodiment of this system is described below.

[1704] System Configuration

[1705] The system consists of the following main components:

[1706] 1. Operation log collection module: Collects user operation logs.

[1707] 2. Task mining module: Mines tasks from the collected operation logs.

[1708] 3. Artificial intelligence generation module: Generates artificial intelligence based on the mined tasks.

[1709] 4. Navigation module: The generated AI navigates the user through operations and tasks.

[1710] 5. Feature extraction module: Extracts features from the operation log.

[1711] 6. Clustering module: Uses a clustering algorithm to find operation patterns.

[1712] 7. Task pattern extraction module: Extracts the discovered operation patterns as task patterns.

[1713] Hardware and software used

[1714] Hardware: Factory robots, sensors for data collection

[1715] Software: Python, Pandas, Scikit-learn

[1716] Data processing and calculation

[1717] 1. Operation log collection: The operation log collection module records user operations in real time and stores them in a database.

[1718] 2. Feature Extraction: The feature extraction module extracts features such as operation type, duration, and parameters from the operation log.

[1719] 3. Clustering: The clustering module clusters the operation patterns using a clustering algorithm such as DBSCAN.

[1720] 4. Task Pattern Extraction: The task pattern extraction module extracts task patterns from the clustering results and recognizes them as efficient tasks.

[1721] 5. AI generation: The AI ​​generation module generates AI based on the extracted task patterns.

[1722] 6. Navigation: The navigation module uses the generated artificial intelligence to navigate the user through operations and tasks.

[1723] Specific examples

[1724] For example, in a part assembly operation in a factory, the operation log collection module collects the operation logs of a robot. The feature extraction module extracts the operation type, duration, and parameters from the operation logs, and the clustering module uses these features to cluster the operation patterns. The task pattern extraction module extracts efficient task patterns from the clustering results, and the artificial intelligence generation module generates artificial intelligence based on these. Finally, the navigation module uses the generated artificial intelligence to navigate the robot's operations and tasks.

[1725] Example prompt sentence:

[1726] Write a program to analyze the operation log data of a factory robot and find efficient task patterns. The operation log contains operation types, durations, and parameters. Use the DBSCAN clustering algorithm to cluster the operation patterns and extract the operations that belong to each cluster as task patterns.

[1727] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1728] Step 1:

[1729] Collecting operation logs

[1730] The server collects user operation logs in real time. Specifically, it records the operations performed by users on the system (clicks, keyboard input, scrolling, etc.) and stores them in a database. The input is the user operation data, and the output is the operation log stored in the database.

[1731] Step 2:

[1732] Feature extraction

[1733] The server extracts features from the collected operation logs. Specifically, it extracts features such as operation type, duration, and parameters for each entry in the operation log. The input is the operation log data, and the output is a dataset containing the features.

[1734] Step 3:

[1735] Clustering

[1736] The server clusters operation patterns using the features. Specifically, it uses a clustering algorithm such as DBSCAN to group similar operations. The input is a feature dataset, and the output is a dataset with cluster labels.

[1737] Step 4:

[1738] Task Pattern Extraction

[1739] The server extracts task patterns from the clustering results. Specifically, it analyzes the operations belonging to each cluster and recognizes them as efficient task patterns. The input is a dataset with cluster labels, and the output is task patterns.

[1740] Step 5:

[1741] Artificial Intelligence Generation

[1742] The server generates artificial intelligence based on the extracted task patterns. Specifically, it uses the task patterns as learning data to train a generative AI model. The input is the task patterns, and the output is the trained artificial intelligence model.

[1743] Step 6:

[1744] Navigation

[1745] The server uses the generated AI to navigate the user's operations and tasks. Specifically, it monitors the user's current operation status and guides the user to the next operation based on the optimal task pattern. The input is the user's current operation data and the AI ​​model, and the output is navigation instructions.

[1746] Example 3

[1747] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1748] In conventional systems, simply collecting user operation logs made it difficult to predict subsequent operations and provide appropriate navigation to users. Furthermore, the AI ​​model for predicting the next operation a user should perform was insufficiently trained, making it impossible to improve user operation efficiency. Furthermore, the system lacked the ability to determine the user's task priorities and navigate accordingly, resulting in a decrease in user work efficiency.

[1749] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1750] In this invention, the server includes means for collecting user operation logs, means for saving the collected operation logs, means for training an AI model using the saved operation logs, means for collecting new operation logs in real time, means for predicting the next operation, means for presenting the predicted results to the user, and means for the user to select and execute the presented predicted results. This makes it possible to predict the next operation with high accuracy based on the user's operation log and provide appropriate navigation to the user. Furthermore, by determining the priority of the user's tasks and navigating based on that, it is possible to significantly improve the user's work efficiency.

[1751] A "user operation log" is data that records a series of operations performed by a user in chronological order.

[1752] "Means of collection" refers to functions and devices for monitoring and recording user operation logs in real time.

[1753] The "storing means" refers to a function or device for storing the collected operation logs in a storage device such as a database.

[1754] An "AI model" is a model that uses machine learning algorithms to learn user operation patterns and predict the next operation.

[1755] "Training means" refers to functions or devices that train an AI model using saved operation logs.

[1756] "Means for collecting data in real time" refers to functions or devices that instantly record new operations performed by the user and send them to a server.

[1757] "Prediction means" refers to functions or devices that use an AI model to estimate the next operation that is likely to be performed.

[1758] "Presentation means" refers to a function or device for visually or audibly notifying the user of the prediction results.

[1759] The "means for selecting and executing" refers to a function or device that allows a user to select a presented prediction result and execute an operation based on that selection.

[1760] MODE FOR CARRYING OUT THE INVENTION

[1761] This invention is a system that collects user operation logs, predicts the next operation using an AI model, and presents it to the user. Specific embodiments of this system are described below.

[1762] System configuration

[1763] The system includes a terminal that collects user operation logs, a server that stores the collected operation logs and trains the AI ​​model, and an interface that presents prediction results to the user.

[1764] Hardware and software used

[1765] Hardware:

[1766] Device: PC or smartphone for recording user actions in real time

[1767] Server: A high-performance computer for storing operation logs and training AI models

[1768] software:

[1769] Operation log collection software: A program for recording user operations

[1770] Database: A database management system for storing operation logs (e.g., MySQL, PostgreSQL)

[1771] Machine learning framework: Software for training AI models (e.g., TensorFlow, PyTorch)

[1772] User Interface: GUI for presenting prediction results to users (e.g. HTML, CSS, JavaScript)

[1773] System Operation

[1774] 1. Collecting and saving operation logs:

[1775] The terminal monitors the series of operations performed by the user in real time and records them as an operation log, which is then sent to a server and stored in a database.

[1776] 2. Training the AI ​​model:

[1777] The server uses the saved operation logs to train an AI model, specifically learning operation patterns using machine learning frameworks such as TensorFlow and PyTorch.

[1778] 3. Real-time operation log collection and prediction:

[1779] When a user initiates a new operation, the device sends a log of that operation in real time to the server, which then inputs the received operation log into the AI ​​model to predict the most likely next operation.

[1780] 4. Prediction and execution:

[1781] The server returns the prediction results to the device, which then presents them to the user. When the user selects one of the prediction results, the device executes the corresponding operation.

[1782] Specific examples

[1783] For example, suppose a user begins to enter data into Excel. The device records this operation as an operation log and sends it to the server. The server saves the received operation log in a database and predicts the next operation using an AI model. Based on past operation patterns, the AI ​​model predicts that the user is likely to graph the data next. The server sends this prediction result back to the device, and the device displays a message to the user asking, "Do you want to graph the data next?" If the user accepts this suggestion, the device automatically starts creating a graph in Excel.

[1784] Prompt Sentence Examples

[1785] "When a user starts entering data into Excel, please design an AI model that predicts the next operation that the user is likely to perform and suggests it to the user." The flow of the specific processing in Example 3 will be described with reference to Figure 15.

[1786] Step 1:

[1787] The terminal collects user operation logs. Specifically, it monitors a series of user operations (e.g., data entry in Excel, web browser operations, etc.) in real time and records them as an operation log. The input is the user's operation, and the output is the operation log data.

[1788] Step 2:

[1789] The terminal sends the collected operation logs to the server. The server stores the received operation logs in a database. The input is the operation log data, and the output is the operation log stored in the database.

[1790] Step 3:

[1791] The server uses the stored operation logs to train an AI model. Specifically, it uses machine learning frameworks such as TensorFlow and PyTorch to learn operation patterns. The input is the operation logs stored in the database, and the output is the trained AI model.

[1792] Step 4:

[1793] When a user starts a new operation, the terminal sends the operation log to the server in real time. The input is the new operation log, and the output is the operation log sent to the server.

[1794] Step 5:

[1795] The server inputs the received operation log into the AI ​​model and predicts the next operation that is likely to be performed. Specifically, the AI ​​model makes predictions based on operation patterns it has learned in the past. The input is the new operation log and the trained AI model, and the output is the predicted next operation.

[1796] Step 6:

[1797] The server returns the prediction results to the terminal. The terminal presents the prediction results to the user. Specifically, it displays a message such as "Would you like to graph the data next?" on the screen. The input is the predicted next operation, and the output is the prediction result presented to the user.

[1798] Step 7:

[1799] When the user selects a predicted result, the terminal executes the operation. Specifically, when the user selects "Graph," the terminal automatically starts creating a graph in Excel. The input is the user's selection, and the output is the executed operation.

[1800] (Application example 3)

[1801] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1802] Conventional factory robots could collect operation logs but were unable to effectively utilize that data, making it difficult to predict the next operation. This resulted in problems such as reduced robot operation efficiency and difficulty in optimizing work. Furthermore, there were insufficient means for users to navigate through operations and tasks, which often led to operational errors and work delays.

[1803] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes means for collecting user operation logs, means for mining tasks from the collected operation logs, means for generating AI based on the mined tasks, means for navigating user operations and tasks using the generated AI, means for collecting robot operation logs and learning operation patterns, means for predicting the next operation to be performed when a new operation log is input, and means for displaying the predicted operation on the robot's interface so that the robot can select and execute the operation. This improves the operating efficiency of the robot and enables work optimization.

[1804] A "user operation log" is a record of a series of operations performed by a user.

[1805] "Task mining" means extracting specific tasks and procedures from collected operation logs.

[1806] "Generating AI" means building an artificial intelligence model based on collected data.

[1807] "Navigating user operations and tasks" means presenting the next operation or task predicted by the AI ​​to the user and assisting them with their work.

[1808] A "robot operation log" is a record of a series of operations performed by a robot.

[1809] "Learning operation patterns" means that the AI ​​learns common operating procedures and trends from collected operation logs.

[1810] "Predicting the next operation" means that when a new operation log is input, the AI ​​guesses what operation should be performed next.

[1811] A "robot interface" is a point of contact through which a robot exchanges information with users and other systems.

[1812] "Selecting and executing an operation" means that the robot selects a predicted operation and actually performs that operation.

[1813] A system for carrying out this invention includes means for collecting user operation logs, mining tasks from the collected operation logs, and generating AI based on the mined tasks. It also includes means for navigating user operations and tasks using the generated AI. It also includes means for collecting robot operation logs and learning operation patterns, means for predicting the next operation to be performed when a new operation log is input, and means for displaying the predicted operation on the robot's interface so that the robot can select and execute the operation.

[1814] Hardware and software used

[1815] Hardware: Factory robots (e.g., general-purpose robotic arms), high-performance GPUs (e.g., NVIDIA Tesla)

[1816] Software: TensorFlow (used to build and train AI models), Python (programming language)

[1817] Data processing and calculation

[1818] Data collection

[1819] The server collects user operation logs and robot operation logs, which include a series of operations performed by the user and a series of operations performed by the robot.

[1820] Task Mining

[1821] The server extracts specific tasks and procedures from the collected operation logs, which makes task patterns clear.

[1822] Generating AI models

[1823] The server generates an AI model based on the mined tasks, which is built using TensorFlow and learns patterns from the operation logs.

[1824] Operation Prediction

[1825] When a new operation log is input, the server uses the AI ​​model to predict the next operation to be performed, and the prediction results are displayed on the robot's interface.

[1826] Executing an operation

[1827] The robot selects the predicted operation from the interface and executes it, improving the robot's operational efficiency and optimizing the work.

[1828] Specific examples

[1829] For example, when a factory robot assembles parts, it can predict the next part or tool to be installed based on past operation logs and provide this information to the robot, improving work efficiency and reducing the occurrence of errors.

[1830] Prompt Sentence Examples

[1831] "Design an AI model that predicts the next operation that a factory robot should perform based on past operation logs when assembling parts, and presents this to the robot."

[1832] In this way, a smart navigation system can be built to optimize the operation of factory robots.

[1833] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1834] Step 1:

[1835] The server collects user operation logs and robot operation logs. The input includes a series of operations performed by the user and a series of operations performed by the robot. These operation logs are stored in a database. The output is the collected operation logs.

[1836] Step 2:

[1837] The server extracts specific tasks and procedures from the collected operation logs. The operation logs collected in step 1 are used as input. Data mining techniques are used to extract task patterns from the operation logs. The output is the mined tasks.

[1838] Step 3:

[1839] The server generates an AI model based on the mined tasks. The task patterns obtained in step 2 are used as input. TensorFlow is used to build an AI model that learns patterns in the operation logs. The generated AI model is obtained as output.

[1840] Step 4:

[1841] When a new operation log is input, the server uses the AI ​​model to predict the next operation to be performed. The new operation log and the AI ​​model generated in step 3 are used as input. The AI ​​model predicts the next operation based on the pattern in the operation log. The predicted operation is obtained as output.

[1842] Step 5:

[1843] The server displays the predicted operation on the robot's interface. As input, the predicted operation obtained in step 4 is used. The predicted operation is displayed on the robot's interface, allowing the robot to select and execute the operation. As output, the operation displayed on the robot's interface is obtained.

[1844] Step 6:

[1845] The robot selects a predicted operation from the interface and executes it. As input, it uses the operation displayed in step 5. The robot executes the selected operation and proceeds with its work. As output, it obtains the executed operation.

[1846] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1847] "Example 1"

[1848] In one embodiment of the present invention, an emotion engine is incorporated into a system that includes means for collecting a user's PC operation log, means for mining tasks from the collected PC operation log, means for generating an AI based on the mined tasks, and means for navigating the user's PC operations and tasks using the generated AI. This emotion engine analyzes the user's emotional state while using the PC in real time and feeds the results back to the AI.

[1849] "Example 2"

[1850] Specifically, the emotion engine estimates a user's emotional state, such as stress level and concentration level, based on their keyboard typing speed, mouse movements, and PC usage time. For example, if the keyboard typing speed is fast and the mouse movements are large, it determines that the user may be feeling stressed. Also, if the PC usage time is long, it determines that the user may be concentrating on their work.

[1851] "Example 3"

[1852] Based on feedback from the emotion engine, the AI ​​provides task navigation that adapts to the user's emotional state. For example, if the AI ​​determines that the user is feeling stressed, it will prioritize tasks to reduce stress (such as displaying a message encouraging them to take a break). If the AI ​​determines that the user is concentrating on a task, it will prioritize tasks related to that task.

[1853] The processing flow of each embodiment will be described below.

[1854] "Example 1"

[1855] Step 1: Collect user PC operation logs.

[1856] Step 2: Mine tasks from the collected PC operation logs.

[1857] Step 3: Generate AI based on the mined tasks.

[1858] Step 4: The generated AI guides the user through PC operations and tasks.

[1859] Step 5: Analyze the user's emotional state in real time while using the PC and provide feedback to the AI.

[1860] "Example 2"

[1861] Step 1: Estimate the user's emotional state, such as stress level and concentration level, based on the user's keyboard typing speed, mouse movements, and PC usage time.

[1862] Step 2: Feed back the estimated emotional state to the AI.

[1863] "Example 3"

[1864] Step 1: The AI ​​provides task navigation adapted to the user's emotional state based on feedback from the emotion engine.

[1865] Step 2: If the AI ​​determines that the user is stressed, it prioritizes tasks to reduce stress (for example, displaying a message encouraging the user to take a break).

[1866] Step 3: If the AI ​​determines that the user is concentrating on a task, it will prioritize navigating tasks related to that task.

[1867] Example 1

[1868] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1869] Conventional information processing device operation support systems were able to collect user operation logs, mine tasks, and generate AI, but they did not provide support that took into account the user's emotional state. As a result, there were operations that users found stressful or that they could not perform efficiently, and the user experience was not sufficiently improved. In addition, the system was limited in its ability to learn and predict user operation patterns and determine task priorities. This led to problems such as reduced user operation efficiency and satisfaction.

[1870] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1871] In this invention, the server includes means for collecting a user's information processing device operation log, means for mining tasks from the collected information processing device operation log, means for generating an artificial intelligence based on the mined tasks, means for navigating the user's information processing device operations and tasks using the generated artificial intelligence, means for analyzing the user's emotional state during operation in real time, and means for feeding back the analysis results to the artificial intelligence. This makes it possible to learn and predict the user's operation patterns and further enable support that takes the user's emotional state into consideration. This can improve the user's operation efficiency and satisfaction.

[1872] A "user" is an individual or group that operates an information processing device.

[1873] An "information processing device" is an electronic device such as a computer or tablet that is operated by a user.

[1874] An "operation log" is a record of operations performed by a user on an information processing device.

[1875] A "task" is a series of operations or tasks that a user performs on an information processing device.

[1876] "Mining" is the process of extracting useful information and patterns from collected operation logs.

[1877] "Artificial intelligence" refers to algorithms and models that learn user operation patterns and make predictions and provide assistance.

[1878] "Navigation" means providing instructions and suggestions to assist users in their operations and tasks, and to make them more efficient.

[1879] "Emotional state" refers to the psychological state, such as stress or joy, that a user feels while operating the device.

[1880] "Real-time analysis" is the process of instantly assessing a user's emotional state while they are interacting.

[1881] "Feedback" is the process of reflecting the results of analysis into artificial intelligence.

[1882] MODE FOR CARRYING OUT THE INVENTION

[1883] This invention is a system that collects user operation logs of information processing devices, mines tasks from the collected operation logs, generates artificial intelligence based on the mined tasks, and uses the generated artificial intelligence to navigate the user's operations and tasks. It also aims to improve user operation efficiency and satisfaction by analyzing the user's emotional state during operation in real time and feeding the results back to the artificial intelligence.

[1884] 1. Collection of user operation logs for information processing devices

[1885] Users install dedicated software on their information processing devices. This software runs in the background and records user operations in real time. Specifically, it collects information such as the type of operation (e.g., clicks, keyboard input, window switching, etc.), the order of operations, and the time taken for the operation. For example, if a user opens a browser, visits a specific website, and then sends an email, the entire series of operations is recorded in detail.

[1886] 2. Task Mining

[1887] The device periodically sends the recorded operation log to the server. The server receives it and stores it in a database. The server analyzes the received operation log and extracts tasks from the series of operations performed by the user. For example, the user's daily email check and reply operation is recognized as one task.

[1888] 3. Generating an AI model

[1889] The server uses the mined work data to generate an AI model that learns the user's operation patterns. This AI model predicts the user's operations and makes suggestions for improving efficiency. The server then distributes the generated AI model to the device. The device then uses this AI model to navigate the user's operations.

[1890] 4. Incorporating an Emotional Engine

[1891] An emotion engine built into the device analyzes the user's emotional state in real time while they are operating the device. For example, it can detect emotions such as stress or joy from the user's facial expressions and voice. The device then sends the analyzed emotional data to a server, which receives it and stores it in a database. The server then feeds the received emotional data back to the AI ​​model, allowing the AI ​​model to navigate operations while taking the user's emotional state into account. For example, if the user is feeling stressed, the AI ​​will suggest ways to simplify operations or...

Claims

1. A means for collecting user operation logs; A means for mining tasks from the collected operation logs; and A means for generating an artificial intelligence model based on the mined tasks; A means for analyzing the emotional state of a user during operation using an emotion engine and feeding back the analysis result to the artificial intelligence model; A means for proposing a work procedure based on the operation log and the emotional state; A system including:

2. The system of claim 1 , wherein the emotion engine analyzes an emotional state of the user based on the operation log.

3. The system according to claim 1 , wherein the means for suggesting a work procedure proposes the work procedure using a prompt sentence and a generative AI that instructs the user to suggest a simplified operation based on the operation log and the emotional state.

Images