System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting behavior data of a user; means for preprocessing the collected behavior data; generative model means for analyzing the preprocessed behavior data and learning behavior patterns; means for generating a manual based on the learned behavior patterns; and means for delivering the generated manual to a terminal of the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many workplaces today face the challenge of operators and employees being overwhelmed with their daily routine tasks and not having the time to create, maintain, or update new manuals. In these situations, many tasks are highly dependent on individual skills, making standardization of operations difficult. Furthermore, information for dealing with irregular business events is often not readily available, leading to inefficiencies. The purpose of this invention is to solve these problems and enable more efficient operations. [Means for solving the problem]

[0005] The present invention is a system including means for collecting user behavioral data, means for preprocessing the collected behavioral data, means for analyzing the preprocessed behavioral data and learning behavioral patterns, means for generating a manual based on the learned behavioral patterns, and means for distributing the generated manual to a user's terminal. This system also includes means for calculating the probability of irregular behavior based on the behavioral data and reflecting the probability in the manual, and means for updating the content of the manual in real time and notifying the user of the updates, thereby efficiently realizing business standardization and information organization.

[0006] "User behavior data" refers to records of operations and activities that occur when a user performs work.

[0007] "Means for collecting" refers to a method or apparatus for obtaining behavioral data from a user's terminal or other device.

[0008] "Preprocessing means" refers to the process of organizing and formatting the collected raw data to convert it into an analyzable format.

[0009] "Generative modeling tools" refer to algorithms and machine learning models that learn and analyze behavioral patterns based on collected and preprocessed data.

[0010] "Means for generating manuals" refers to methods and systems for creating documents that include business procedures and important points based on learned behavioral patterns.

[0011] "User's device" refers to an electronic device such as a computer, smartphone, or tablet that a user uses to perform their work.

[0012] "Means of distribution" refers to the communication means and software used to deliver the generated manual to the user's terminal.

[0013] "Probability of irregular behavior" refers to a numerical value that indicates the possibility of unusual behavior that differs from normal behavior patterns.

[0014] "Means for real-time updates" refers to a system for instantly changing and reflecting the contents of the manual in response to the results of behavioral data analysis and changes in the environment.

[0015] "Notification means" refers to a method or system for promptly notifying users of updates to the manual or important information. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] This invention is a system that collects user behavior data, analyzes behavioral patterns based on that data, and automatically generates and updates effective manuals, allowing operators and employees to focus on their daily work and improving the efficiency and standardization of work.

[0038] System configuration

[0039] This system is broadly composed of the following elements:

[0040] 1. Data Collection Methods

[0041] 2. Data preprocessing methods

[0042] 3. Learning methods for behavioral patterns (generative models)

[0043] 4. Manual Generation Methods

[0044] 5. Manual Distribution Methods

[0045] 6. Updates and Notifications

[0046] Specific processing flow

[0047] 1. Data Collection

[0048] When a user performs work, work data on the terminal is recorded in real time, such as what time the user logged in to the system and how long each application was used.

[0049] The device collects this behavioral data and sends it to a server at regular intervals.

[0050] 2. Data Preprocessing

[0051] The server analyzes the collected raw data and performs data cleansing, a process that removes incomplete data and outliers and converts the data into a consistent format.

[0052] For example, if the time record for an operation is missing, a reasonable estimate is made from the data before and after.

[0053] 3. Learning behavioral patterns

[0054] The generative model on the server receives the preprocessed data as input and analyzes and learns from the user's behavioral patterns using machine learning and statistical methods.

[0055] The probability of certain patterns and irregular behavior is calculated and accumulated.

[0056] 4. Manual Generation

[0057] Based on the learned behavioral patterns, the server automatically generates a manual containing specific work procedures and points to note. This manual details standard procedures that users should refer to and how to deal with irregular situations.

[0058] For example, if there is a pattern where data entry starts at 9:00 every day, email is checked at 10:00, and lunch is taken at 12:00, the corresponding procedures will be reflected in the manual.

[0059] 5. Manual Distribution

[0060] The server automatically distributes the generated manual to the user's terminal via the network, allowing the user to always have the latest manual at hand.

[0061] 6. Updates and Notifications

[0062] The server analyzes data in real time and immediately updates the manual if any changes occur in behavioral patterns. This update information is notified to the user, and the new manual is automatically distributed to the terminal.

[0063] For example, if system maintenance is performed on Thursdays only during a particular week, that information will also be reflected promptly.

[0064] Specific examples

[0065] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative model.

[0066] Based on the behavioral patterns learned by the generative model, the following manual is automatically generated:

[0067] 1. Data entry begins at 9:00 every day.

[0068] Required tools: Datasheet, input software

[0069] Note: Double-check to avoid input errors.

[0070] 2. Check email at 10:00.

[0071] Tools used: Email client

[0072] Important Note: Prioritize important emails

[0073] 3. System maintenance will be performed every Friday.

[0074] Tools used: Maintenance software

[0075] Important note: Make a backup of your entire system beforehand.

[0076] This manual is distributed to User A's terminal, and User A follows it to perform his / her work, resulting in efficient and standardized work. Furthermore, if there is a change in the actual behavioral pattern (e.g., system maintenance is changed to Thursday), an updated manual is immediately distributed from the server, allowing the user to follow the latest procedures.

[0077] The processing flow will be explained below.

[0078] Step 1:

[0079] The user begins their daily work, specifically using the device to perform tasks such as logging in, launching applications, data entry, checking email, and attending meetings.

[0080] Step 2:

[0081] The device records real-time data about user operations and activities, including application usage time, task start and end times, and data entered.

[0082] Step 3:

[0083] The device transmits the collected data to the server at regular intervals. This data transmission is performed periodically, and the behavioral data is efficiently aggregated on the server via a communication protocol.

[0084] Step 4:

[0085] The server preprocesses the received behavioral data, such as filling in missing data, removing noise, and normalizing the data, to make it ready for analysis.

[0086] Step 5:

[0087] The server feeds the preprocessed data into a generative model, which uses machine learning algorithms to learn the user's behavioral patterns.

[0088] Step 6:

[0089] The generative model analyzes user behavior patterns and calculates the probability of repeated tasks and irregular behavior, thereby providing a more accurate understanding of work flows.

[0090] Step 7:

[0091] Based on the learned behavioral patterns, the server generates a manual detailing work procedures and precautions, including commonly performed tasks and procedures, as well as how to deal with irregular behavior.

[0092] Step 8:

[0093] The server distributes the generated manual to the user's terminal, where the user can check the latest manual and perform their work accordingly.

[0094] Step 9:

[0095] The server analyzes the behavioral data in real time, and if any changes in behavioral patterns are detected, the manual is updated immediately. This update information is notified to the user, and the new manual is automatically distributed to the device.

[0096] Step 10:

[0097] Users receive notifications and follow the updated manuals to carry out their work. By repeating this process, work is continuously standardized and made more efficient.

[0098] Example 1

[0099] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0100] Conventional work support systems have difficulty in properly collecting and analyzing user behavior data, making it impossible to automatically generate work instructions optimized for individual users. As a result, improvements in work efficiency and standardization are not fully achieved, and there is a lack of appropriate countermeasures, particularly for irregular behavior. Furthermore, real-time work management is difficult because work instructions are not updated or notified immediately.

[0101] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0102] In this invention, the server includes means for collecting user activity data, means for preprocessing the collected activity data, means for analyzing the preprocessed activity data and learning behavioral patterns, means for generating work instructions based on the learned behavioral patterns, and means for distributing the generated work instructions to the user's device. This makes it possible to efficiently collect and analyze user activity data and automatically generate and distribute work instructions optimized for each user. Furthermore, real-time work management is realized by calculating the probability of atypical behavior and instantly updating and notifying work instructions.

[0103] "User" refers to an individual or organization that uses this system and is the entity that carries out specific tasks.

[0104] "Activity data" is a series of digital information generated when a user performs work, and includes login time, application usage history, operation details, and the like.

[0105] "Preprocessing" refers to a series of steps that analyze the collected raw data, remove incomplete data and outliers, and convert the data into a certain format.

[0106] A "generative model" refers to an algorithm or program that uses machine learning or statistical techniques to learn user behavior patterns from preprocessed data.

[0107] "Work instructions" refers to documents containing work procedures and points to note that are generated based on learned behavioral patterns, and serve as guidelines for users to carry out their daily work in an efficient and standardized manner.

[0108] "Distribution" refers to the process of sending the generated work instructions to the user's device so that the user can view them.

[0109] "Atypical behavior" refers to irregular operations or actions that deviate from normal behavior patterns, and is reflected in the manual by calculating the probability of such behavior.

[0110] "Immediate update" refers to the process of quickly updating the contents of work instructions based on data collected in real time, and notifying the user of the results immediately.

[0111] This invention is a system that collects user activity data, analyzes behavioral patterns based on that data, and automatically generates and updates work instructions, allowing operators and employees to focus on their daily work and improving work efficiency and standardization.

[0112] System configuration

[0113] The system consists of the following main components:

[0114] 1. Data Collection Methods

[0115] When a user starts work, they log in to the system, and their login time and operation details are collected.

[0116] The device collects a series of user operation data in real time and sends it to the server at regular intervals, for example, every minute.

[0117] 2. Data preprocessing methods

[0118] The server performs an initial analysis of the raw data received and performs data cleansing, a process that removes incomplete data and outliers and converts all data into a unified format.

[0119] For example, the timestamp format is standardized to Coordinated Universal Time (UTC) and missing parts of the log are reasonably estimated and supplemented.

[0120] 3. Behavioral pattern learning method (generative AI model)

[0121] A generative AI model hosted on a server receives preprocessed data as input and analyzes and learns from user behavior patterns using machine learning algorithms and statistical methods.

[0122] To learn behavioral patterns, the probability and typical patterns of these activities are stored in the model based on past data.

[0123] 4. Manual Generation Methods

[0124] Based on the learned behavioral patterns, the server automatically generates work instructions, which detail standard work procedures and how to handle irregular situations.

[0125] For example, a pattern such as "Start data entry at 9:00 every day and check email at 10:00" is clearly stated.

[0126] 5. Manual Distribution Methods

[0127] The server automatically delivers the generated work instructions to the user's device, allowing the user to always have the latest work instructions at hand.

[0128] The terminal displays the received work instructions in an appropriate format for easy reference by the user.

[0129] 6. Updates and Notifications

[0130] The server analyzes the data collected in real time and immediately updates the work instructions if there is a change in the behavioral pattern. This updated information is notified and new work instructions are automatically sent to the terminal.

[0131] For example, if system maintenance is performed only on Thursdays of certain weeks, that change will also be reflected immediately.

[0132] Specific examples

[0133] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative AI model.

[0134] Based on the behavioral patterns learned by the generative model, the following work instructions are automatically generated:

[0135] markdown

[0136] 1. Data entry begins at 9:00 every day.

[0137] Required tools: Datasheet, input software

[0138] Note: Double-check to avoid input errors.

[0139] 2. Check email at 10:00.

[0140] Tools used: Email client

[0141] Important Note: Prioritize important emails

[0142] 3. System maintenance will be performed every Friday.

[0143] Tools used: Maintenance software

[0144] Important note: Make a backup of your entire system beforehand.

[0145] This work instruction is delivered to User A's terminal, and User A follows it to perform the work, resulting in efficient and standardized work. In addition, if there is a change in the actual behavior pattern (for example, if system maintenance is changed to Thursday), the server immediately delivers updated work instructions, allowing the user to follow the latest procedures.

[0146] Prompt Sentence Examples

[0147] An example of a prompt to input to a generative AI model is as follows:

[0148] Please tell me the specific steps to take when entering data.

[0149] What should I pay attention to when checking my email?

[0150] "What is your standard procedure for system maintenance?"

[0151] keyword

[0152] Generative AI Models

[0153] Prompt statement

[0154] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0155] Step 1:

[0156] Data collection

[0157] When a user starts work, he logs in to the system, and the login time is recorded on the terminal.

[0158] For each application a user uses during work, its start and end times are collected in real time.

[0159] The terminal sends the collected data to the server in batch processing every minute, allowing operation logs on the terminal to be accumulated in real time.

[0160] Input: User operation data (e.g., login time, application usage time).

[0161] Output: The collected operation data is sent to the server.

[0162] Step 2:

[0163] Data Preprocessing

[0164] The server analyzes the raw data it receives and performs data cleansing, which includes removing incomplete data and outliers.

[0165] The server standardizes the timestamp format (e.g., converts it to Coordinated Universal Time (UTC)) and fills in missing data with estimated values.

[0166] Specific operation of data cleansing: If there is a missing login time, the average value is calculated from the data before and after the missing time to fill in the gaps.

[0167] Input: Collected operational data.

[0168] Output: The cleansed data.

[0169] Step 3:

[0170] Learning behavioral patterns

[0171] A generative AI model on a server receives the preprocessed data as input and begins analyzing the data using machine learning algorithms (e.g., recurrent neural networks).

[0172] The probability of certain patterns or abnormal behavior is calculated, and these models are gradually refined.

[0173] Specific operation: The generative AI model learns the average start time and frequency of a user's actions and stores the results in a database.

[0174] Input: Preprocessed data.

[0175] Output: Learned behavioral patterns (model).

[0176] Step 4:

[0177] Manual Generation

[0178] The server automatically generates work instructions based on the learned behavioral patterns, including routine work procedures and countermeasures for irregular situations.

[0179] Example: A standardized procedure such as "Start data entry at 9:00 and check email at 10:00 every day" is described.

[0180] Input: Learned behavioral patterns.

[0181] Output: The generated work order.

[0182] Step 5:

[0183] Manual Distribution

[0184] The server automatically delivers the generated work instructions to the user's device, allowing the user to always refer to the latest work instructions.

[0185] The terminal displays the received work instructions in an appropriate format for easy access by the user.

[0186] Notification function: Displays when a new manual has arrived on your device.

[0187] Input: Generated work order.

[0188] Output: Work instructions delivered to the terminal.

[0189] Step 6:

[0190] Updates and Notifications

[0191] The server analyzes the data collected in real time and immediately updates the work instructions if any changes occur in the behavioral patterns. The new work instructions are then delivered to the user's device along with the updated information.

[0192] Example: If the system maintenance schedule is changed, users will be notified immediately.

[0193] Input: Real-time behavioral data.

[0194] Output: Updated work order and notification.

[0195] (Application example 1)

[0196] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0197] There are already systems that collect user behavior data and analyze behavioral patterns based on that data to improve business efficiency and standardize operations. However, these systems lack real-time update capabilities or the ability to specifically control device operation, resulting in a lack of responsiveness and adaptability to business operations. Furthermore, they lack the ability to automatically generate manuals that reflect irregular behavior, limiting their ability to maximize user work efficiency. The purpose of this invention is to solve these problems and achieve higher levels of business efficiency and adaptability.

[0198] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0199] In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, means for analyzing the preprocessed behavioral data and learning behavioral patterns, means for generating a manual based on the learned behavioral patterns, means for distributing the generated manual to the user's device, means for controlling the operation of the device according to the set pattern, means for instantly updating the manual to the latest version based on real-time behavioral data, and means for notifying the device of the contents of the updated manual. This enables real-time business optimization and standardization based on the user's business data.

[0200] "Behavioral data" refers to the history and log information of specific operations and actions performed by a user.

[0201] "Preprocessing" is the process of converting raw data into a form suitable for analysis, including data cleansing and format conversion.

[0202] A "generative model" is a machine learning algorithm or statistical model that learns behavioral patterns based on preprocessed data.

[0203] A "manual" is a document or guideline that describes procedures and instructions for users to perform their work efficiently.

[0204] "Devices" refer to hardware devices such as terminals and robots used by users.

[0205] "Control" refers to the operation of directing the behavior or state of equipment, and is the process of managing behavior according to specific patterns.

[0206] "Real-time update" is the process of instantly updating the manual based on the latest behavioral data and delivering that information to the user's device.

[0207] "Irregular behavior" refers to unusual actions or operations that deviate from normal patterns of behavior, and procedures for dealing with such behavior are reflected in the manual.

[0208] This invention is a system that automatically generates and updates business manuals by collecting user behavior data and analyzing behavioral patterns based on the collected data. An embodiment of this system will be described in detail below.

[0209] System Configuration

[0210] The system mainly consists of the following components:

[0211] 1. How to collect user behavior data

[0212] 2. A means of preprocessing the collected behavioral data

[0213] 3. A generative modeling method that analyzes preprocessed behavioral data and learns behavioral patterns

[0214] 4. A method for generating manuals based on learned behavioral patterns

[0215] 5. Means for delivering the generated manual to the user's device

[0216] 6. Means of controlling the operation of equipment according to set patterns

[0217] 7. A way to instantly update manuals based on real-time behavioral data

[0218] 8. Means of notifying the device of updated manual contents

[0219] Operation overview

[0220] First, as users perform their work, sensors and cameras on devices (e.g., terminals and factory robots) collect behavioral data in real time. This data includes operating hours, tools used, movement patterns, and more. The collected data is sent to a server where preprocessing is performed. This involves filling in missing data and removing outliers. Next, a generative model learns behavioral patterns based on the preprocessed data. A machine learning algorithm is used for this purpose.

[0221] Based on the learned behavioral patterns, the system generates a work manual that includes specific operating procedures, tools to use, and important points to note. For example, the following manual may be generated:

[0222] 1. Starts operation at 8:00 every day.

[0223] Tools used: Gripper, drill.

[0224] Important note: Perform initial calibration of the gripper.

[0225] 2. Replace part A at 12:00.

[0226] Tools used: Torque wrench.

[0227] Note: Check the torque value.

[0228] 3. Full maintenance every Friday.

[0229] Tools used: Various tools.

[0230] Important note: Make a full system backup.

[0231] The generated manual is automatically distributed to the user's device and used to appropriately control the device's operation. Furthermore, the manual is instantly updated in response to changes in behavior patterns in real time. This update information is notified to the device, so the latest procedures are always reflected.

[0232] Hardware and software used

[0233] Collection devices: sensors, cameras, terminals, factory robots

[0234] Data preprocessing: Pandas (Python library)

[0235] Generative models: Random Forest (machine learning algorithm), other AI models

[0236] Data analysis server: High performance server

[0237] Distribution System: Network Communication

[0238] Prompt Sentence Examples

[0239] An example prompt based on a generative AI model is:

[0240] Based on the user's behavioral data, generate a robot operation manual like the one below.

[0241] 1. Starts operation at 8:00 every day.

[0242] Tools used: Gripper, drill.

[0243] Important note: Perform initial calibration of the gripper.

[0244] 2. Replace part A at 12:00.

[0245] Tools used: Torque wrench.

[0246] Note: Check the torque value.

[0247] 3. Full maintenance is performed automatically every Tuesday.

[0248] Tools used: Various tools.

[0249] Important note: Make a full system backup.

[0250] The above describes a specific embodiment of the invention. By using this system, it is possible to achieve greater efficiency and standardization of work, thereby maximizing the user's work performance.

[0251] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0252] Step 1:

[0253] The server collects user behavior data. This input data includes user uptime, tools and applications used, and movement patterns. This data is collected in real time from sensors, cameras, and devices.

[0254] Step 2:

[0255] The server preprocesses the collected behavioral data. This preprocessing includes filling in missing data, removing outliers, and converting the data into a format suitable for analysis. Raw data is used as input, and cleansed data is output after processing. Data processing libraries such as Pandas are used.

[0256] Step 3:

[0257] The server supplies the preprocessed data as input to the generative model. The generative model uses machine learning algorithms such as Random Forest to learn behavioral patterns. The input data is preprocessed behavioral data, and the output is learned behavioral patterns. The model runs on the server and performs analysis.

[0258] Step 4:

[0259] The server generates a manual based on the learned behavioral patterns. Specifically, a manual is automatically generated that describes the user's work procedures, tools to be used, and points to note. In this process, the learned behavioral patterns are used as input data, and the work manual is output.

[0260] Step 5:

[0261] The server distributes the generated manual to the user's device. The manual is automatically sent to the device via network communication. The input data is the generated manual, and the output is the state that has been distributed to the user's device.

[0262] Step 6:

[0263] The server controls the operation of the equipment according to the set patterns. Specifically, it manages the equipment so that it follows the manual and performs its tasks efficiently and accurately. The input data is the distributed manual, and the output is the actual operation control of the equipment.

[0264] Step 7:

[0265] The server instantly updates the manual to the latest version based on real-time behavioral data. It continuously collects user behavioral data and immediately updates the manual if there are any changes. The input data is the latest behavioral data, and the output is an updated manual.

[0266] Step 8:

[0267] The server notifies the user's device of the contents of the updated manual, allowing the user to always perform their work based on the latest procedures. The input data is the updated manual, and the output is a notification sent to the user's device.

[0268] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0269] This invention is a system that collects user behavior data, analyzes and learns behavioral patterns based on this data, and automatically generates and updates effective manuals. Furthermore, by combining it with an emotion engine that recognizes the user's emotional state, it realizes flexible responses based on the user's emotions. This system makes it possible to improve work efficiency and introduce human-centered design.

[0270] System configuration

[0271] The system consists of the following components:

[0272] 1. Data Collection Methods

[0273] 2. Data preprocessing methods

[0274] 3. Learning methods for behavioral patterns (generative models)

[0275] 4. Emotion Engine

[0276] 5. Manual Generation Methods

[0277] 6. Manual Distribution Methods

[0278] 7. Updates and Notifications

[0279] Specific processing flow

[0280] 1. Data Collection

[0281] Users begin their daily work and perform tasks on the device, such as logging in to the system, using applications, entering data, and checking email.

[0282] The device records the user's operation data in real time and periodically transmits the data to the server. In addition, the emotion engine recognizes the user's emotional state, which is also included in the behavioral data.

[0283] 2. Data Preprocessing

[0284] The server preprocesses the received data, filling in missing data, removing noise, and normalizing the data. The server also preprocesses the user's emotion data.

[0285] 3. Learning behavioral patterns and emotions

[0286] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns and emotional state. Specifically, it uses machine learning algorithms to analyze the correlation between user behavior and emotions.

[0287] For example, the model learns which specific tasks or times of day users find stressful.

[0288] 4. Manual Generation

[0289] Based on the learned behavioral patterns and emotional data, the server automatically generates a manual detailing work procedures and important points to note. This manual includes not only standard work procedures but also customizations based on the user's emotional state.

[0290] For example, if the user is feeling stressed, it may suggest taking a short break.

[0291] 5. Manual Distribution

[0292] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[0293] 6. Updates and Notifications

[0294] The server analyzes the data in real time and immediately updates the manual if any changes in behavioral patterns or emotional states are detected. This update information is notified to the user, and the new manual is automatically distributed to the device.

[0295] Specific examples

[0296] User B, who works for a certain company, records behavioral and emotional data on his device as he goes about his daily work. User B starts entering data at 9:00 and checks his email at 10:00 every day. However, the data shows that he begins to feel stressed around 14:00 every day. This data is sent to a server in real time and pre-processed on the server.

[0297] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[0298] 1. Data entry begins at 9:00 every day.

[0299] Required tools: Datasheet, input software

[0300] Note: Double-check to avoid input errors.

[0301] 2. Check email at 10:00.

[0302] Tools used: Email client

[0303] Important Note: Prioritize important emails

[0304] 3. Take a short break every day at 2:00 PM.

[0305] The break time is 15 minutes

[0306] Note: It is recommended to rest in a relaxing environment.

[0307] This manual is distributed to User B's device, and User B can follow it to perform his / her work efficiently. Taking breaks during times of stress in particular improves work efficiency and maintains health. The program automates the entire process, from data collection to manual generation and updating, reducing the user's workload.

[0308] This system allows users to perform tasks using procedures that are optimal for their emotional state, which not only improves work efficiency and standardizes work, but also contributes to health management.

[0309] The processing flow will be explained below.

[0310] Step 1:

[0311] A user begins their daily routine, for example, logging into a terminal and beginning a data entry task.

[0312] Step 2:

[0313] The device records user operation data in real time, including the start time of data entry, the application used, and the progress of the work.

[0314] Step 3:

[0315] The emotion engine recognizes the user's emotional state using biometric data such as facial expressions, voice, and heart rate, and is supported by cameras, microphones, and wearable devices attached to the device.

[0316] Step 4:

[0317] The device transmits the recorded behavioral and emotional data to a server at regular intervals, where the data is efficiently collected via a communication protocol.

[0318] Step 5:

[0319] The server preprocesses the received data, such as filling in missing data, removing noise, and normalizing the data. The same preprocessing is performed on emotion data.

[0320] Step 6:

[0321] The server inputs the preprocessed data into a generative model, which uses machine learning algorithms to analyze and learn correlations between user behavior patterns and emotions.

[0322] Step 7:

[0323] The generative model calculates the probability of certain patterns or irregular behavior based on the user's behavioral patterns and emotional data. For example, it determines that a user is likely to feel stressed at a certain time of day.

[0324] Step 8:

[0325] The server generates a manual based on the learned behavioral patterns and emotional data, which includes commonly performed tasks and procedures, as well as customization according to the user's emotional state.

[0326] Step 9:

[0327] The server delivers the generated manual to the user's terminal, where the user can check the latest manual and proceed with their work accordingly.

[0328] Step 10:

[0329] The server continues to analyze data in real time, and if any changes in behavioral patterns or emotional states are detected, the manual is immediately updated. This updated information is notified to the user, and the new manual is automatically distributed to the device.

[0330] Step 11:

[0331] Users receive notifications and follow the updated manual to carry out their work, which not only improves work efficiency and standardization but also provides a work environment that takes into account the user's emotional state.

[0332] Example 2

[0333] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0334] Conventional systems only collect user behavioral data and do not consider emotional states, which means they are unable to improve work efficiency or adequately manage the user's health. In particular, they are unable to grasp when a user feels stressed or their emotional state during a specific task, making it difficult to provide appropriate manuals. As a result, users' work efficiency declines and their mental burden increases.

[0335] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0336] In this invention, the server includes means for collecting user behavioral data and emotional data, means for preprocessing the collected behavioral data and emotional data, and generative model means for analyzing the preprocessed behavioral data and emotional data and learning behavioral patterns and emotional states. This makes it possible to generate and distribute optimal manuals that take into account not only the user's behavior but also their emotional states.

[0337] A "user" is a person who operates the system and provides behavioral and emotional data.

[0338] "Behavioral data" is information related to a user's terminal operations, and includes specific operation content, operation timing, applications used, input data, and the like.

[0339] "Emotion data" is information that represents the user's emotional state, and includes psychological states such as stress, joy, anger, and sadness that can be obtained from facial expressions and voice.

[0340] "Means for collection" is a general term for hardware and software for recording user behavioral data and emotional data in real time and transmitting it to a server.

[0341] "Preprocessing means" is a general term for the process of filling in missing data, removing noise, normalizing data, etc. on collected raw data, as well as the technical equipment and software used for this process.

[0342] "Generative model means" is a general term for a machine learning model that learns a user's behavioral patterns and emotional states using preprocessed behavioral data and emotional data, and the system that operates it.

[0343] "Means for generating manuals" is a general term for algorithms and systems that automatically create manuals that list optimal work procedures and important points based on learned behavioral patterns and emotional states.

[0344] "Distribution means" is a general term for a network and related hardware and software for transmitting the generated manual to a user's terminal and enabling the user to access it.

[0345] This invention relates to a system that collects user behavioral and emotional data, analyzes and learns behavioral patterns and emotional states based on this data, and automatically generates and updates effective manuals. This system makes it possible to improve work efficiency and introduce human-centered design.

[0346] The system consists of the following elements:

[0347] 1. Data Collection Methods

[0348] 2. Data preprocessing methods

[0349] 3. Learning methods for behavioral patterns and emotions (generative models)

[0350] 4. Manual Generation Methods

[0351] 5. Manual Distribution Methods

[0352] 6. Updates and Notifications

[0353] Data collection methods

[0354] Users go about their daily work and log in to the system's terminals to perform various tasks. Specifically, they log in to the system, use applications, enter data, check email, etc. The terminals record user operations in real time. For example, they collect operation data such as keyboard input, mouse operation, and application launch history, and record them along with timestamps. In addition, an emotion engine is used to recognize the user's emotional state from facial expression and voice data, and this information is also included in the behavioral data.

[0355] Data preprocessing measures

[0356] The server receives the raw data sent from the device. It preprocesses the received data separately for behavioral data and emotional data. This includes filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data and noise such as momentary emotional changes is removed. Data normalization converts data of different scales into a consistent range.

[0357] A means of learning behavioral patterns and emotions

[0358] The server inputs the preprocessed behavioral and emotional data into a generative model to learn the user's behavioral patterns and emotional state. Using machine learning algorithms, such as deep learning libraries like TensorFlow and PyTorch, correlations between the user's behavioral and emotional data can be analyzed. At this stage, the model identifies stress responses for specific tasks or times of day.

[0359] Manual Generation Method

[0360] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customizations based on the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed.

[0361] Manual distribution method

[0362] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[0363] Updates and Notifications

[0364] The server analyzes data in real time, and if any changes in behavioral patterns or emotional states are detected, it immediately updates the manual and notifies the user. The new manual is automatically distributed to the device.

[0365] Specific examples

[0366] For example, when User B, who works for a company, goes about his daily work, his device records behavioral and emotional data. User B starts entering data at 9:00 every day and checks his email at 10:00, but the data shows that he starts to feel stressed around 14:00 every day. This data is sent to the server in real time and pre-processed on the server.

[0367] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[0368] 1. Data entry begins at 9:00 every day.

[0369] Required tools: Datasheet, input software

[0370] Note: Double-check to avoid input errors.

[0371] 2. Check email at 10:00.

[0372] Tools used: Email client

[0373] Important Note: Prioritize important emails

[0374] 3. Take a short break every day at 2:00 PM.

[0375] The break time is 15 minutes

[0376] Note: It is recommended to rest in a relaxing environment.

[0377] This manual is delivered to User B's device, and User B can follow it to perform his / her work efficiently. In particular, taking a break during times when he / she feels stressed improves work efficiency and maintains his / her health.

[0378] Example prompts to input to the generative AI model

[0379] For example, the following prompt sentence is input to the generative AI model:

[0380] Please provide prompts to automatically generate optimal work procedures based on User B's behavioral patterns and emotional state when performing daily work at the company.

[0381] As a specific data-based procedure, User B starts entering data at 9:00 every morning and checks email at 10:00, but tends to feel stressed by 14:00.

[0382] Please generate a manual based on this.

[0383] In this way, the system of the present invention provides optimal work procedures that match the user's emotional state, thereby improving work efficiency and managing the user's health.

[0384] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0385] Step 1: Data collection

[0386] Users go about their daily work and log in to the system's terminals to perform various tasks, such as data entry, email checking, and report creation. The terminals monitor these operations in real time and collect operation data. Furthermore, the emotion engine uses the terminal's camera and microphone to analyze facial expressions and voice to collect emotional data. The input is the user's operation data and emotional state, and this data is recorded with a timestamp and a data package is generated to be sent to the server as output.

[0387] Step 2: Data Preprocessing

[0388] The server receives raw data sent from the device. The received data undergoes preprocessing such as filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data, and noise caused by momentary changes in emotions is removed. It also performs normalization to convert data of different scales into a consistent range. The input is the raw data sent from the device, and the output is the preprocessed, clean data.

[0389] Step 3: Learning behavioral patterns and emotions

[0390] The server inputs the preprocessed behavioral data and emotional data into a generative model to learn the user's behavioral patterns and emotional state. For example, it uses machine learning libraries such as Python's TensorFlow or PyTorch to analyze the correlation between the user's behavioral data and emotional data. The learning model identifies the user's stress response for specific tasks and time periods. The input is the preprocessed data, and the output is the trained generative model.

[0391] Step 4: Manual generation

[0392] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customized content according to the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed. The input is the trained generative model and real-time user data, and the output is the generated manual.

[0393] Step 5: Manual distribution

[0394] The server delivers the generated manual to the user's terminal. The user can perform their work efficiently through the latest manual. The generated manual is sent to the terminal, and the user proceeds with their work based on it. The input is the generated manual, and the output is the delivered manual.

[0395] Step 6: Updates and Notifications

[0396] The server analyzes new data in real time and immediately updates the manual if it detects changes in the user's behavioral patterns or emotional state. The updated manual is immediately notified to the user, and the new manual is automatically delivered to the device. The input is new user data and the existing generative model, and the output is the updated manual and its notification.

[0397] (Application example 2)

[0398] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0399] Conventional work manuals are generally rigid and designed without considering the behavioral data or emotional state of individual users, so they cannot be said to be effective in improving work efficiency within a company or managing employee health. In particular, there is a demand for systems that can flexibly respond to stress and irregular behavior that occur during work.

[0400] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, generative model means for analyzing the preprocessed behavioral data and learning behavioral patterns, emotion engine means for recognizing the user's emotional state, means for generating a manual based on the learned behavioral patterns and emotional state, and means for delivering the generated manual to the user's terminal. This makes it possible to provide a flexible and effective business manual that corresponds to the user's behavioral data and emotional state.

[0401] "Means for collecting user behavior data" refers to a device or software for collecting in real time operation logs, usage histories, and other behavioral information generated when a user operates the system.

[0402] "Means for preprocessing collected behavioral data" refers to a device or software that performs processes such as filling in missing data, normalizing data, and removing noise in order to convert the collected behavioral data into a format suitable for analysis.

[0403] The "generative model means for learning behavioral patterns" is a device or software for analyzing a user's behavioral patterns based on preprocessed behavioral data and learning specific behavioral patterns using a machine learning algorithm.

[0404] The "emotion engine means for recognizing the user's emotional state" is a device or software for analyzing the user's operation data, facial expressions, voice, etc. to recognize the user's emotional state.

[0405] The "means for generating a manual based on learned behavioral patterns and emotional states" refers to a device or software for automatically generating a manual including user-specific work procedures and points of caution based on learned behavioral patterns and emotional data.

[0406] The "means for distributing the generated manual to the user's terminal" refers to a device or software for providing the generated manual to the terminal used by the user.

[0407] This invention is a system that collects user behavioral data and emotional states, analyzes and learns from them, and automatically generates and updates flexible business manuals. A specific embodiment of this system is described below.

[0408] System configuration

[0409] Hardware configuration:

[0410] Data collection device: A smartphone or other device operated by the user.

[0411] Server: A central server that performs data preprocessing, training, manual generation, and distribution.

[0412] Software configuration:

[0413] Data collection module: collects user behavior data.

[0414] Pre-processing module: The collected data is imputed, denoised, and normalized.

[0415] Machine learning algorithms (e.g. scikit-learn, numpy): Generative models that learn behavioral patterns.

[0416] Emotion Engine: Recognize the user's emotional state (Example: EmotionEngine).

[0417] Manual generation module: Generates a manual based on learned behavioral patterns and emotion data.

[0418] Distribution module: distributes the generated manual to the user's terminal.

[0419] What the program does

[0420] 1. Data Collection Methods

[0421] The server monitors user operations in real time and collects behavioral data, including application usage history, operation logs, and other behavioral information, and simultaneously recognizes the user's emotional state using an emotion engine.

[0422] 2. Data preprocessing methods

[0423] The collected behavioral data is preprocessed on the server. The preprocessing module performs data filling, noise removal, and data normalization to convert the data into a format suitable for analysis and learning.

[0424] 3. Behavioral pattern learning method

[0425] The preprocessed data is then fed into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning technique that analyzes behavioral patterns and correlates them with emotional states.

[0426] 4. Emotional Engine Means

[0427] The emotion engine recognizes the user's emotional state and correlates it with behavioral patterns, for example, analyzing stress levels during specific tasks or times of day.

[0428] 5. Manual Generation Methods

[0429] The server generates a work manual based on the learned behavioral patterns and emotional states, which includes specific work procedures, points to note, and advice based on the emotional state.

[0430] 6. Manual Distribution Methods

[0431] The generated manual is automatically distributed from the server to the user's device, allowing the user to always perform their work based on the latest operational manual.

[0432] Specific examples

[0433] For example, when a user works in a physical store, the system collects the user's behavioral and emotional data. The data shows that the user puts products on the shelves at 9:00 every morning and rings up the cash register at 10:00. However, it is detected that the user begins to feel stressed around 2:00 pm. Based on this information, the system generates the following manual:

[0434] Example prompt sentence:

[0435] Based on User A's behavioral patterns and emotional data, you can see that he feels stressed at 2 PM. Based on this data, generate an optimal work manual. The manual should include work procedures and points to note depending on the user's emotional state.

[0436] This allows users to receive specific advice on how to reduce stress, along with the optimal work procedures based on their emotional state. This system can simultaneously improve work efficiency and manage employee health.

[0437] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0438] Step 1:

[0439] Data collection

[0440] The terminal collects operation logs, usage history, and other behavioral data generated when the user operates the system in real time. It also uses an emotion engine to simultaneously recognize and record the user's emotional state. Specifically, the terminal records the user's operation details (e.g., arranging products, ringing up cash registers, etc.) and the time of operation, and sends this data to the server. The input is the user's operation data and emotional state, and the output is the collected raw data.

[0441] Step 2:

[0442] Data Preprocessing

[0443] The server receives the collected behavioral data and performs preprocessing. The preprocessing module performs processes such as filling in missing data, removing noise, and normalizing the data, converting it into a format suitable for analysis and learning. For example, if the collected data contains missing data, it is filled in based on past data and outliers are removed. The input is the collected raw data, and the output is preprocessed, clean data.

[0444] Step 3:

[0445] Behavioral pattern learning

[0446] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning method that analyzes behavioral patterns and correlates them with emotional states. For example, the server learns what tasks the user is performing at a particular time and how their emotional state is changing. The input is the preprocessed data, and the output is the learned behavioral patterns.

[0447] Step 4:

[0448] Emotion Engine Analysis

[0449] The server uses an emotion engine to analyze the relationship between the user's emotional state and behavioral patterns. Specifically, it analyzes stress levels during specific tasks and time periods. The input is the collected emotion data, and the output is the emotion analysis results.

[0450] Step 5:

[0451] Manual Generation

[0452] The server generates a work manual based on the learned behavioral patterns and emotional state. The generated manual includes specific work procedures, points to note, and advice tailored to the employee's emotional state. For example, it suggests taking a break during times when employees are likely to feel stressed. The input is the learned behavioral patterns and the results of emotional analysis, and the output is the generated work manual.

[0453] Step 6:

[0454] Manual Distribution

[0455] The server distributes the generated manual to the user's terminal. The terminal receives it and displays it for easy reference by the user. If the manual needs to be updated, a notification is sent in real time. The input is the generated business manual, and the output is the distribution of the manual to the user's terminal and notification.

[0456] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0457] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0458] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0459] [Second embodiment]

[0460] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0461] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0462] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0463] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0464] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0465] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0466] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0467] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0468] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0469] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0470] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0471] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0472] This invention is a system that collects user behavior data, analyzes behavioral patterns based on that data, and automatically generates and updates effective manuals, allowing operators and employees to focus on their daily work and improving the efficiency and standardization of work.

[0473] System configuration

[0474] This system is broadly composed of the following elements:

[0475] 1. Data Collection Methods

[0476] 2. Data preprocessing methods

[0477] 3. Learning methods for behavioral patterns (generative models)

[0478] 4. Manual Generation Methods

[0479] 5. Manual Distribution Methods

[0480] 6. Updates and Notifications

[0481] Specific processing flow

[0482] 1. Data Collection

[0483] When a user performs work, work data on the terminal is recorded in real time, such as what time the user logged in to the system and how long each application was used.

[0484] The device collects this behavioral data and sends it to a server at regular intervals.

[0485] 2. Data Preprocessing

[0486] The server analyzes the collected raw data and performs data cleansing, a process that removes incomplete data and outliers and converts the data into a consistent format.

[0487] For example, if the time record for an operation is missing, a reasonable estimate is made from the data before and after.

[0488] 3. Learning behavioral patterns

[0489] The generative model on the server receives the preprocessed data as input and analyzes and learns from the user's behavioral patterns using machine learning and statistical methods.

[0490] The probability of certain patterns and irregular behavior is calculated and accumulated.

[0491] 4. Manual Generation

[0492] Based on the learned behavioral patterns, the server automatically generates a manual containing specific work procedures and points to note. This manual details standard procedures that users should refer to and how to deal with irregular situations.

[0493] For example, if there is a pattern where data entry starts at 9:00 every day, email is checked at 10:00, and lunch is taken at 12:00, the corresponding procedures will be reflected in the manual.

[0494] 5. Manual Distribution

[0495] The server automatically distributes the generated manual to the user's terminal via the network, allowing the user to always have the latest manual at hand.

[0496] 6. Updates and Notifications

[0497] The server analyzes data in real time and immediately updates the manual if any changes occur in behavioral patterns. This update information is notified to the user, and the new manual is automatically distributed to the terminal.

[0498] For example, if system maintenance is performed on Thursdays only during a particular week, that information will also be reflected promptly.

[0499] Specific examples

[0500] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative model.

[0501] Based on the behavioral patterns learned by the generative model, the following manual is automatically generated:

[0502] 1. Data entry begins at 9:00 every day.

[0503] Required tools: Datasheet, input software

[0504] Note: Double-check to avoid input errors.

[0505] 2. Check email at 10:00.

[0506] Tools used: Email client

[0507] Important Note: Prioritize important emails

[0508] 3. System maintenance will be performed every Friday.

[0509] Tools used: Maintenance software

[0510] Important note: Make a backup of your entire system beforehand.

[0511] This manual is distributed to User A's terminal, and User A follows it to perform his / her work, resulting in efficient and standardized work. Furthermore, if there is a change in the actual behavioral pattern (e.g., system maintenance is changed to Thursday), an updated manual is immediately distributed from the server, allowing the user to follow the latest procedures.

[0512] The processing flow will be explained below.

[0513] Step 1:

[0514] The user begins their daily work, specifically using the device to perform tasks such as logging in, launching applications, data entry, checking email, and attending meetings.

[0515] Step 2:

[0516] The device records real-time data about user operations and activities, including application usage time, task start and end times, and data entered.

[0517] Step 3:

[0518] The device transmits the collected data to the server at regular intervals. This data transmission is performed periodically, and the behavioral data is efficiently aggregated on the server via a communication protocol.

[0519] Step 4:

[0520] The server preprocesses the received behavioral data, such as filling in missing data, removing noise, and normalizing the data, to make it ready for analysis.

[0521] Step 5:

[0522] The server feeds the preprocessed data into a generative model, which uses machine learning algorithms to learn the user's behavioral patterns.

[0523] Step 6:

[0524] The generative model analyzes user behavior patterns and calculates the probability of repeated tasks and irregular behavior, thereby providing a more accurate understanding of work flows.

[0525] Step 7:

[0526] Based on the learned behavioral patterns, the server generates a manual detailing work procedures and precautions, including commonly performed tasks and procedures, as well as how to deal with irregular behavior.

[0527] Step 8:

[0528] The server distributes the generated manual to the user's terminal, where the user can check the latest manual and perform their work accordingly.

[0529] Step 9:

[0530] The server analyzes the behavioral data in real time, and if any changes in behavioral patterns are detected, the manual is updated immediately. This update information is notified to the user, and the new manual is automatically distributed to the device.

[0531] Step 10:

[0532] Users receive notifications and follow the updated manuals to carry out their work. By repeating this process, work is continuously standardized and made more efficient.

[0533] Example 1

[0534] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0535] Conventional work support systems have difficulty in properly collecting and analyzing user behavior data, making it impossible to automatically generate work instructions optimized for individual users. As a result, improvements in work efficiency and standardization are not fully achieved, and there is a lack of appropriate countermeasures, particularly for irregular behavior. Furthermore, real-time work management is difficult because work instructions are not updated or notified immediately.

[0536] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0537] In this invention, the server includes means for collecting user activity data, means for preprocessing the collected activity data, means for analyzing the preprocessed activity data and learning behavioral patterns, means for generating work instructions based on the learned behavioral patterns, and means for distributing the generated work instructions to the user's device. This makes it possible to efficiently collect and analyze user activity data and automatically generate and distribute work instructions optimized for each user. Furthermore, real-time work management is realized by calculating the probability of atypical behavior and instantly updating and notifying work instructions.

[0538] "User" refers to an individual or organization that uses this system and is the entity that carries out specific tasks.

[0539] "Activity data" is a series of digital information generated when a user performs work, and includes login time, application usage history, operation details, and the like.

[0540] "Preprocessing" refers to a series of steps that analyze the collected raw data, remove incomplete data and outliers, and convert the data into a certain format.

[0541] A "generative model" refers to an algorithm or program that uses machine learning or statistical techniques to learn user behavior patterns from preprocessed data.

[0542] "Work instructions" refers to documents containing work procedures and points to note that are generated based on learned behavioral patterns, and serve as guidelines for users to carry out their daily work in an efficient and standardized manner.

[0543] "Distribution" refers to the process of sending the generated work instructions to the user's device so that the user can view them.

[0544] "Atypical behavior" refers to irregular operations or actions that deviate from normal behavior patterns, and is reflected in the manual by calculating the probability of such behavior.

[0545] "Immediate update" refers to the process of quickly updating the contents of work instructions based on data collected in real time, and notifying the user of the results immediately.

[0546] This invention is a system that collects user activity data, analyzes behavioral patterns based on that data, and automatically generates and updates work instructions, allowing operators and employees to focus on their daily work and improving work efficiency and standardization.

[0547] System configuration

[0548] The system consists of the following main components:

[0549] 1. Data Collection Methods

[0550] When a user starts work, they log in to the system, and their login time and operation details are collected.

[0551] The device collects a series of user operation data in real time and sends it to the server at regular intervals, for example, every minute.

[0552] 2. Data preprocessing methods

[0553] The server performs an initial analysis of the raw data received and performs data cleansing, a process that removes incomplete data and outliers and converts all data into a unified format.

[0554] For example, the timestamp format is standardized to Coordinated Universal Time (UTC) and missing parts of the log are reasonably estimated and supplemented.

[0555] 3. Behavioral pattern learning method (generative AI model)

[0556] A generative AI model hosted on a server receives preprocessed data as input and analyzes and learns from user behavior patterns using machine learning algorithms and statistical methods.

[0557] To learn behavioral patterns, the probability and typical patterns of these activities are stored in the model based on past data.

[0558] 4. Manual Generation Methods

[0559] Based on the learned behavioral patterns, the server automatically generates work instructions, which detail standard work procedures and how to handle irregular situations.

[0560] For example, a pattern such as "Start data entry at 9:00 every day and check email at 10:00" is clearly stated.

[0561] 5. Manual Distribution Methods

[0562] The server automatically delivers the generated work instructions to the user's device, allowing the user to always have the latest work instructions at hand.

[0563] The terminal displays the received work instructions in an appropriate format for easy reference by the user.

[0564] 6. Updates and Notifications

[0565] The server analyzes the data collected in real time and immediately updates the work instructions if there is a change in the behavioral pattern. This updated information is notified and new work instructions are automatically sent to the terminal.

[0566] For example, if system maintenance is performed only on Thursdays of certain weeks, that change will also be reflected immediately.

[0567] Specific examples

[0568] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative AI model.

[0569] Based on the behavioral patterns learned by the generative model, the following work instructions are automatically generated:

[0570] markdown

[0571] 1. Data entry begins at 9:00 every day.

[0572] Required tools: Datasheet, input software

[0573] Note: Double-check to avoid input errors.

[0574] 2. Check email at 10:00.

[0575] Tools used: Email client

[0576] Important Note: Prioritize important emails

[0577] 3. System maintenance will be performed every Friday.

[0578] Tools used: Maintenance software

[0579] Important note: Make a backup of your entire system beforehand.

[0580] This work instruction is delivered to User A's terminal, and User A follows it to perform the work, resulting in efficient and standardized work. In addition, if there is a change in the actual behavior pattern (for example, if system maintenance is changed to Thursday), the server immediately delivers updated work instructions, allowing the user to follow the latest procedures.

[0581] Prompt Sentence Examples

[0582] An example of a prompt to input to a generative AI model is as follows:

[0583] Please tell me the specific steps to take when entering data.

[0584] What should I pay attention to when checking my email?

[0585] "What is your standard procedure for system maintenance?"

[0586] keyword

[0587] Generative AI Models

[0588] Prompt statement

[0589] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0590] Step 1:

[0591] Data collection

[0592] When a user starts work, he logs in to the system, and the login time is recorded on the terminal.

[0593] For each application a user uses during work, its start and end times are collected in real time.

[0594] The terminal sends the collected data to the server in batch processing every minute, allowing operation logs on the terminal to be accumulated in real time.

[0595] Input: User operation data (e.g., login time, application usage time).

[0596] Output: The collected operation data is sent to the server.

[0597] Step 2:

[0598] Data Preprocessing

[0599] The server analyzes the raw data it receives and performs data cleansing, which includes removing incomplete data and outliers.

[0600] The server standardizes the timestamp format (e.g., converts it to Coordinated Universal Time (UTC)) and fills in missing data with estimated values.

[0601] Specific operation of data cleansing: If there is a missing login time, the average value is calculated from the data before and after the missing time to fill in the gaps.

[0602] Input: Collected operational data.

[0603] Output: The cleansed data.

[0604] Step 3:

[0605] Learning behavioral patterns

[0606] A generative AI model on a server receives the preprocessed data as input and begins analyzing the data using machine learning algorithms (e.g., recurrent neural networks).

[0607] The probability of certain patterns or abnormal behavior is calculated, and these models are gradually refined.

[0608] Specific operation: The generative AI model learns the average start time and frequency of a user's actions and stores the results in a database.

[0609] Input: Preprocessed data.

[0610] Output: Learned behavioral patterns (model).

[0611] Step 4:

[0612] Manual Generation

[0613] The server automatically generates work instructions based on the learned behavioral patterns, including routine work procedures and countermeasures for irregular situations.

[0614] Example: A standardized procedure such as "Start data entry at 9:00 and check email at 10:00 every day" is described.

[0615] Input: Learned behavioral patterns.

[0616] Output: The generated work order.

[0617] Step 5:

[0618] Manual Distribution

[0619] The server automatically delivers the generated work instructions to the user's device, allowing the user to always refer to the latest work instructions.

[0620] The terminal displays the received work instructions in an appropriate format for easy access by the user.

[0621] Notification function: Displays when a new manual has arrived on your device.

[0622] Input: Generated work order.

[0623] Output: Work instructions delivered to the terminal.

[0624] Step 6:

[0625] Updates and Notifications

[0626] The server analyzes the data collected in real time and immediately updates the work instructions if any changes occur in the behavioral patterns. The new work instructions are then delivered to the user's device along with the updated information.

[0627] Example: If the system maintenance schedule is changed, users will be notified immediately.

[0628] Input: Real-time behavioral data.

[0629] Output: Updated work order and notification.

[0630] (Application example 1)

[0631] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0632] There are already systems that collect user behavior data and analyze behavioral patterns based on that data to improve business efficiency and standardize operations. However, these systems lack real-time update capabilities or the ability to specifically control device operation, resulting in a lack of responsiveness and adaptability to business operations. Furthermore, they lack the ability to automatically generate manuals that reflect irregular behavior, limiting their ability to maximize user work efficiency. The purpose of this invention is to solve these problems and achieve higher levels of business efficiency and adaptability.

[0633] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0634] In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, means for analyzing the preprocessed behavioral data and learning behavioral patterns, means for generating a manual based on the learned behavioral patterns, means for distributing the generated manual to the user's device, means for controlling the operation of the device according to the set pattern, means for instantly updating the manual to the latest version based on real-time behavioral data, and means for notifying the device of the contents of the updated manual. This enables real-time business optimization and standardization based on the user's business data.

[0635] "Behavioral data" refers to the history and log information of specific operations and actions performed by a user.

[0636] "Preprocessing" is the process of converting raw data into a form suitable for analysis, including data cleansing and format conversion.

[0637] A "generative model" is a machine learning algorithm or statistical model that learns behavioral patterns based on preprocessed data.

[0638] A "manual" is a document or guideline that describes procedures and instructions for users to perform their work efficiently.

[0639] "Devices" refer to hardware devices such as terminals and robots used by users.

[0640] "Control" refers to the operation of directing the behavior or state of equipment, and is the process of managing behavior according to specific patterns.

[0641] "Real-time update" is the process of instantly updating the manual based on the latest behavioral data and delivering that information to the user's device.

[0642] "Irregular behavior" refers to unusual actions or operations that deviate from normal patterns of behavior, and procedures for dealing with such behavior are reflected in the manual.

[0643] This invention is a system that automatically generates and updates business manuals by collecting user behavior data and analyzing behavioral patterns based on the collected data. An embodiment of this system will be described in detail below.

[0644] System Configuration

[0645] The system mainly consists of the following components:

[0646] 1. How to collect user behavior data

[0647] 2. A means of preprocessing the collected behavioral data

[0648] 3. A generative modeling method that analyzes preprocessed behavioral data and learns behavioral patterns

[0649] 4. A method for generating manuals based on learned behavioral patterns

[0650] 5. Means for delivering the generated manual to the user's device

[0651] 6. Means of controlling the operation of equipment according to set patterns

[0652] 7. A way to instantly update manuals based on real-time behavioral data

[0653] 8. Means of notifying the device of updated manual contents

[0654] Operation overview

[0655] First, as users perform their work, sensors and cameras on devices (e.g., terminals and factory robots) collect behavioral data in real time. This data includes operating hours, tools used, movement patterns, and more. The collected data is sent to a server where preprocessing is performed. This involves filling in missing data and removing outliers. Next, a generative model learns behavioral patterns based on the preprocessed data. A machine learning algorithm is used for this purpose.

[0656] Based on the learned behavioral patterns, the system generates a work manual that includes specific operating procedures, tools to use, and important points to note. For example, the following manual may be generated:

[0657] 1. Starts operation at 8:00 every day.

[0658] Tools used: Gripper, drill.

[0659] Important note: Perform initial calibration of the gripper.

[0660] 2. Replace part A at 12:00.

[0661] Tools used: Torque wrench.

[0662] Note: Check the torque value.

[0663] 3. Full maintenance every Friday.

[0664] Tools used: Various tools.

[0665] Important note: Make a full system backup.

[0666] The generated manual is automatically distributed to the user's device and used to appropriately control the device's operation. Furthermore, the manual is instantly updated in response to changes in behavior patterns in real time. This update information is notified to the device, so the latest procedures are always reflected.

[0667] Hardware and software used

[0668] Collection devices: sensors, cameras, terminals, factory robots

[0669] Data preprocessing: Pandas (Python library)

[0670] Generative models: Random Forest (machine learning algorithm), other AI models

[0671] Data analysis server: High performance server

[0672] Distribution System: Network Communication

[0673] Prompt Sentence Examples

[0674] An example prompt based on a generative AI model is:

[0675] Based on the user's behavioral data, generate a robot operation manual like the one below.

[0676] 1. Starts operation at 8:00 every day.

[0677] Tools used: Gripper, drill.

[0678] Important note: Perform initial calibration of the gripper.

[0679] 2. Replace part A at 12:00.

[0680] Tools used: Torque wrench.

[0681] Note: Check the torque value.

[0682] 3. Full maintenance is performed automatically every Tuesday.

[0683] Tools used: Various tools.

[0684] Important note: Make a full system backup.

[0685] The above describes a specific embodiment of the invention. By using this system, it is possible to achieve greater efficiency and standardization of work, thereby maximizing the user's work performance.

[0686] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0687] Step 1:

[0688] The server collects user behavior data. This input data includes user uptime, tools and applications used, and movement patterns. This data is collected in real time from sensors, cameras, and devices.

[0689] Step 2:

[0690] The server preprocesses the collected behavioral data. This preprocessing includes filling in missing data, removing outliers, and converting the data into a format suitable for analysis. Raw data is used as input, and cleansed data is output after processing. Data processing libraries such as Pandas are used.

[0691] Step 3:

[0692] The server supplies the preprocessed data as input to the generative model. The generative model uses machine learning algorithms such as Random Forest to learn behavioral patterns. The input data is preprocessed behavioral data, and the output is learned behavioral patterns. The model runs on the server and performs analysis.

[0693] Step 4:

[0694] The server generates a manual based on the learned behavioral patterns. Specifically, a manual is automatically generated that describes the user's work procedures, tools to be used, and points to note. In this process, the learned behavioral patterns are used as input data, and the work manual is output.

[0695] Step 5:

[0696] The server distributes the generated manual to the user's device. The manual is automatically sent to the device via network communication. The input data is the generated manual, and the output is the state that has been distributed to the user's device.

[0697] Step 6:

[0698] The server controls the operation of the equipment according to the set patterns. Specifically, it manages the equipment so that it follows the manual and performs its tasks efficiently and accurately. The input data is the distributed manual, and the output is the actual operation control of the equipment.

[0699] Step 7:

[0700] The server instantly updates the manual to the latest version based on real-time behavioral data. It continuously collects user behavioral data and immediately updates the manual if there are any changes. The input data is the latest behavioral data, and the output is an updated manual.

[0701] Step 8:

[0702] The server notifies the user's device of the contents of the updated manual, allowing the user to always perform their work based on the latest procedures. The input data is the updated manual, and the output is a notification sent to the user's device.

[0703] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0704] This invention is a system that collects user behavior data, analyzes and learns behavioral patterns based on this data, and automatically generates and updates effective manuals. Furthermore, by combining it with an emotion engine that recognizes the user's emotional state, it realizes flexible responses based on the user's emotions. This system makes it possible to improve work efficiency and introduce human-centered design.

[0705] System configuration

[0706] The system consists of the following components:

[0707] 1. Data Collection Methods

[0708] 2. Data preprocessing methods

[0709] 3. Learning methods for behavioral patterns (generative models)

[0710] 4. Emotion Engine

[0711] 5. Manual Generation Methods

[0712] 6. Manual Distribution Methods

[0713] 7. Updates and Notifications

[0714] Specific processing flow

[0715] 1. Data Collection

[0716] Users begin their daily work and perform tasks on the device, such as logging in to the system, using applications, entering data, and checking email.

[0717] The device records the user's operation data in real time and periodically transmits the data to the server. In addition, the emotion engine recognizes the user's emotional state, which is also included in the behavioral data.

[0718] 2. Data Preprocessing

[0719] The server preprocesses the received data, filling in missing data, removing noise, and normalizing the data. The server also preprocesses the user's emotion data.

[0720] 3. Learning behavioral patterns and emotions

[0721] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns and emotional state. Specifically, it uses machine learning algorithms to analyze the correlation between user behavior and emotions.

[0722] For example, the model learns which specific tasks or times of day users find stressful.

[0723] 4. Manual Generation

[0724] Based on the learned behavioral patterns and emotional data, the server automatically generates a manual detailing work procedures and important points to note. This manual includes not only standard work procedures but also customizations based on the user's emotional state.

[0725] For example, if the user is feeling stressed, it may suggest taking a short break.

[0726] 5. Manual Distribution

[0727] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[0728] 6. Updates and Notifications

[0729] The server analyzes the data in real time and immediately updates the manual if any changes in behavioral patterns or emotional states are detected. This update information is notified to the user, and the new manual is automatically distributed to the device.

[0730] Specific examples

[0731] User B, who works for a certain company, records behavioral and emotional data on his device as he goes about his daily work. User B starts entering data at 9:00 and checks his email at 10:00 every day. However, the data shows that he begins to feel stressed around 14:00 every day. This data is sent to a server in real time and pre-processed on the server.

[0732] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[0733] 1. Data entry begins at 9:00 every day.

[0734] Required tools: Datasheet, input software

[0735] Note: Double-check to avoid input errors.

[0736] 2. Check email at 10:00.

[0737] Tools used: Email client

[0738] Important Note: Prioritize important emails

[0739] 3. Take a short break every day at 2:00 PM.

[0740] The break time is 15 minutes

[0741] Note: It is recommended to rest in a relaxing environment.

[0742] This manual is distributed to User B's device, and User B can follow it to perform his / her work efficiently. Taking breaks during times of stress in particular improves work efficiency and maintains health. The program automates the entire process, from data collection to manual generation and updating, reducing the user's workload.

[0743] This system allows users to perform tasks using procedures that are optimal for their emotional state, which not only improves work efficiency and standardizes work, but also contributes to health management.

[0744] The processing flow will be explained below.

[0745] Step 1:

[0746] A user begins their daily routine, for example, logging into a terminal and beginning a data entry task.

[0747] Step 2:

[0748] The device records user operation data in real time, including the start time of data entry, the application used, and the progress of the work.

[0749] Step 3:

[0750] The emotion engine recognizes the user's emotional state using biometric data such as facial expressions, voice, and heart rate, and is supported by cameras, microphones, and wearable devices attached to the device.

[0751] Step 4:

[0752] The device transmits the recorded behavioral and emotional data to a server at regular intervals, where the data is efficiently collected via a communication protocol.

[0753] Step 5:

[0754] The server preprocesses the received data, such as filling in missing data, removing noise, and normalizing the data. The same preprocessing is performed on emotion data.

[0755] Step 6:

[0756] The server inputs the preprocessed data into a generative model, which uses machine learning algorithms to analyze and learn correlations between user behavior patterns and emotions.

[0757] Step 7:

[0758] The generative model calculates the probability of certain patterns or irregular behavior based on the user's behavioral patterns and emotional data. For example, it determines that a user is likely to feel stressed at a certain time of day.

[0759] Step 8:

[0760] The server generates a manual based on the learned behavioral patterns and emotional data, which includes commonly performed tasks and procedures, as well as customization according to the user's emotional state.

[0761] Step 9:

[0762] The server delivers the generated manual to the user's terminal, where the user can check the latest manual and proceed with their work accordingly.

[0763] Step 10:

[0764] The server continues to analyze data in real time, and if any changes in behavioral patterns or emotional states are detected, the manual is immediately updated. This updated information is notified to the user, and the new manual is automatically distributed to the device.

[0765] Step 11:

[0766] Users receive notifications and follow the updated manual to carry out their work, which not only improves work efficiency and standardization but also provides a work environment that takes into account the user's emotional state.

[0767] Example 2

[0768] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0769] Conventional systems only collect user behavioral data and do not consider emotional states, which means they are unable to improve work efficiency or adequately manage the user's health. In particular, they are unable to grasp when a user feels stressed or their emotional state during a specific task, making it difficult to provide appropriate manuals. As a result, users' work efficiency declines and their mental burden increases.

[0770] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0771] In this invention, the server includes means for collecting user behavioral data and emotional data, means for preprocessing the collected behavioral data and emotional data, and generative model means for analyzing the preprocessed behavioral data and emotional data and learning behavioral patterns and emotional states. This makes it possible to generate and distribute optimal manuals that take into account not only the user's behavior but also their emotional states.

[0772] A "user" is a person who operates the system and provides behavioral and emotional data.

[0773] "Behavioral data" is information related to a user's terminal operations, and includes specific operation content, operation timing, applications used, input data, and the like.

[0774] "Emotion data" is information that represents the user's emotional state, and includes psychological states such as stress, joy, anger, and sadness that can be obtained from facial expressions and voice.

[0775] "Means for collection" is a general term for hardware and software for recording user behavioral data and emotional data in real time and transmitting it to a server.

[0776] "Preprocessing means" is a general term for the process of filling in missing data, removing noise, normalizing data, etc. on collected raw data, as well as the technical equipment and software used for this process.

[0777] "Generative model means" is a general term for a machine learning model that learns a user's behavioral patterns and emotional states using preprocessed behavioral data and emotional data, and the system that operates it.

[0778] "Means for generating manuals" is a general term for algorithms and systems that automatically create manuals that list optimal work procedures and important points based on learned behavioral patterns and emotional states.

[0779] "Distribution means" is a general term for a network and related hardware and software for transmitting the generated manual to a user's terminal and enabling the user to access it.

[0780] This invention relates to a system that collects user behavioral and emotional data, analyzes and learns behavioral patterns and emotional states based on this data, and automatically generates and updates effective manuals. This system makes it possible to improve work efficiency and introduce human-centered design.

[0781] The system consists of the following elements:

[0782] 1. Data Collection Methods

[0783] 2. Data preprocessing methods

[0784] 3. Learning methods for behavioral patterns and emotions (generative models)

[0785] 4. Manual Generation Methods

[0786] 5. Manual Distribution Methods

[0787] 6. Updates and Notifications

[0788] Data collection methods

[0789] Users go about their daily work and log in to the system's terminals to perform various tasks. Specifically, they log in to the system, use applications, enter data, check email, etc. The terminals record user operations in real time. For example, they collect operation data such as keyboard input, mouse operation, and application launch history, and record them along with timestamps. In addition, an emotion engine is used to recognize the user's emotional state from facial expression and voice data, and this information is also included in the behavioral data.

[0790] Data preprocessing measures

[0791] The server receives the raw data sent from the device. It preprocesses the received data separately for behavioral data and emotional data. This includes filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data and noise such as momentary emotional changes is removed. Data normalization converts data of different scales into a consistent range.

[0792] A means of learning behavioral patterns and emotions

[0793] The server inputs the preprocessed behavioral and emotional data into a generative model to learn the user's behavioral patterns and emotional state. Using machine learning algorithms, such as deep learning libraries like TensorFlow and PyTorch, correlations between the user's behavioral and emotional data can be analyzed. At this stage, the model identifies stress responses for specific tasks or times of day.

[0794] Manual Generation Method

[0795] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customizations based on the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed.

[0796] Manual distribution method

[0797] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[0798] Updates and Notifications

[0799] The server analyzes data in real time, and if any changes in behavioral patterns or emotional states are detected, it immediately updates the manual and notifies the user. The new manual is automatically distributed to the device.

[0800] Specific examples

[0801] For example, when User B, who works for a company, goes about his daily work, his device records behavioral and emotional data. User B starts entering data at 9:00 every day and checks his email at 10:00, but the data shows that he starts to feel stressed around 14:00 every day. This data is sent to the server in real time and pre-processed on the server.

[0802] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[0803] 1. Data entry begins at 9:00 every day.

[0804] Required tools: Datasheet, input software

[0805] Note: Double-check to avoid input errors.

[0806] 2. Check email at 10:00.

[0807] Tools used: Email client

[0808] Important Note: Prioritize important emails

[0809] 3. Take a short break every day at 2:00 PM.

[0810] The break time is 15 minutes

[0811] Note: It is recommended to rest in a relaxing environment.

[0812] This manual is delivered to User B's device, and User B can follow it to perform his / her work efficiently. In particular, taking a break during times when he / she feels stressed improves work efficiency and maintains his / her health.

[0813] Example prompts to input to the generative AI model

[0814] For example, the following prompt sentence is input to the generative AI model:

[0815] Please provide prompts to automatically generate optimal work procedures based on User B's behavioral patterns and emotional state when performing daily work at the company.

[0816] As a specific data-based procedure, User B starts entering data at 9:00 every morning and checks email at 10:00, but tends to feel stressed by 14:00.

[0817] Please generate a manual based on this.

[0818] In this way, the system of the present invention provides optimal work procedures that match the user's emotional state, thereby improving work efficiency and managing the user's health.

[0819] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0820] Step 1: Data collection

[0821] Users go about their daily work and log in to the system's terminals to perform various tasks, such as data entry, email checking, and report creation. The terminals monitor these operations in real time and collect operation data. Furthermore, the emotion engine uses the terminal's camera and microphone to analyze facial expressions and voice to collect emotional data. The input is the user's operation data and emotional state, and this data is recorded with a timestamp and a data package is generated to be sent to the server as output.

[0822] Step 2: Data Preprocessing

[0823] The server receives raw data sent from the device. The received data undergoes preprocessing such as filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data, and noise caused by momentary changes in emotions is removed. It also performs normalization to convert data of different scales into a consistent range. The input is the raw data sent from the device, and the output is the preprocessed, clean data.

[0824] Step 3: Learning behavioral patterns and emotions

[0825] The server inputs the preprocessed behavioral data and emotional data into a generative model to learn the user's behavioral patterns and emotional state. For example, it uses machine learning libraries such as Python's TensorFlow or PyTorch to analyze the correlation between the user's behavioral data and emotional data. The learning model identifies the user's stress response for specific tasks and time periods. The input is the preprocessed data, and the output is the trained generative model.

[0826] Step 4: Manual generation

[0827] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customized content according to the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed. The input is the trained generative model and real-time user data, and the output is the generated manual.

[0828] Step 5: Manual distribution

[0829] The server delivers the generated manual to the user's terminal. The user can perform their work efficiently through the latest manual. The generated manual is sent to the terminal, and the user proceeds with their work based on it. The input is the generated manual, and the output is the delivered manual.

[0830] Step 6: Updates and Notifications

[0831] The server analyzes new data in real time and immediately updates the manual if it detects changes in the user's behavioral patterns or emotional state. The updated manual is immediately notified to the user, and the new manual is automatically delivered to the device. The input is new user data and the existing generative model, and the output is the updated manual and its notification.

[0832] (Application example 2)

[0833] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0834] Conventional work manuals are generally rigid and designed without considering the behavioral data or emotional state of individual users, so they cannot be said to be effective in improving work efficiency within a company or managing employee health. In particular, there is a demand for systems that can flexibly respond to stress and irregular behavior that occur during work.

[0835] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, generative model means for analyzing the preprocessed behavioral data and learning behavioral patterns, emotion engine means for recognizing the user's emotional state, means for generating a manual based on the learned behavioral patterns and emotional state, and means for delivering the generated manual to the user's terminal. This makes it possible to provide a flexible and effective business manual that corresponds to the user's behavioral data and emotional state.

[0836] "Means for collecting user behavior data" refers to a device or software for collecting in real time operation logs, usage histories, and other behavioral information generated when a user operates the system.

[0837] "Means for preprocessing collected behavioral data" refers to a device or software that performs processes such as filling in missing data, normalizing data, and removing noise in order to convert the collected behavioral data into a format suitable for analysis.

[0838] The "generative model means for learning behavioral patterns" is a device or software for analyzing a user's behavioral patterns based on preprocessed behavioral data and learning specific behavioral patterns using a machine learning algorithm.

[0839] The "emotion engine means for recognizing the user's emotional state" is a device or software for analyzing the user's operation data, facial expressions, voice, etc. to recognize the user's emotional state.

[0840] The "means for generating a manual based on learned behavioral patterns and emotional states" refers to a device or software for automatically generating a manual including user-specific work procedures and points of caution based on learned behavioral patterns and emotional data.

[0841] The "means for distributing the generated manual to the user's terminal" refers to a device or software for providing the generated manual to the terminal used by the user.

[0842] This invention is a system that collects user behavioral data and emotional states, analyzes and learns from them, and automatically generates and updates flexible business manuals. A specific embodiment of this system is described below.

[0843] System configuration

[0844] Hardware configuration:

[0845] Data collection device: A smartphone or other device operated by the user.

[0846] Server: A central server that performs data preprocessing, training, manual generation, and distribution.

[0847] Software configuration:

[0848] Data collection module: collects user behavior data.

[0849] Pre-processing module: The collected data is imputed, denoised, and normalized.

[0850] Machine learning algorithms (e.g. scikit-learn, numpy): Generative models that learn behavioral patterns.

[0851] Emotion Engine: Recognize the user's emotional state (Example: EmotionEngine).

[0852] Manual generation module: Generates a manual based on learned behavioral patterns and emotion data.

[0853] Distribution module: distributes the generated manual to the user's terminal.

[0854] What the program does

[0855] 1. Data Collection Methods

[0856] The server monitors user operations in real time and collects behavioral data, including application usage history, operation logs, and other behavioral information, and simultaneously recognizes the user's emotional state using an emotion engine.

[0857] 2. Data preprocessing methods

[0858] The collected behavioral data is preprocessed on the server. The preprocessing module performs data filling, noise removal, and data normalization to convert the data into a format suitable for analysis and learning.

[0859] 3. Behavioral pattern learning method

[0860] The preprocessed data is then fed into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning technique that analyzes behavioral patterns and correlates them with emotional states.

[0861] 4. Emotional Engine Means

[0862] The emotion engine recognizes the user's emotional state and correlates it with behavioral patterns, for example, analyzing stress levels during specific tasks or times of day.

[0863] 5. Manual Generation Methods

[0864] The server generates a work manual based on the learned behavioral patterns and emotional states, which includes specific work procedures, points to note, and advice based on the emotional state.

[0865] 6. Manual Distribution Methods

[0866] The generated manual is automatically distributed from the server to the user's device, allowing the user to always perform their work based on the latest operational manual.

[0867] Specific examples

[0868] For example, when a user works in a physical store, the system collects the user's behavioral and emotional data. The data shows that the user puts products on the shelves at 9:00 every morning and rings up the cash register at 10:00. However, it is detected that the user begins to feel stressed around 2:00 pm. Based on this information, the system generates the following manual:

[0869] Example prompt sentence:

[0870] Based on User A's behavioral patterns and emotional data, you can see that he feels stressed at 2 PM. Based on this data, generate an optimal work manual. The manual should include work procedures and points to note depending on the user's emotional state.

[0871] This allows users to receive specific advice on how to reduce stress, along with the optimal work procedures based on their emotional state. This system can simultaneously improve work efficiency and manage employee health.

[0872] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0873] Step 1:

[0874] Data collection

[0875] The terminal collects operation logs, usage history, and other behavioral data generated when the user operates the system in real time. It also uses an emotion engine to simultaneously recognize and record the user's emotional state. Specifically, the terminal records the user's operation details (e.g., arranging products, ringing up cash registers, etc.) and the time of operation, and sends this data to the server. The input is the user's operation data and emotional state, and the output is the collected raw data.

[0876] Step 2:

[0877] Data Preprocessing

[0878] The server receives the collected behavioral data and performs preprocessing. The preprocessing module performs processes such as filling in missing data, removing noise, and normalizing the data, converting it into a format suitable for analysis and learning. For example, if the collected data contains missing data, it is filled in based on past data and outliers are removed. The input is the collected raw data, and the output is preprocessed, clean data.

[0879] Step 3:

[0880] Behavioral pattern learning

[0881] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning method that analyzes behavioral patterns and correlates them with emotional states. For example, the server learns what tasks the user is performing at a particular time and how their emotional state is changing. The input is the preprocessed data, and the output is the learned behavioral patterns.

[0882] Step 4:

[0883] Emotion Engine Analysis

[0884] The server uses an emotion engine to analyze the relationship between the user's emotional state and behavioral patterns. Specifically, it analyzes stress levels during specific tasks and time periods. The input is the collected emotion data, and the output is the emotion analysis results.

[0885] Step 5:

[0886] Manual Generation

[0887] The server generates a work manual based on the learned behavioral patterns and emotional state. The generated manual includes specific work procedures, points to note, and advice tailored to the employee's emotional state. For example, it suggests taking a break during times when employees are likely to feel stressed. The input is the learned behavioral patterns and the results of emotional analysis, and the output is the generated work manual.

[0888] Step 6:

[0889] Manual Distribution

[0890] The server distributes the generated manual to the user's terminal. The terminal receives it and displays it for easy reference by the user. If the manual needs to be updated, a notification is sent in real time. The input is the generated business manual, and the output is the distribution of the manual to the user's terminal and notification.

[0891] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0892] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0893] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0894] [Third embodiment]

[0895] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0896] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0897] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0898] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0899] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0900] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0901] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0902] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0903] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0904] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0905] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0906] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0907] This invention is a system that collects user behavior data, analyzes behavioral patterns based on that data, and automatically generates and updates effective manuals, allowing operators and employees to focus on their daily work and improving the efficiency and standardization of work.

[0908] System configuration

[0909] This system is broadly composed of the following elements:

[0910] 1. Data Collection Methods

[0911] 2. Data preprocessing methods

[0912] 3. Learning methods for behavioral patterns (generative models)

[0913] 4. Manual Generation Methods

[0914] 5. Manual Distribution Methods

[0915] 6. Updates and Notifications

[0916] Specific processing flow

[0917] 1. Data Collection

[0918] When a user performs work, work data on the terminal is recorded in real time, such as what time the user logged in to the system and how long each application was used.

[0919] The device collects this behavioral data and sends it to a server at regular intervals.

[0920] 2. Data Preprocessing

[0921] The server analyzes the collected raw data and performs data cleansing, a process that removes incomplete data and outliers and converts the data into a consistent format.

[0922] For example, if the time record for an operation is missing, a reasonable estimate is made from the data before and after.

[0923] 3. Learning behavioral patterns

[0924] The generative model on the server receives the preprocessed data as input and analyzes and learns from the user's behavioral patterns using machine learning and statistical methods.

[0925] The probability of certain patterns and irregular behavior is calculated and accumulated.

[0926] 4. Manual Generation

[0927] Based on the learned behavioral patterns, the server automatically generates a manual containing specific work procedures and points to note. This manual details standard procedures that users should refer to and how to deal with irregular situations.

[0928] For example, if there is a pattern where data entry starts at 9:00 every day, email is checked at 10:00, and lunch is taken at 12:00, the corresponding procedures will be reflected in the manual.

[0929] 5. Manual Distribution

[0930] The server automatically distributes the generated manual to the user's terminal via the network, allowing the user to always have the latest manual at hand.

[0931] 6. Updates and Notifications

[0932] The server analyzes data in real time and immediately updates the manual if any changes occur in behavioral patterns. This update information is notified to the user, and the new manual is automatically distributed to the terminal.

[0933] For example, if system maintenance is performed on Thursdays only during a particular week, that information will also be reflected promptly.

[0934] Specific examples

[0935] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative model.

[0936] Based on the behavioral patterns learned by the generative model, the following manual is automatically generated:

[0937] 1. Data entry begins at 9:00 every day.

[0938] Required tools: Datasheet, input software

[0939] Note: Double-check to avoid input errors.

[0940] 2. Check email at 10:00.

[0941] Tools used: Email client

[0942] Important Note: Prioritize important emails

[0943] 3. System maintenance will be performed every Friday.

[0944] Tools used: Maintenance software

[0945] Important note: Make a backup of your entire system beforehand.

[0946] This manual is distributed to User A's terminal, and User A follows it to perform his / her work, resulting in efficient and standardized work. Furthermore, if there is a change in the actual behavioral pattern (e.g., system maintenance is changed to Thursday), an updated manual is immediately distributed from the server, allowing the user to follow the latest procedures.

[0947] The processing flow will be explained below.

[0948] Step 1:

[0949] The user begins their daily work, specifically using the device to perform tasks such as logging in, launching applications, data entry, checking email, and attending meetings.

[0950] Step 2:

[0951] The device records real-time data about user operations and activities, including application usage time, task start and end times, and data entered.

[0952] Step 3:

[0953] The device transmits the collected data to the server at regular intervals. This data transmission is performed periodically, and the behavioral data is efficiently aggregated on the server via a communication protocol.

[0954] Step 4:

[0955] The server preprocesses the received behavioral data, such as filling in missing data, removing noise, and normalizing the data, to make it ready for analysis.

[0956] Step 5:

[0957] The server feeds the preprocessed data into a generative model, which uses machine learning algorithms to learn the user's behavioral patterns.

[0958] Step 6:

[0959] The generative model analyzes user behavior patterns and calculates the probability of repeated tasks and irregular behavior, thereby providing a more accurate understanding of work flows.

[0960] Step 7:

[0961] Based on the learned behavioral patterns, the server generates a manual detailing work procedures and precautions, including commonly performed tasks and procedures, as well as how to deal with irregular behavior.

[0962] Step 8:

[0963] The server distributes the generated manual to the user's terminal, where the user can check the latest manual and perform their work accordingly.

[0964] Step 9:

[0965] The server analyzes the behavioral data in real time, and if any changes in behavioral patterns are detected, the manual is updated immediately. This update information is notified to the user, and the new manual is automatically distributed to the device.

[0966] Step 10:

[0967] Users receive notifications and follow the updated manuals to carry out their work. By repeating this process, work is continuously standardized and made more efficient.

[0968] Example 1

[0969] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0970] Conventional work support systems have difficulty in properly collecting and analyzing user behavior data, making it impossible to automatically generate work instructions optimized for individual users. As a result, improvements in work efficiency and standardization are not fully achieved, and there is a lack of appropriate countermeasures, particularly for irregular behavior. Furthermore, real-time work management is difficult because work instructions are not updated or notified immediately.

[0971] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0972] In this invention, the server includes means for collecting user activity data, means for preprocessing the collected activity data, means for analyzing the preprocessed activity data and learning behavioral patterns, means for generating work instructions based on the learned behavioral patterns, and means for distributing the generated work instructions to the user's device. This makes it possible to efficiently collect and analyze user activity data and automatically generate and distribute work instructions optimized for each user. Furthermore, real-time work management is realized by calculating the probability of atypical behavior and instantly updating and notifying work instructions.

[0973] "User" refers to an individual or organization that uses this system and is the entity that carries out specific tasks.

[0974] "Activity data" is a series of digital information generated when a user performs work, and includes login time, application usage history, operation details, and the like.

[0975] "Preprocessing" refers to a series of steps that analyze the collected raw data, remove incomplete data and outliers, and convert the data into a certain format.

[0976] A "generative model" refers to an algorithm or program that uses machine learning or statistical techniques to learn user behavior patterns from preprocessed data.

[0977] "Work instructions" refers to documents containing work procedures and points to note that are generated based on learned behavioral patterns, and serve as guidelines for users to carry out their daily work in an efficient and standardized manner.

[0978] "Distribution" refers to the process of sending the generated work instructions to the user's device so that the user can view them.

[0979] "Atypical behavior" refers to irregular operations or actions that deviate from normal behavior patterns, and is reflected in the manual by calculating the probability of such behavior.

[0980] "Immediate update" refers to the process of quickly updating the contents of work instructions based on data collected in real time, and notifying the user of the results immediately.

[0981] This invention is a system that collects user activity data, analyzes behavioral patterns based on that data, and automatically generates and updates work instructions, allowing operators and employees to focus on their daily work and improving work efficiency and standardization.

[0982] System configuration

[0983] The system consists of the following main components:

[0984] 1. Data Collection Methods

[0985] When a user starts work, they log in to the system, and their login time and operation details are collected.

[0986] The device collects a series of user operation data in real time and sends it to the server at regular intervals, for example, every minute.

[0987] 2. Data preprocessing methods

[0988] The server performs an initial analysis of the raw data received and performs data cleansing, a process that removes incomplete data and outliers and converts all data into a unified format.

[0989] For example, the timestamp format is standardized to Coordinated Universal Time (UTC) and missing parts of the log are reasonably estimated and supplemented.

[0990] 3. Behavioral pattern learning method (generative AI model)

[0991] A generative AI model hosted on a server receives preprocessed data as input and analyzes and learns from user behavior patterns using machine learning algorithms and statistical methods.

[0992] To learn behavioral patterns, the probability and typical patterns of these activities are stored in the model based on past data.

[0993] 4. Manual Generation Methods

[0994] Based on the learned behavioral patterns, the server automatically generates work instructions, which detail standard work procedures and how to handle irregular situations.

[0995] For example, a pattern such as "Start data entry at 9:00 every day and check email at 10:00" is clearly stated.

[0996] 5. Manual Distribution Methods

[0997] The server automatically delivers the generated work instructions to the user's device, allowing the user to always have the latest work instructions at hand.

[0998] The terminal displays the received work instructions in an appropriate format for easy reference by the user.

[0999] 6. Updates and Notifications

[1000] The server analyzes the data collected in real time and immediately updates the work instructions if there is a change in the behavioral pattern. This updated information is notified and new work instructions are automatically sent to the terminal.

[1001] For example, if system maintenance is performed only on Thursdays of certain weeks, that change will also be reflected immediately.

[1002] Specific examples

[1003] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative AI model.

[1004] Based on the behavioral patterns learned by the generative model, the following work instructions are automatically generated:

[1005] markdown

[1006] 1. Data entry begins at 9:00 every day.

[1007] Required tools: Datasheet, input software

[1008] Note: Double-check to avoid input errors.

[1009] 2. Check email at 10:00.

[1010] Tools used: Email client

[1011] Important Note: Prioritize important emails

[1012] 3. System maintenance will be performed every Friday.

[1013] Tools used: Maintenance software

[1014] Important note: Make a backup of your entire system beforehand.

[1015] This work instruction is delivered to User A's terminal, and User A follows it to perform the work, resulting in efficient and standardized work. In addition, if there is a change in the actual behavior pattern (for example, if system maintenance is changed to Thursday), the server immediately delivers updated work instructions, allowing the user to follow the latest procedures.

[1016] Prompt Sentence Examples

[1017] An example of a prompt to input to a generative AI model is as follows:

[1018] Please tell me the specific steps to take when entering data.

[1019] What should I pay attention to when checking my email?

[1020] "What is your standard procedure for system maintenance?"

[1021] keyword

[1022] Generative AI Models

[1023] Prompt statement

[1024] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1025] Step 1:

[1026] Data collection

[1027] When a user starts work, he logs in to the system, and the login time is recorded on the terminal.

[1028] For each application a user uses during work, its start and end times are collected in real time.

[1029] The terminal sends the collected data to the server in batch processing every minute, allowing operation logs on the terminal to be accumulated in real time.

[1030] Input: User operation data (e.g., login time, application usage time).

[1031] Output: The collected operation data is sent to the server.

[1032] Step 2:

[1033] Data Preprocessing

[1034] The server analyzes the raw data it receives and performs data cleansing, which includes removing incomplete data and outliers.

[1035] The server standardizes the timestamp format (e.g., converts it to Coordinated Universal Time (UTC)) and fills in missing data with estimated values.

[1036] Specific operation of data cleansing: If there is a missing login time, the average value is calculated from the data before and after the missing time to fill in the gaps.

[1037] Input: Collected operational data.

[1038] Output: The cleansed data.

[1039] Step 3:

[1040] Learning behavioral patterns

[1041] A generative AI model on a server receives the preprocessed data as input and begins analyzing the data using machine learning algorithms (e.g., recurrent neural networks).

[1042] The probability of certain patterns or abnormal behavior is calculated, and these models are gradually refined.

[1043] Specific operation: The generative AI model learns the average start time and frequency of a user's actions and stores the results in a database.

[1044] Input: Preprocessed data.

[1045] Output: Learned behavioral patterns (model).

[1046] Step 4:

[1047] Manual Generation

[1048] The server automatically generates work instructions based on the learned behavioral patterns, including routine work procedures and countermeasures for irregular situations.

[1049] Example: A standardized procedure such as "Start data entry at 9:00 and check email at 10:00 every day" is described.

[1050] Input: Learned behavioral patterns.

[1051] Output: The generated work order.

[1052] Step 5:

[1053] Manual Distribution

[1054] The server automatically delivers the generated work instructions to the user's device, allowing the user to always refer to the latest work instructions.

[1055] The terminal displays the received work instructions in an appropriate format for easy access by the user.

[1056] Notification function: Displays when a new manual has arrived on your device.

[1057] Input: Generated work order.

[1058] Output: Work instructions delivered to the terminal.

[1059] Step 6:

[1060] Updates and Notifications

[1061] The server analyzes the data collected in real time and immediately updates the work instructions if any changes occur in the behavioral patterns. The new work instructions are then delivered to the user's device along with the updated information.

[1062] Example: If the system maintenance schedule is changed, users will be notified immediately.

[1063] Input: Real-time behavioral data.

[1064] Output: Updated work order and notification.

[1065] (Application example 1)

[1066] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1067] There are already systems that collect user behavior data and analyze behavioral patterns based on that data to improve business efficiency and standardize operations. However, these systems lack real-time update capabilities or the ability to specifically control device operation, resulting in a lack of responsiveness and adaptability to business operations. Furthermore, they lack the ability to automatically generate manuals that reflect irregular behavior, limiting their ability to maximize user work efficiency. The purpose of this invention is to solve these problems and achieve higher levels of business efficiency and adaptability.

[1068] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1069] In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, means for analyzing the preprocessed behavioral data and learning behavioral patterns, means for generating a manual based on the learned behavioral patterns, means for distributing the generated manual to the user's device, means for controlling the operation of the device according to the set pattern, means for instantly updating the manual to the latest version based on real-time behavioral data, and means for notifying the device of the contents of the updated manual. This enables real-time business optimization and standardization based on the user's business data.

[1070] "Behavioral data" refers to the history and log information of specific operations and actions performed by a user.

[1071] "Preprocessing" is the process of converting raw data into a form suitable for analysis, including data cleansing and format conversion.

[1072] A "generative model" is a machine learning algorithm or statistical model that learns behavioral patterns based on preprocessed data.

[1073] A "manual" is a document or guideline that describes procedures and instructions for users to perform their work efficiently.

[1074] "Devices" refer to hardware devices such as terminals and robots used by users.

[1075] "Control" refers to the operation of directing the behavior or state of equipment, and is the process of managing behavior according to specific patterns.

[1076] "Real-time update" is the process of instantly updating the manual based on the latest behavioral data and delivering that information to the user's device.

[1077] "Irregular behavior" refers to unusual actions or operations that deviate from normal patterns of behavior, and procedures for dealing with such behavior are reflected in the manual.

[1078] This invention is a system that automatically generates and updates business manuals by collecting user behavior data and analyzing behavioral patterns based on the collected data. An embodiment of this system will be described in detail below.

[1079] System Configuration

[1080] The system mainly consists of the following components:

[1081] 1. How to collect user behavior data

[1082] 2. A means of preprocessing the collected behavioral data

[1083] 3. A generative modeling method that analyzes preprocessed behavioral data and learns behavioral patterns

[1084] 4. A method for generating manuals based on learned behavioral patterns

[1085] 5. Means for delivering the generated manual to the user's device

[1086] 6. Means of controlling the operation of equipment according to set patterns

[1087] 7. A way to instantly update manuals based on real-time behavioral data

[1088] 8. Means of notifying the device of updated manual contents

[1089] Operation overview

[1090] First, as users perform their work, sensors and cameras on devices (e.g., terminals and factory robots) collect behavioral data in real time. This data includes operating hours, tools used, movement patterns, and more. The collected data is sent to a server where preprocessing is performed. This involves filling in missing data and removing outliers. Next, a generative model learns behavioral patterns based on the preprocessed data. A machine learning algorithm is used for this purpose.

[1091] Based on the learned behavioral patterns, the system generates a work manual that includes specific operating procedures, tools to use, and important points to note. For example, the following manual may be generated:

[1092] 1. Starts operation at 8:00 every day.

[1093] Tools used: Gripper, drill.

[1094] Important note: Perform initial calibration of the gripper.

[1095] 2. Replace part A at 12:00.

[1096] Tools used: Torque wrench.

[1097] Note: Check the torque value.

[1098] 3. Full maintenance every Friday.

[1099] Tools used: Various tools.

[1100] Important note: Make a full system backup.

[1101] The generated manual is automatically distributed to the user's device and used to appropriately control the device's operation. Furthermore, the manual is instantly updated in response to changes in behavior patterns in real time. This update information is notified to the device, so the latest procedures are always reflected.

[1102] Hardware and software used

[1103] Collection devices: sensors, cameras, terminals, factory robots

[1104] Data preprocessing: Pandas (Python library)

[1105] Generative models: Random Forest (machine learning algorithm), other AI models

[1106] Data analysis server: High performance server

[1107] Distribution System: Network Communication

[1108] Prompt Sentence Examples

[1109] An example prompt based on a generative AI model is:

[1110] Based on the user's behavioral data, generate a robot operation manual like the one below.

[1111] 1. Starts operation at 8:00 every day.

[1112] Tools used: Gripper, drill.

[1113] Important note: Perform initial calibration of the gripper.

[1114] 2. Replace part A at 12:00.

[1115] Tools used: Torque wrench.

[1116] Note: Check the torque value.

[1117] 3. Full maintenance is performed automatically every Tuesday.

[1118] Tools used: Various tools.

[1119] Important note: Make a full system backup.

[1120] The above describes a specific embodiment of the invention. By using this system, it is possible to achieve greater efficiency and standardization of work, thereby maximizing the user's work performance.

[1121] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1122] Step 1:

[1123] The server collects user behavior data. This input data includes user uptime, tools and applications used, and movement patterns. This data is collected in real time from sensors, cameras, and devices.

[1124] Step 2:

[1125] The server preprocesses the collected behavioral data. This preprocessing includes filling in missing data, removing outliers, and converting the data into a format suitable for analysis. Raw data is used as input, and cleansed data is output after processing. Data processing libraries such as Pandas are used.

[1126] Step 3:

[1127] The server supplies the preprocessed data as input to the generative model. The generative model uses machine learning algorithms such as Random Forest to learn behavioral patterns. The input data is preprocessed behavioral data, and the output is learned behavioral patterns. The model runs on the server and performs analysis.

[1128] Step 4:

[1129] The server generates a manual based on the learned behavioral patterns. Specifically, a manual is automatically generated that describes the user's work procedures, tools to be used, and points to note. In this process, the learned behavioral patterns are used as input data, and the work manual is output.

[1130] Step 5:

[1131] The server distributes the generated manual to the user's device. The manual is automatically sent to the device via network communication. The input data is the generated manual, and the output is the state that has been distributed to the user's device.

[1132] Step 6:

[1133] The server controls the operation of the equipment according to the set patterns. Specifically, it manages the equipment so that it follows the manual and performs its tasks efficiently and accurately. The input data is the distributed manual, and the output is the actual operation control of the equipment.

[1134] Step 7:

[1135] The server instantly updates the manual to the latest version based on real-time behavioral data. It continuously collects user behavioral data and immediately updates the manual if there are any changes. The input data is the latest behavioral data, and the output is an updated manual.

[1136] Step 8:

[1137] The server notifies the user's device of the contents of the updated manual, allowing the user to always perform their work based on the latest procedures. The input data is the updated manual, and the output is a notification sent to the user's device.

[1138] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1139] This invention is a system that collects user behavior data, analyzes and learns behavioral patterns based on this data, and automatically generates and updates effective manuals. Furthermore, by combining it with an emotion engine that recognizes the user's emotional state, it realizes flexible responses based on the user's emotions. This system makes it possible to improve work efficiency and introduce human-centered design.

[1140] System configuration

[1141] The system consists of the following components:

[1142] 1. Data Collection Methods

[1143] 2. Data preprocessing methods

[1144] 3. Learning methods for behavioral patterns (generative models)

[1145] 4. Emotion Engine

[1146] 5. Manual Generation Methods

[1147] 6. Manual Distribution Methods

[1148] 7. Updates and Notifications

[1149] Specific processing flow

[1150] 1. Data Collection

[1151] Users begin their daily work and perform tasks on the device, such as logging in to the system, using applications, entering data, and checking email.

[1152] The device records the user's operation data in real time and periodically transmits the data to the server. In addition, the emotion engine recognizes the user's emotional state, which is also included in the behavioral data.

[1153] 2. Data Preprocessing

[1154] The server preprocesses the received data, filling in missing data, removing noise, and normalizing the data. The server also preprocesses the user's emotion data.

[1155] 3. Learning behavioral patterns and emotions

[1156] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns and emotional state. Specifically, it uses machine learning algorithms to analyze the correlation between user behavior and emotions.

[1157] For example, the model learns which specific tasks or times of day users find stressful.

[1158] 4. Manual Generation

[1159] Based on the learned behavioral patterns and emotional data, the server automatically generates a manual detailing work procedures and important points to note. This manual includes not only standard work procedures but also customizations based on the user's emotional state.

[1160] For example, if the user is feeling stressed, it may suggest taking a short break.

[1161] 5. Manual Distribution

[1162] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[1163] 6. Updates and Notifications

[1164] The server analyzes the data in real time and immediately updates the manual if any changes in behavioral patterns or emotional states are detected. This update information is notified to the user, and the new manual is automatically distributed to the device.

[1165] Specific examples

[1166] User B, who works for a certain company, records behavioral and emotional data on his device as he goes about his daily work. User B starts entering data at 9:00 and checks his email at 10:00 every day. However, the data shows that he begins to feel stressed around 14:00 every day. This data is sent to a server in real time and pre-processed on the server.

[1167] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[1168] 1. Data entry begins at 9:00 every day.

[1169] Required tools: Datasheet, input software

[1170] Note: Double-check to avoid input errors.

[1171] 2. Check email at 10:00.

[1172] Tools used: Email client

[1173] Important Note: Prioritize important emails

[1174] 3. Take a short break every day at 2:00 PM.

[1175] The break time is 15 minutes

[1176] Note: It is recommended to rest in a relaxing environment.

[1177] This manual is distributed to User B's device, and User B can follow it to perform his / her work efficiently. Taking breaks during times of stress in particular improves work efficiency and maintains health. The program automates the entire process, from data collection to manual generation and updating, reducing the user's workload.

[1178] This system allows users to perform tasks using procedures that are optimal for their emotional state, which not only improves work efficiency and standardizes work, but also contributes to health management.

[1179] The processing flow will be explained below.

[1180] Step 1:

[1181] A user begins their daily routine, for example, logging into a terminal and beginning a data entry task.

[1182] Step 2:

[1183] The device records user operation data in real time, including the start time of data entry, the application used, and the progress of the work.

[1184] Step 3:

[1185] The emotion engine recognizes the user's emotional state using biometric data such as facial expressions, voice, and heart rate, and is supported by cameras, microphones, and wearable devices attached to the device.

[1186] Step 4:

[1187] The device transmits the recorded behavioral and emotional data to a server at regular intervals, where the data is efficiently collected via a communication protocol.

[1188] Step 5:

[1189] The server preprocesses the received data, such as filling in missing data, removing noise, and normalizing the data. The same preprocessing is performed on emotion data.

[1190] Step 6:

[1191] The server inputs the preprocessed data into a generative model, which uses machine learning algorithms to analyze and learn correlations between user behavior patterns and emotions.

[1192] Step 7:

[1193] The generative model calculates the probability of certain patterns or irregular behavior based on the user's behavioral patterns and emotional data. For example, it determines that a user is likely to feel stressed at a certain time of day.

[1194] Step 8:

[1195] The server generates a manual based on the learned behavioral patterns and emotional data, which includes commonly performed tasks and procedures, as well as customization according to the user's emotional state.

[1196] Step 9:

[1197] The server delivers the generated manual to the user's terminal, where the user can check the latest manual and proceed with their work accordingly.

[1198] Step 10:

[1199] The server continues to analyze data in real time, and if any changes in behavioral patterns or emotional states are detected, the manual is immediately updated. This updated information is notified to the user, and the new manual is automatically distributed to the device.

[1200] Step 11:

[1201] Users receive notifications and follow the updated manual to carry out their work, which not only improves work efficiency and standardization but also provides a work environment that takes into account the user's emotional state.

[1202] Example 2

[1203] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1204] Conventional systems only collect user behavioral data and do not consider emotional states, which means they are unable to improve work efficiency or adequately manage the user's health. In particular, they are unable to grasp when a user feels stressed or their emotional state during a specific task, making it difficult to provide appropriate manuals. As a result, users' work efficiency declines and their mental burden increases.

[1205] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1206] In this invention, the server includes means for collecting user behavioral data and emotional data, means for preprocessing the collected behavioral data and emotional data, and generative model means for analyzing the preprocessed behavioral data and emotional data and learning behavioral patterns and emotional states. This makes it possible to generate and distribute optimal manuals that take into account not only the user's behavior but also their emotional states.

[1207] A "user" is a person who operates the system and provides behavioral and emotional data.

[1208] "Behavioral data" is information related to a user's terminal operations, and includes specific operation content, operation timing, applications used, input data, and the like.

[1209] "Emotion data" is information that represents the user's emotional state, and includes psychological states such as stress, joy, anger, and sadness that can be obtained from facial expressions and voice.

[1210] "Means for collection" is a general term for hardware and software for recording user behavioral data and emotional data in real time and transmitting it to a server.

[1211] "Preprocessing means" is a general term for the process of filling in missing data, removing noise, normalizing data, etc. on collected raw data, as well as the technical equipment and software used for this process.

[1212] "Generative model means" is a general term for a machine learning model that learns a user's behavioral patterns and emotional states using preprocessed behavioral data and emotional data, and the system that operates it.

[1213] "Means for generating manuals" is a general term for algorithms and systems that automatically create manuals that list optimal work procedures and important points based on learned behavioral patterns and emotional states.

[1214] "Distribution means" is a general term for a network and related hardware and software for transmitting the generated manual to a user's terminal and enabling the user to access it.

[1215] This invention relates to a system that collects user behavioral and emotional data, analyzes and learns behavioral patterns and emotional states based on this data, and automatically generates and updates effective manuals. This system makes it possible to improve work efficiency and introduce human-centered design.

[1216] The system consists of the following elements:

[1217] 1. Data Collection Methods

[1218] 2. Data preprocessing methods

[1219] 3. Learning methods for behavioral patterns and emotions (generative models)

[1220] 4. Manual Generation Methods

[1221] 5. Manual Distribution Methods

[1222] 6. Updates and Notifications

[1223] Data collection methods

[1224] Users go about their daily work and log in to the system's terminals to perform various tasks. Specifically, they log in to the system, use applications, enter data, check email, etc. The terminals record user operations in real time. For example, they collect operation data such as keyboard input, mouse operation, and application launch history, and record them along with timestamps. In addition, an emotion engine is used to recognize the user's emotional state from facial expression and voice data, and this information is also included in the behavioral data.

[1225] Data preprocessing measures

[1226] The server receives the raw data sent from the device. It preprocesses the received data separately for behavioral data and emotional data. This includes filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data and noise such as momentary emotional changes is removed. Data normalization converts data of different scales into a consistent range.

[1227] A means of learning behavioral patterns and emotions

[1228] The server inputs the preprocessed behavioral and emotional data into a generative model to learn the user's behavioral patterns and emotional state. Using machine learning algorithms, such as deep learning libraries like TensorFlow and PyTorch, correlations between the user's behavioral and emotional data can be analyzed. At this stage, the model identifies stress responses for specific tasks or times of day.

[1229] Manual Generation Method

[1230] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customizations based on the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed.

[1231] Manual distribution method

[1232] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[1233] Updates and Notifications

[1234] The server analyzes data in real time, and if any changes in behavioral patterns or emotional states are detected, it immediately updates the manual and notifies the user. The new manual is automatically distributed to the device.

[1235] Specific examples

[1236] For example, when User B, who works for a company, goes about his daily work, his device records behavioral and emotional data. User B starts entering data at 9:00 every day and checks his email at 10:00, but the data shows that he starts to feel stressed around 14:00 every day. This data is sent to the server in real time and pre-processed on the server.

[1237] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[1238] 1. Data entry begins at 9:00 every day.

[1239] Required tools: Datasheet, input software

[1240] Note: Double-check to avoid input errors.

[1241] 2. Check email at 10:00.

[1242] Tools used: Email client

[1243] Important Note: Prioritize important emails

[1244] 3. Take a short break every day at 2:00 PM.

[1245] The break time is 15 minutes

[1246] Note: It is recommended to rest in a relaxing environment.

[1247] This manual is delivered to User B's device, and User B can follow it to perform his / her work efficiently. In particular, taking a break during times when he / she feels stressed improves work efficiency and maintains his / her health.

[1248] Example prompts to input to the generative AI model

[1249] For example, the following prompt sentence is input to the generative AI model:

[1250] Please provide prompts to automatically generate optimal work procedures based on User B's behavioral patterns and emotional state when performing daily work at the company.

[1251] As a specific data-based procedure, User B starts entering data at 9:00 every morning and checks email at 10:00, but tends to feel stressed by 14:00.

[1252] Please generate a manual based on this.

[1253] In this way, the system of the present invention provides optimal work procedures that match the user's emotional state, thereby improving work efficiency and managing the user's health.

[1254] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1255] Step 1: Data collection

[1256] Users go about their daily work and log in to the system's terminals to perform various tasks, such as data entry, email checking, and report creation. The terminals monitor these operations in real time and collect operation data. Furthermore, the emotion engine uses the terminal's camera and microphone to analyze facial expressions and voice to collect emotional data. The input is the user's operation data and emotional state, and this data is recorded with a timestamp and a data package is generated to be sent to the server as output.

[1257] Step 2: Data Preprocessing

[1258] The server receives raw data sent from the device. The received data undergoes preprocessing such as filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data, and noise caused by momentary changes in emotions is removed. It also performs normalization to convert data of different scales into a consistent range. The input is the raw data sent from the device, and the output is the preprocessed, clean data.

[1259] Step 3: Learning behavioral patterns and emotions

[1260] The server inputs the preprocessed behavioral data and emotional data into a generative model to learn the user's behavioral patterns and emotional state. For example, it uses machine learning libraries such as Python's TensorFlow or PyTorch to analyze the correlation between the user's behavioral data and emotional data. The learning model identifies the user's stress response for specific tasks and time periods. The input is the preprocessed data, and the output is the trained generative model.

[1261] Step 4: Manual generation

[1262] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customized content according to the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed. The input is the trained generative model and real-time user data, and the output is the generated manual.

[1263] Step 5: Manual distribution

[1264] The server delivers the generated manual to the user's terminal. The user can perform their work efficiently through the latest manual. The generated manual is sent to the terminal, and the user proceeds with their work based on it. The input is the generated manual, and the output is the delivered manual.

[1265] Step 6: Updates and Notifications

[1266] The server analyzes new data in real time and immediately updates the manual if it detects changes in the user's behavioral patterns or emotional state. The updated manual is immediately notified to the user, and the new manual is automatically delivered to the device. The input is new user data and the existing generative model, and the output is the updated manual and its notification.

[1267] (Application example 2)

[1268] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1269] Conventional work manuals are generally rigid and designed without considering the behavioral data or emotional state of individual users, so they cannot be said to be effective in improving work efficiency within a company or managing employee health. In particular, there is a demand for systems that can flexibly respond to stress and irregular behavior that occur during work.

[1270] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, generative model means for analyzing the preprocessed behavioral data and learning behavioral patterns, emotion engine means for recognizing the user's emotional state, means for generating a manual based on the learned behavioral patterns and emotional state, and means for delivering the generated manual to the user's terminal. This makes it possible to provide a flexible and effective business manual that corresponds to the user's behavioral data and emotional state.

[1271] "Means for collecting user behavior data" refers to a device or software for collecting in real time operation logs, usage histories, and other behavioral information generated when a user operates the system.

[1272] "Means for preprocessing collected behavioral data" refers to a device or software that performs processes such as filling in missing data, normalizing data, and removing noise in order to convert the collected behavioral data into a format suitable for analysis.

[1273] The "generative model means for learning behavioral patterns" is a device or software for analyzing a user's behavioral patterns based on preprocessed behavioral data and learning specific behavioral patterns using a machine learning algorithm.

[1274] The "emotion engine means for recognizing the user's emotional state" is a device or software for analyzing the user's operation data, facial expressions, voice, etc. to recognize the user's emotional state.

[1275] The "means for generating a manual based on learned behavioral patterns and emotional states" refers to a device or software for automatically generating a manual including user-specific work procedures and points of caution based on learned behavioral patterns and emotional data.

[1276] The "means for distributing the generated manual to the user's terminal" refers to a device or software for providing the generated manual to the terminal used by the user.

[1277] This invention is a system that collects user behavioral data and emotional states, analyzes and learns from them, and automatically generates and updates flexible business manuals. A specific embodiment of this system is described below.

[1278] System configuration

[1279] Hardware configuration:

[1280] Data collection device: A smartphone or other device operated by the user.

[1281] Server: A central server that performs data preprocessing, training, manual generation, and distribution.

[1282] Software configuration:

[1283] Data collection module: collects user behavior data.

[1284] Pre-processing module: The collected data is imputed, denoised, and normalized.

[1285] Machine learning algorithms (e.g. scikit-learn, numpy): Generative models that learn behavioral patterns.

[1286] Emotion Engine: Recognize the user's emotional state (Example: EmotionEngine).

[1287] Manual generation module: Generates a manual based on learned behavioral patterns and emotion data.

[1288] Distribution module: distributes the generated manual to the user's terminal.

[1289] What the program does

[1290] 1. Data Collection Methods

[1291] The server monitors user operations in real time and collects behavioral data, including application usage history, operation logs, and other behavioral information, and simultaneously recognizes the user's emotional state using an emotion engine.

[1292] 2. Data preprocessing methods

[1293] The collected behavioral data is preprocessed on the server. The preprocessing module performs data filling, noise removal, and data normalization to convert the data into a format suitable for analysis and learning.

[1294] 3. Behavioral pattern learning method

[1295] The preprocessed data is then fed into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning technique that analyzes behavioral patterns and correlates them with emotional states.

[1296] 4. Emotional Engine Means

[1297] The emotion engine recognizes the user's emotional state and correlates it with behavioral patterns, for example, analyzing stress levels during specific tasks or times of day.

[1298] 5. Manual Generation Methods

[1299] The server generates a work manual based on the learned behavioral patterns and emotional states, which includes specific work procedures, points to note, and advice based on the emotional state.

[1300] 6. Manual Distribution Methods

[1301] The generated manual is automatically distributed from the server to the user's device, allowing the user to always perform their work based on the latest operational manual.

[1302] Specific examples

[1303] For example, when a user works in a physical store, the system collects the user's behavioral and emotional data. The data shows that the user puts products on the shelves at 9:00 every morning and rings up the cash register at 10:00. However, it is detected that the user begins to feel stressed around 2:00 pm. Based on this information, the system generates the following manual:

[1304] Example prompt sentence:

[1305] Based on User A's behavioral patterns and emotional data, you can see that he feels stressed at 2 PM. Based on this data, generate an optimal work manual. The manual should include work procedures and points to note depending on the user's emotional state.

[1306] This allows users to receive specific advice on how to reduce stress, along with the optimal work procedures based on their emotional state. This system can simultaneously improve work efficiency and manage employee health.

[1307] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1308] Step 1:

[1309] Data collection

[1310] The terminal collects operation logs, usage history, and other behavioral data generated when the user operates the system in real time. It also uses an emotion engine to simultaneously recognize and record the user's emotional state. Specifically, the terminal records the user's operation details (e.g., arranging products, ringing up cash registers, etc.) and the time of operation, and sends this data to the server. The input is the user's operation data and emotional state, and the output is the collected raw data.

[1311] Step 2:

[1312] Data Preprocessing

[1313] The server receives the collected behavioral data and performs preprocessing. The preprocessing module performs processes such as filling in missing data, removing noise, and normalizing the data, converting it into a format suitable for analysis and learning. For example, if the collected data contains missing data, it is filled in based on past data and outliers are removed. The input is the collected raw data, and the output is preprocessed, clean data.

[1314] Step 3:

[1315] Behavioral pattern learning

[1316] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning method that analyzes behavioral patterns and correlates them with emotional states. For example, the server learns what tasks the user is performing at a particular time and how their emotional state is changing. The input is the preprocessed data, and the output is the learned behavioral patterns.

[1317] Step 4:

[1318] Emotion Engine Analysis

[1319] The server uses an emotion engine to analyze the relationship between the user's emotional state and behavioral patterns. Specifically, it analyzes stress levels during specific tasks and time periods. The input is the collected emotion data, and the output is the emotion analysis results.

[1320] Step 5:

[1321] Manual Generation

[1322] The server generates a work manual based on the learned behavioral patterns and emotional state. The generated manual includes specific work procedures, points to note, and advice tailored to the employee's emotional state. For example, it suggests taking a break during times when employees are likely to feel stressed. The input is the learned behavioral patterns and the results of emotional analysis, and the output is the generated work manual.

[1323] Step 6:

[1324] Manual Distribution

[1325] The server distributes the generated manual to the user's terminal. The terminal receives it and displays it for easy reference by the user. If the manual needs to be updated, a notification is sent in real time. The input is the generated business manual, and the output is the distribution of the manual to the user's terminal and notification.

[1326] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1327] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1328] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1329] [Fourth embodiment]

[1330] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1331] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1332] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1333] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1334] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1335] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1336] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1337] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1338] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1339] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1340] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1341] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1342] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1343] This invention is a system that collects user behavior data, analyzes behavioral patterns based on that data, and automatically generates and updates effective manuals, allowing operators and employees to focus on their daily work and improving the efficiency and standardization of work.

[1344] System configuration

[1345] This system is broadly composed of the following elements:

[1346] 1. Data Collection Methods

[1347] 2. Data preprocessing methods

[1348] 3. Learning methods for behavioral patterns (generative models)

[1349] 4. Manual Generation Methods

[1350] 5. Manual Distribution Methods

[1351] 6. Updates and Notifications

[1352] Specific processing flow

[1353] 1. Data Collection

[1354] When a user performs work, work data on the terminal is recorded in real time, such as what time the user logged in to the system and how long each application was used.

[1355] The device collects this behavioral data and sends it to a server at regular intervals.

[1356] 2. Data Preprocessing

[1357] The server analyzes the collected raw data and performs data cleansing, a process that removes incomplete data and outliers and converts the data into a consistent format.

[1358] For example, if the time record for an operation is missing, a reasonable estimate is made from the data before and after.

[1359] 3. Learning behavioral patterns

[1360] The generative model on the server receives the preprocessed data as input and analyzes and learns from the user's behavioral patterns using machine learning and statistical methods.

[1361] The probability of certain patterns and irregular behavior is calculated and accumulated.

[1362] 4. Manual Generation

[1363] Based on the learned behavioral patterns, the server automatically generates a manual containing specific work procedures and points to note. This manual details standard procedures that users should refer to and how to deal with irregular situations.

[1364] For example, if there is a pattern where data entry starts at 9:00 every day, email is checked at 10:00, and lunch is taken at 12:00, the corresponding procedures will be reflected in the manual.

[1365] 5. Manual Distribution

[1366] The server automatically distributes the generated manual to the user's terminal via the network, allowing the user to always have the latest manual at hand.

[1367] 6. Updates and Notifications

[1368] The server analyzes data in real time and immediately updates the manual if any changes occur in behavioral patterns. This update information is notified to the user, and the new manual is automatically distributed to the terminal.

[1369] For example, if system maintenance is performed on Thursdays only during a particular week, that information will also be reflected promptly.

[1370] Specific examples

[1371] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative model.

[1372] Based on the behavioral patterns learned by the generative model, the following manual is automatically generated:

[1373] 1. Data entry begins at 9:00 every day.

[1374] Required tools: Datasheet, input software

[1375] Note: Double-check to avoid input errors.

[1376] 2. Check email at 10:00.

[1377] Tools used: Email client

[1378] Important Note: Prioritize important emails

[1379] 3. System maintenance will be performed every Friday.

[1380] Tools used: Maintenance software

[1381] Important note: Make a backup of your entire system beforehand.

[1382] This manual is distributed to User A's terminal, and User A follows it to perform his / her work, resulting in efficient and standardized work. Furthermore, if there is a change in the actual behavioral pattern (e.g., system maintenance is changed to Thursday), an updated manual is immediately distributed from the server, allowing the user to follow the latest procedures.

[1383] The processing flow will be explained below.

[1384] Step 1:

[1385] The user begins their daily work, specifically using the device to perform tasks such as logging in, launching applications, data entry, checking email, and attending meetings.

[1386] Step 2:

[1387] The device records real-time data about user operations and activities, including application usage time, task start and end times, and data entered.

[1388] Step 3:

[1389] The device transmits the collected data to the server at regular intervals. This data transmission is performed periodically, and the behavioral data is efficiently aggregated on the server via a communication protocol.

[1390] Step 4:

[1391] The server preprocesses the received behavioral data, such as filling in missing data, removing noise, and normalizing the data, to make it ready for analysis.

[1392] Step 5:

[1393] The server feeds the preprocessed data into a generative model, which uses machine learning algorithms to learn the user's behavioral patterns.

[1394] Step 6:

[1395] The generative model analyzes user behavior patterns and calculates the probability of repeated tasks and irregular behavior, thereby providing a more accurate understanding of work flows.

[1396] Step 7:

[1397] Based on the learned behavioral patterns, the server generates a manual detailing work procedures and precautions, including commonly performed tasks and procedures, as well as how to deal with irregular behavior.

[1398] Step 8:

[1399] The server distributes the generated manual to the user's terminal, where the user can check the latest manual and perform their work accordingly.

[1400] Step 9:

[1401] The server analyzes the behavioral data in real time, and if any changes in behavioral patterns are detected, the manual is updated immediately. This update information is notified to the user, and the new manual is automatically distributed to the device.

[1402] Step 10:

[1403] Users receive notifications and follow the updated manuals to carry out their work. By repeating this process, work is continuously standardized and made more efficient.

[1404] Example 1

[1405] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1406] Conventional work support systems have difficulty in properly collecting and analyzing user behavior data, making it impossible to automatically generate work instructions optimized for individual users. As a result, improvements in work efficiency and standardization are not fully achieved, and there is a lack of appropriate countermeasures, particularly for irregular behavior. Furthermore, real-time work management is difficult because work instructions are not updated or notified immediately.

[1407] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1408] In this invention, the server includes means for collecting user activity data, means for preprocessing the collected activity data, means for analyzing the preprocessed activity data and learning behavioral patterns, means for generating work instructions based on the learned behavioral patterns, and means for distributing the generated work instructions to the user's device. This makes it possible to efficiently collect and analyze user activity data and automatically generate and distribute work instructions optimized for each user. Furthermore, real-time work management is realized by calculating the probability of atypical behavior and instantly updating and notifying work instructions.

[1409] "User" refers to an individual or organization that uses this system and is the entity that carries out specific tasks.

[1410] "Activity data" is a series of digital information generated when a user performs work, and includes login time, application usage history, operation details, and the like.

[1411] "Preprocessing" refers to a series of steps that analyze the collected raw data, remove incomplete data and outliers, and convert the data into a certain format.

[1412] A "generative model" refers to an algorithm or program that uses machine learning or statistical techniques to learn user behavior patterns from preprocessed data.

[1413] "Work instructions" refers to documents containing work procedures and points to note that are generated based on learned behavioral patterns, and serve as guidelines for users to carry out their daily work in an efficient and standardized manner.

[1414] "Distribution" refers to the process of sending the generated work instructions to the user's device so that the user can view them.

[1415] "Atypical behavior" refers to irregular operations or actions that deviate from normal behavior patterns, and is reflected in the manual by calculating the probability of such behavior.

[1416] "Immediate update" refers to the process of quickly updating the contents of work instructions based on data collected in real time, and notifying the user of the results immediately.

[1417] This invention is a system that collects user activity data, analyzes behavioral patterns based on that data, and automatically generates and updates work instructions, allowing operators and employees to focus on their daily work and improving work efficiency and standardization.

[1418] System configuration

[1419] The system consists of the following main components:

[1420] 1. Data Collection Methods

[1421] When a user starts work, they log in to the system, and their login time and operation details are collected.

[1422] The device collects a series of user operation data in real time and sends it to the server at regular intervals, for example, every minute.

[1423] 2. Data preprocessing methods

[1424] The server performs an initial analysis of the raw data received and performs data cleansing, a process that removes incomplete data and outliers and converts all data into a unified format.

[1425] For example, the timestamp format is standardized to Coordinated Universal Time (UTC) and missing parts of the log are reasonably estimated and supplemented.

[1426] 3. Behavioral pattern learning method (generative AI model)

[1427] A generative AI model hosted on a server receives preprocessed data as input and analyzes and learns from user behavior patterns using machine learning algorithms and statistical methods.

[1428] To learn behavioral patterns, the probability and typical patterns of these activities are stored in the model based on past data.

[1429] 4. Manual Generation Methods

[1430] Based on the learned behavioral patterns, the server automatically generates work instructions, which detail standard work procedures and how to handle irregular situations.

[1431] For example, a pattern such as "Start data entry at 9:00 every day and check email at 10:00" is clearly stated.

[1432] 5. Manual Distribution Methods

[1433] The server automatically delivers the generated work instructions to the user's device, allowing the user to always have the latest work instructions at hand.

[1434] The terminal displays the received work instructions in an appropriate format for easy reference by the user.

[1435] 6. Updates and Notifications

[1436] The server analyzes the data collected in real time and immediately updates the work instructions if there is a change in the behavioral pattern. This updated information is notified and new work instructions are automatically sent to the terminal.

[1437] For example, if system maintenance is performed only on Thursdays of certain weeks, that change will also be reflected immediately.

[1438] Specific examples

[1439] Let's take the example of User A, who works at a certain company. User A starts data entry at 9:00 every day and checks email at 10:00. In addition, system maintenance is performed once a week. This behavioral data is recorded by the device and sent to the server. The server preprocesses this data and learns behavioral patterns using a generative AI model.

[1440] Based on the behavioral patterns learned by the generative model, the following work instructions are automatically generated:

[1441] markdown

[1442] 1. Data entry begins at 9:00 every day.

[1443] Required tools: Datasheet, input software

[1444] Note: Double-check to avoid input errors.

[1445] 2. Check email at 10:00.

[1446] Tools used: Email client

[1447] Important Note: Prioritize important emails

[1448] 3. System maintenance will be performed every Friday.

[1449] Tools used: Maintenance software

[1450] Important note: Make a backup of your entire system beforehand.

[1451] This work instruction is delivered to User A's terminal, and User A follows it to perform the work, resulting in efficient and standardized work. In addition, if there is a change in the actual behavior pattern (for example, if system maintenance is changed to Thursday), the server immediately delivers updated work instructions, allowing the user to follow the latest procedures.

[1452] Prompt Sentence Examples

[1453] An example of a prompt to input to a generative AI model is as follows:

[1454] Please tell me the specific steps to take when entering data.

[1455] What should I pay attention to when checking my email?

[1456] "What is your standard procedure for system maintenance?"

[1457] keyword

[1458] Generative AI Models

[1459] Prompt statement

[1460] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1461] Step 1:

[1462] Data collection

[1463] When a user starts work, he logs in to the system, and the login time is recorded on the terminal.

[1464] For each application a user uses during work, its start and end times are collected in real time.

[1465] The terminal sends the collected data to the server in batch processing every minute, allowing operation logs on the terminal to be accumulated in real time.

[1466] Input: User operation data (e.g., login time, application usage time).

[1467] Output: The collected operation data is sent to the server.

[1468] Step 2:

[1469] Data Preprocessing

[1470] The server analyzes the raw data it receives and performs data cleansing, which includes removing incomplete data and outliers.

[1471] The server standardizes the timestamp format (e.g., converts it to Coordinated Universal Time (UTC)) and fills in missing data with estimated values.

[1472] Specific operation of data cleansing: If there is a missing login time, the average value is calculated from the data before and after the missing time to fill in the gaps.

[1473] Input: Collected operational data.

[1474] Output: The cleansed data.

[1475] Step 3:

[1476] Learning behavioral patterns

[1477] A generative AI model on a server receives the preprocessed data as input and begins analyzing the data using machine learning algorithms (e.g., recurrent neural networks).

[1478] The probability of certain patterns or abnormal behavior is calculated, and these models are gradually refined.

[1479] Specific operation: The generative AI model learns the average start time and frequency of a user's actions and stores the results in a database.

[1480] Input: Preprocessed data.

[1481] Output: Learned behavioral patterns (model).

[1482] Step 4:

[1483] Manual Generation

[1484] The server automatically generates work instructions based on the learned behavioral patterns, including routine work procedures and countermeasures for irregular situations.

[1485] Example: A standardized procedure such as "Start data entry at 9:00 and check email at 10:00 every day" is described.

[1486] Input: Learned behavioral patterns.

[1487] Output: The generated work order.

[1488] Step 5:

[1489] Manual Distribution

[1490] The server automatically delivers the generated work instructions to the user's device, allowing the user to always refer to the latest work instructions.

[1491] The terminal displays the received work instructions in an appropriate format for easy access by the user.

[1492] Notification function: Displays when a new manual has arrived on your device.

[1493] Input: Generated work order.

[1494] Output: Work instructions delivered to the terminal.

[1495] Step 6:

[1496] Updates and Notifications

[1497] The server analyzes the data collected in real time and immediately updates the work instructions if any changes occur in the behavioral patterns. The new work instructions are then delivered to the user's device along with the updated information.

[1498] Example: If the system maintenance schedule is changed, users will be notified immediately.

[1499] Input: Real-time behavioral data.

[1500] Output: Updated work order and notification.

[1501] (Application example 1)

[1502] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1503] There are already systems that collect user behavior data and analyze behavioral patterns based on that data to improve business efficiency and standardize operations. However, these systems lack real-time update capabilities or the ability to specifically control device operation, resulting in a lack of responsiveness and adaptability to business operations. Furthermore, they lack the ability to automatically generate manuals that reflect irregular behavior, limiting their ability to maximize user work efficiency. The purpose of this invention is to solve these problems and achieve higher levels of business efficiency and adaptability.

[1504] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1505] In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, means for analyzing the preprocessed behavioral data and learning behavioral patterns, means for generating a manual based on the learned behavioral patterns, means for distributing the generated manual to the user's device, means for controlling the operation of the device according to the set pattern, means for instantly updating the manual to the latest version based on real-time behavioral data, and means for notifying the device of the contents of the updated manual. This enables real-time business optimization and standardization based on the user's business data.

[1506] "Behavioral data" refers to the history and log information of specific operations and actions performed by a user.

[1507] "Preprocessing" is the process of converting raw data into a form suitable for analysis, including data cleansing and format conversion.

[1508] A "generative model" is a machine learning algorithm or statistical model that learns behavioral patterns based on preprocessed data.

[1509] A "manual" is a document or guideline that describes procedures and instructions for users to perform their work efficiently.

[1510] "Devices" refer to hardware devices such as terminals and robots used by users.

[1511] "Control" refers to the operation of directing the behavior or state of equipment, and is the process of managing behavior according to specific patterns.

[1512] "Real-time update" is the process of instantly updating the manual based on the latest behavioral data and delivering that information to the user's device.

[1513] "Irregular behavior" refers to unusual actions or operations that deviate from normal patterns of behavior, and procedures for dealing with such behavior are reflected in the manual.

[1514] This invention is a system that automatically generates and updates business manuals by collecting user behavior data and analyzing behavioral patterns based on the collected data. An embodiment of this system will be described in detail below.

[1515] System Configuration

[1516] The system mainly consists of the following components:

[1517] 1. How to collect user behavior data

[1518] 2. A means of preprocessing the collected behavioral data

[1519] 3. A generative modeling method that analyzes preprocessed behavioral data and learns behavioral patterns

[1520] 4. A method for generating manuals based on learned behavioral patterns

[1521] 5. Means for delivering the generated manual to the user's device

[1522] 6. Means of controlling the operation of equipment according to set patterns

[1523] 7. A way to instantly update manuals based on real-time behavioral data

[1524] 8. Means of notifying the device of updated manual contents

[1525] Operation overview

[1526] First, as users perform their work, sensors and cameras on devices (e.g., terminals and factory robots) collect behavioral data in real time. This data includes operating hours, tools used, movement patterns, and more. The collected data is sent to a server where preprocessing is performed. This involves filling in missing data and removing outliers. Next, a generative model learns behavioral patterns based on the preprocessed data. A machine learning algorithm is used for this purpose.

[1527] Based on the learned behavioral patterns, the system generates a work manual that includes specific operating procedures, tools to use, and important points to note. For example, the following manual may be generated:

[1528] 1. Starts operation at 8:00 every day.

[1529] Tools used: Gripper, drill.

[1530] Important note: Perform initial calibration of the gripper.

[1531] 2. Replace part A at 12:00.

[1532] Tools used: Torque wrench.

[1533] Note: Check the torque value.

[1534] 3. Full maintenance every Friday.

[1535] Tools used: Various tools.

[1536] Important note: Make a full system backup.

[1537] The generated manual is automatically distributed to the user's device and used to appropriately control the device's operation. Furthermore, the manual is instantly updated in response to changes in behavior patterns in real time. This update information is notified to the device, so the latest procedures are always reflected.

[1538] Hardware and software used

[1539] Collection devices: sensors, cameras, terminals, factory robots

[1540] Data preprocessing: Pandas (Python library)

[1541] Generative models: Random Forest (machine learning algorithm), other AI models

[1542] Data analysis server: High performance server

[1543] Distribution System: Network Communication

[1544] Prompt Sentence Examples

[1545] An example prompt based on a generative AI model is:

[1546] Based on the user's behavioral data, generate a robot operation manual like the one below.

[1547] 1. Starts operation at 8:00 every day.

[1548] Tools used: Gripper, drill.

[1549] Important note: Perform initial calibration of the gripper.

[1550] 2. Replace part A at 12:00.

[1551] Tools used: Torque wrench.

[1552] Note: Check the torque value.

[1553] 3. Full maintenance is performed automatically every Tuesday.

[1554] Tools used: Various tools.

[1555] Important note: Make a full system backup.

[1556] The above describes a specific embodiment of the invention. By using this system, it is possible to achieve greater efficiency and standardization of work, thereby maximizing the user's work performance.

[1557] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1558] Step 1:

[1559] The server collects user behavior data. This input data includes user uptime, tools and applications used, and movement patterns. This data is collected in real time from sensors, cameras, and devices.

[1560] Step 2:

[1561] The server preprocesses the collected behavioral data. This preprocessing includes filling in missing data, removing outliers, and converting the data into a format suitable for analysis. Raw data is used as input, and cleansed data is output after processing. Data processing libraries such as Pandas are used.

[1562] Step 3:

[1563] The server supplies the preprocessed data as input to the generative model. The generative model uses machine learning algorithms such as Random Forest to learn behavioral patterns. The input data is preprocessed behavioral data, and the output is learned behavioral patterns. The model runs on the server and performs analysis.

[1564] Step 4:

[1565] The server generates a manual based on the learned behavioral patterns. Specifically, a manual is automatically generated that describes the user's work procedures, tools to be used, and points to note. In this process, the learned behavioral patterns are used as input data, and the work manual is output.

[1566] Step 5:

[1567] The server distributes the generated manual to the user's device. The manual is automatically sent to the device via network communication. The input data is the generated manual, and the output is the state that has been distributed to the user's device.

[1568] Step 6:

[1569] The server controls the operation of the equipment according to the set patterns. Specifically, it manages the equipment so that it follows the manual and performs its tasks efficiently and accurately. The input data is the distributed manual, and the output is the actual operation control of the equipment.

[1570] Step 7:

[1571] The server instantly updates the manual to the latest version based on real-time behavioral data. It continuously collects user behavioral data and immediately updates the manual if there are any changes. The input data is the latest behavioral data, and the output is an updated manual.

[1572] Step 8:

[1573] The server notifies the user's device of the contents of the updated manual, allowing the user to always perform their work based on the latest procedures. The input data is the updated manual, and the output is a notification sent to the user's device.

[1574] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1575] This invention is a system that collects user behavior data, analyzes and learns behavioral patterns based on this data, and automatically generates and updates effective manuals. Furthermore, by combining it with an emotion engine that recognizes the user's emotional state, it realizes flexible responses based on the user's emotions. This system makes it possible to improve work efficiency and introduce human-centered design.

[1576] System configuration

[1577] The system consists of the following components:

[1578] 1. Data Collection Methods

[1579] 2. Data preprocessing methods

[1580] 3. Learning methods for behavioral patterns (generative models)

[1581] 4. Emotion Engine

[1582] 5. Manual Generation Methods

[1583] 6. Manual Distribution Methods

[1584] 7. Updates and Notifications

[1585] Specific processing flow

[1586] 1. Data Collection

[1587] Users begin their daily work and perform tasks on the device, such as logging in to the system, using applications, entering data, and checking email.

[1588] The device records the user's operation data in real time and periodically transmits the data to the server. In addition, the emotion engine recognizes the user's emotional state, which is also included in the behavioral data.

[1589] 2. Data Preprocessing

[1590] The server preprocesses the received data, filling in missing data, removing noise, and normalizing the data. The server also preprocesses the user's emotion data.

[1591] 3. Learning behavioral patterns and emotions

[1592] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns and emotional state. Specifically, it uses machine learning algorithms to analyze the correlation between user behavior and emotions.

[1593] For example, the model learns which specific tasks or times of day users find stressful.

[1594] 4. Manual Generation

[1595] Based on the learned behavioral patterns and emotional data, the server automatically generates a manual detailing work procedures and important points to note. This manual includes not only standard work procedures but also customizations based on the user's emotional state.

[1596] For example, if the user is feeling stressed, it may suggest taking a short break.

[1597] 5. Manual Distribution

[1598] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[1599] 6. Updates and Notifications

[1600] The server analyzes the data in real time and immediately updates the manual if any changes in behavioral patterns or emotional states are detected. This update information is notified to the user, and the new manual is automatically distributed to the device.

[1601] Specific examples

[1602] User B, who works for a certain company, records behavioral and emotional data on his device as he goes about his daily work. User B starts entering data at 9:00 and checks his email at 10:00 every day. However, the data shows that he begins to feel stressed around 14:00 every day. This data is sent to a server in real time and pre-processed on the server.

[1603] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[1604] 1. Data entry begins at 9:00 every day.

[1605] Required tools: Datasheet, input software

[1606] Note: Double-check to avoid input errors.

[1607] 2. Check email at 10:00.

[1608] Tools used: Email client

[1609] Important Note: Prioritize important emails

[1610] 3. Take a short break every day at 2:00 PM.

[1611] The break time is 15 minutes

[1612] Note: It is recommended to rest in a relaxing environment.

[1613] This manual is distributed to User B's device, and User B can follow it to perform his / her work efficiently. Taking breaks during times of stress in particular improves work efficiency and maintains health. The program automates the entire process, from data collection to manual generation and updating, reducing the user's workload.

[1614] This system allows users to perform tasks using procedures that are optimal for their emotional state, which not only improves work efficiency and standardizes work, but also contributes to health management.

[1615] The processing flow will be explained below.

[1616] Step 1:

[1617] A user begins their daily routine, for example, logging into a terminal and beginning a data entry task.

[1618] Step 2:

[1619] The device records user operation data in real time, including the start time of data entry, the application used, and the progress of the work.

[1620] Step 3:

[1621] The emotion engine recognizes the user's emotional state using biometric data such as facial expressions, voice, and heart rate, and is supported by cameras, microphones, and wearable devices attached to the device.

[1622] Step 4:

[1623] The device transmits the recorded behavioral and emotional data to a server at regular intervals, where the data is efficiently collected via a communication protocol.

[1624] Step 5:

[1625] The server preprocesses the received data, such as filling in missing data, removing noise, and normalizing the data. The same preprocessing is performed on emotion data.

[1626] Step 6:

[1627] The server inputs the preprocessed data into a generative model, which uses machine learning algorithms to analyze and learn correlations between user behavior patterns and emotions.

[1628] Step 7:

[1629] The generative model calculates the probability of certain patterns or irregular behavior based on the user's behavioral patterns and emotional data. For example, it determines that a user is likely to feel stressed at a certain time of day.

[1630] Step 8:

[1631] The server generates a manual based on the learned behavioral patterns and emotional data, which includes commonly performed tasks and procedures, as well as customization according to the user's emotional state.

[1632] Step 9:

[1633] The server delivers the generated manual to the user's terminal, where the user can check the latest manual and proceed with their work accordingly.

[1634] Step 10:

[1635] The server continues to analyze data in real time, and if any changes in behavioral patterns or emotional states are detected, the manual is immediately updated. This updated information is notified to the user, and the new manual is automatically distributed to the device.

[1636] Step 11:

[1637] Users receive notifications and follow the updated manual to carry out their work, which not only improves work efficiency and standardization but also provides a work environment that takes into account the user's emotional state.

[1638] Example 2

[1639] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1640] Conventional systems only collect user behavioral data and do not consider emotional states, which means they are unable to improve work efficiency or adequately manage the user's health. In particular, they are unable to grasp when a user feels stressed or their emotional state during a specific task, making it difficult to provide appropriate manuals. As a result, users' work efficiency declines and their mental burden increases.

[1641] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1642] In this invention, the server includes means for collecting user behavioral data and emotional data, means for preprocessing the collected behavioral data and emotional data, and generative model means for analyzing the preprocessed behavioral data and emotional data and learning behavioral patterns and emotional states. This makes it possible to generate and distribute optimal manuals that take into account not only the user's behavior but also their emotional states.

[1643] A "user" is a person who operates the system and provides behavioral and emotional data.

[1644] "Behavioral data" is information related to a user's terminal operations, and includes specific operation content, operation timing, applications used, input data, and the like.

[1645] "Emotion data" is information that represents the user's emotional state, and includes psychological states such as stress, joy, anger, and sadness that can be obtained from facial expressions and voice.

[1646] "Means for collection" is a general term for hardware and software for recording user behavioral data and emotional data in real time and transmitting it to a server.

[1647] "Preprocessing means" is a general term for the process of filling in missing data, removing noise, normalizing data, etc. on collected raw data, as well as the technical equipment and software used for this process.

[1648] "Generative model means" is a general term for a machine learning model that learns a user's behavioral patterns and emotional states using preprocessed behavioral data and emotional data, and the system that operates it.

[1649] "Means for generating manuals" is a general term for algorithms and systems that automatically create manuals that list optimal work procedures and important points based on learned behavioral patterns and emotional states.

[1650] "Distribution means" is a general term for a network and related hardware and software for transmitting the generated manual to a user's terminal and enabling the user to access it.

[1651] This invention relates to a system that collects user behavioral and emotional data, analyzes and learns behavioral patterns and emotional states based on this data, and automatically generates and updates effective manuals. This system makes it possible to improve work efficiency and introduce human-centered design.

[1652] The system consists of the following elements:

[1653] 1. Data Collection Methods

[1654] 2. Data preprocessing methods

[1655] 3. Learning methods for behavioral patterns and emotions (generative models)

[1656] 4. Manual Generation Methods

[1657] 5. Manual Distribution Methods

[1658] 6. Updates and Notifications

[1659] Data collection methods

[1660] Users go about their daily work and log in to the system's terminals to perform various tasks. Specifically, they log in to the system, use applications, enter data, check email, etc. The terminals record user operations in real time. For example, they collect operation data such as keyboard input, mouse operation, and application launch history, and record them along with timestamps. In addition, an emotion engine is used to recognize the user's emotional state from facial expression and voice data, and this information is also included in the behavioral data.

[1661] Data preprocessing measures

[1662] The server receives the raw data sent from the device. It preprocesses the received data separately for behavioral data and emotional data. This includes filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data and noise such as momentary emotional changes is removed. Data normalization converts data of different scales into a consistent range.

[1663] A means of learning behavioral patterns and emotions

[1664] The server inputs the preprocessed behavioral and emotional data into a generative model to learn the user's behavioral patterns and emotional state. Using machine learning algorithms, such as deep learning libraries like TensorFlow and PyTorch, correlations between the user's behavioral and emotional data can be analyzed. At this stage, the model identifies stress responses for specific tasks or times of day.

[1665] Manual Generation Method

[1666] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customizations based on the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed.

[1667] Manual distribution method

[1668] The server distributes the generated manual to the user's terminal, allowing the user to carry out their work efficiently through the latest manual.

[1669] Updates and Notifications

[1670] The server analyzes data in real time, and if any changes in behavioral patterns or emotional states are detected, it immediately updates the manual and notifies the user. The new manual is automatically distributed to the device.

[1671] Specific examples

[1672] For example, when User B, who works for a company, goes about his daily work, his device records behavioral and emotional data. User B starts entering data at 9:00 every day and checks his email at 10:00, but the data shows that he starts to feel stressed around 14:00 every day. This data is sent to the server in real time and pre-processed on the server.

[1673] The preprocessed data is input into a generative model, which learns the behavioral patterns and emotional state of User B. The generated model then automatically generates the following manual:

[1674] 1. Data entry begins at 9:00 every day.

[1675] Required tools: Datasheet, input software

[1676] Note: Double-check to avoid input errors.

[1677] 2. Check email at 10:00.

[1678] Tools used: Email client

[1679] Important Note: Prioritize important emails

[1680] 3. Take a short break every day at 2:00 PM.

[1681] The break time is 15 minutes

[1682] Note: It is recommended to rest in a relaxing environment.

[1683] This manual is delivered to User B's device, and User B can follow it to perform his / her work efficiently. In particular, taking a break during times when he / she feels stressed improves work efficiency and maintains his / her health.

[1684] Example prompts to input to the generative AI model

[1685] For example, the following prompt sentence is input to the generative AI model:

[1686] Please provide prompts to automatically generate optimal work procedures based on User B's behavioral patterns and emotional state when performing daily work at the company.

[1687] As a specific data-based procedure, User B starts entering data at 9:00 every morning and checks email at 10:00, but tends to feel stressed by 14:00.

[1688] Please generate a manual based on this.

[1689] In this way, the system of the present invention provides optimal work procedures that match the user's emotional state, thereby improving work efficiency and managing the user's health.

[1690] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1691] Step 1: Data collection

[1692] Users go about their daily work and log in to the system's terminals to perform various tasks, such as data entry, email checking, and report creation. The terminals monitor these operations in real time and collect operation data. Furthermore, the emotion engine uses the terminal's camera and microphone to analyze facial expressions and voice to collect emotional data. The input is the user's operation data and emotional state, and this data is recorded with a timestamp and a data package is generated to be sent to the server as output.

[1693] Step 2: Data Preprocessing

[1694] The server receives raw data sent from the device. The received data undergoes preprocessing such as filling in missing data, removing noise, and normalizing the data. For example, if there is missing data, it is filled in using the previous data, and noise caused by momentary changes in emotions is removed. It also performs normalization to convert data of different scales into a consistent range. The input is the raw data sent from the device, and the output is the preprocessed, clean data.

[1695] Step 3: Learning behavioral patterns and emotions

[1696] The server inputs the preprocessed behavioral data and emotional data into a generative model to learn the user's behavioral patterns and emotional state. For example, it uses machine learning libraries such as Python's TensorFlow or PyTorch to analyze the correlation between the user's behavioral data and emotional data. The learning model identifies the user's stress response for specific tasks and time periods. The input is the preprocessed data, and the output is the trained generative model.

[1697] Step 4: Manual generation

[1698] The server automatically generates a manual detailing work procedures and important points based on the learned behavioral patterns and emotional data. This manual includes not only standard work procedures but also customized content according to the user's emotional state. For example, it suggests taking a temporary break during times when the user feels stressed. The input is the trained generative model and real-time user data, and the output is the generated manual.

[1699] Step 5: Manual distribution

[1700] The server delivers the generated manual to the user's terminal. The user can perform their work efficiently through the latest manual. The generated manual is sent to the terminal, and the user proceeds with their work based on it. The input is the generated manual, and the output is the delivered manual.

[1701] Step 6: Updates and Notifications

[1702] The server analyzes new data in real time and immediately updates the manual if it detects changes in the user's behavioral patterns or emotional state. The updated manual is immediately notified to the user, and the new manual is automatically delivered to the device. The input is new user data and the existing generative model, and the output is the updated manual and its notification.

[1703] (Application example 2)

[1704] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1705] Conventional work manuals are generally rigid and designed without considering the behavioral data or emotional state of individual users, so they cannot be said to be effective in improving work efficiency within a company or managing employee health. In particular, there is a demand for systems that can flexibly respond to stress and irregular behavior that occur during work.

[1706] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting user behavioral data, means for preprocessing the collected behavioral data, generative model means for analyzing the preprocessed behavioral data and learning behavioral patterns, emotion engine means for recognizing the user's emotional state, means for generating a manual based on the learned behavioral patterns and emotional state, and means for delivering the generated manual to the user's terminal. This makes it possible to provide a flexible and effective business manual that corresponds to the user's behavioral data and emotional state.

[1707] "Means for collecting user behavior data" refers to a device or software for collecting in real time operation logs, usage histories, and other behavioral information generated when a user operates the system.

[1708] "Means for preprocessing collected behavioral data" refers to a device or software that performs processes such as filling in missing data, normalizing data, and removing noise in order to convert the collected behavioral data into a format suitable for analysis.

[1709] The "generative model means for learning behavioral patterns" is a device or software for analyzing a user's behavioral patterns based on preprocessed behavioral data and learning specific behavioral patterns using a machine learning algorithm.

[1710] The "emotion engine means for recognizing the user's emotional state" is a device or software for analyzing the user's operation data, facial expressions, voice, etc. to recognize the user's emotional state.

[1711] The "means for generating a manual based on learned behavioral patterns and emotional states" refers to a device or software for automatically generating a manual including user-specific work procedures and points of caution based on learned behavioral patterns and emotional data.

[1712] The "means for distributing the generated manual to the user's terminal" refers to a device or software for providing the generated manual to the terminal used by the user.

[1713] This invention is a system that collects user behavioral data and emotional states, analyzes and learns from them, and automatically generates and updates flexible business manuals. A specific embodiment of this system is described below.

[1714] System configuration

[1715] Hardware configuration:

[1716] Data collection device: A smartphone or other device operated by the user.

[1717] Server: A central server that performs data preprocessing, training, manual generation, and distribution.

[1718] Software configuration:

[1719] Data collection module: collects user behavior data.

[1720] Pre-processing module: The collected data is imputed, denoised, and normalized.

[1721] Machine learning algorithms (e.g. scikit-learn, numpy): Generative models that learn behavioral patterns.

[1722] Emotion Engine: Recognize the user's emotional state (Example: EmotionEngine).

[1723] Manual generation module: Generates a manual based on learned behavioral patterns and emotion data.

[1724] Distribution module: distributes the generated manual to the user's terminal.

[1725] What the program does

[1726] 1. Data Collection Methods

[1727] The server monitors user operations in real time and collects behavioral data, including application usage history, operation logs, and other behavioral information, and simultaneously recognizes the user's emotional state using an emotion engine.

[1728] 2. Data preprocessing methods

[1729] The collected behavioral data is preprocessed on the server. The preprocessing module performs data filling, noise removal, and data normalization to convert the data into a format suitable for analysis and learning.

[1730] 3. Behavioral pattern learning method

[1731] The preprocessed data is then fed into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning technique that analyzes behavioral patterns and correlates them with emotional states.

[1732] 4. Emotional Engine Means

[1733] The emotion engine recognizes the user's emotional state and correlates it with behavioral patterns, for example, analyzing stress levels during specific tasks or times of day.

[1734] 5. Manual Generation Methods

[1735] The server generates a work manual based on the learned behavioral patterns and emotional states, which includes specific work procedures, points to note, and advice based on the emotional state.

[1736] 6. Manual Distribution Methods

[1737] The generated manual is automatically distributed from the server to the user's device, allowing the user to always perform their work based on the latest operational manual.

[1738] Specific examples

[1739] For example, when a user works in a physical store, the system collects the user's behavioral and emotional data. The data shows that the user puts products on the shelves at 9:00 every morning and rings up the cash register at 10:00. However, it is detected that the user begins to feel stressed around 2:00 pm. Based on this information, the system generates the following manual:

[1740] Example prompt sentence:

[1741] Based on User A's behavioral patterns and emotional data, you can see that he feels stressed at 2 PM. Based on this data, generate an optimal work manual. The manual should include work procedures and points to note depending on the user's emotional state.

[1742] This allows users to receive specific advice on how to reduce stress, along with the optimal work procedures based on their emotional state. This system can simultaneously improve work efficiency and manage employee health.

[1743] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1744] Step 1:

[1745] Data collection

[1746] The terminal collects operation logs, usage history, and other behavioral data generated when the user operates the system in real time. It also uses an emotion engine to simultaneously recognize and record the user's emotional state. Specifically, the terminal records the user's operation details (e.g., arranging products, ringing up cash registers, etc.) and the time of operation, and sends this data to the server. The input is the user's operation data and emotional state, and the output is the collected raw data.

[1747] Step 2:

[1748] Data Preprocessing

[1749] The server receives the collected behavioral data and performs preprocessing. The preprocessing module performs processes such as filling in missing data, removing noise, and normalizing the data, converting it into a format suitable for analysis and learning. For example, if the collected data contains missing data, it is filled in based on past data and outliers are removed. The input is the collected raw data, and the output is preprocessed, clean data.

[1750] Step 3:

[1751] Behavioral pattern learning

[1752] The server inputs the preprocessed data into a generative model to learn the user's behavioral patterns. The algorithm used is a machine learning method that analyzes behavioral patterns and correlates them with emotional states. For example, the server learns what tasks the user is performing at a particular time and how their emotional state is changing. The input is the preprocessed data, and the output is the learned behavioral patterns.

[1753] Step 4:

[1754] Emotion Engine Analysis

[1755] The server uses an emotion engine to analyze the relationship between the user's emotional state and behavioral patterns. Specifically, it analyzes stress levels during specific tasks and time periods. The input is the collected emotion data, and the output is the emotion analysis results.

[1756] Step 5:

[1757] Manual Generation

[1758] The server generates a work manual based on the learned behavioral patterns and emotional state. The generated manual includes specific work procedures, points to note, and advice tailored to the employee's emotional state. For example, it suggests taking a break during times when employees are likely to feel stressed. The input is the learned behavioral patterns and the results of emotional analysis, and the output is the generated work manual.

[1759] Step 6:

[1760] Manual Distribution

[1761] The server distributes the generated manual to the user's terminal. The terminal receives it and displays it for easy reference by the user. If the manual needs to be updated, a notification is sent in real time. The input is the generated business manual, and the output is the distribution of the manual to the user's terminal and notification.

[1762] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1763] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1764] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1765] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1766] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1767] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1768] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1769] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1770] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1771] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1772] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1773] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1774] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1775] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1776] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1777] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1778] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1779] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1780] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1781] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1782] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1783] The following is further disclosed regarding the above embodiment.

[1784] (Claim 1)

[1785] a means for collecting user behavior data;

[1786] means for preprocessing the collected behavioral data;

[1787] a generative modeling means for analyzing the preprocessed behavioral data and learning behavioral patterns;

[1788] A means for generating a manual based on the learned behavioral patterns;

[1789] A means for distributing the generated manual to a user's terminal;

[1790] A system including:

[1791] (Claim 2)

[1792] The method further includes a means for calculating a probability of irregular behavior based on the behavior data and reflecting the probability in a manual.

[1793] 10. The system of claim 1.

[1794] (Claim 3)

[1795] means for updating the contents of the manual in real time and notifying the user of the update;

[1796] 10. The system of claim 1.

[1797] (Claim 4)

[1798] The behavioral data collection means is an application running on the user's terminal.

[1799] 10. The system of claim 1.

[1800] "Example 1"

[1801] (Claim 1)

[1802] means for collecting user activity data;

[1803] means for pre-processing the collected activity data;

[1804] a generative modeling means for analyzing the preprocessed activity data and learning behavioral patterns;

[1805] A means for generating work instructions based on the learned behavioral patterns;

[1806] means for distributing the generated work instructions to a user's device;

[1807] A system including:

[1808] (Claim 2)

[1809] a means for calculating a probability of atypical behavior based on the activity data and reflecting the probability in a work instruction;

[1810] 10. The system of claim 1.

[1811] (Claim 3)

[1812] The system further includes a means for instantly updating the content of the work instruction and notifying the user of the update.

[1813] 10. The system of claim 1.

[1814] "Application Example 1"

[1815] (Claim 1)

[1816] a means for collecting user behavior data;

[1817] means for preprocessing the collected behavioral data;

[1818] a generative modeling means for analyzing the preprocessed behavioral data and learning behavioral patterns;

[1819] A means for generating a manual based on the learned behavioral patterns;

[1820] means for delivering the generated manual to a user's device;

[1821] means for controlling the operation of the device according to a set pattern;

[1822] A means to instantly update manuals based on real-time behavioral data,

[1823] a means for notifying the device of the contents of the updated manual;

[1824] A system including:

[1825] (Claim 2)

[1826] The method further includes a means for calculating a probability of irregular behavior based on the behavior data and reflecting the probability in a manual.

[1827] 10. The system of claim 1.

[1828] (Claim 3)

[1829] means for updating the contents of the manual in real time and notifying the user of the update;

[1830] 10. The system of claim 1.

[1831] "Example 2: Combining Emotion Engines"

[1832] (Claim 1)

[1833] means for collecting user behavioral and emotional data;

[1834] means for preprocessing the collected behavioral and emotional data;

[1835] a generative modeling means for analyzing the preprocessed behavioral data and emotion data to learn behavioral patterns and emotion states;

[1836] means for generating a manual based on the learned behavioral patterns and emotional states;

[1837] A means for distributing the generated manual to a user's terminal;

[1838] A system including:

[1839] (Claim 2)

[1840] The system further includes a means for calculating the probability of irregular behavior or emotional state based on the behavior data and emotional data, and reflecting the probability in a manual.

[1841] 10. The system of claim 1.

[1842] (Claim 3)

[1843] means for updating the contents of the manual in real time and notifying the user of the update;

[1844] 10. The system of claim 1.

[1845] "Application example 2 when combining emotion engines"

[1846] (Claim 1)

[1847] a means for collecting user behavior data;

[1848] means for preprocessing the collected behavioral data;

[1849] a generative modeling means for analyzing the preprocessed behavioral data and learning behavioral patterns;

[1850] emotion engine means for recognizing an emotional state of a user;

[1851] means for generating a manual based on the learned behavioral patterns and emotional states;

[1852] A means for distributing the generated manual to a user's terminal;

[1853] A system including:

[1854] (Claim 2)

[1855] The method further includes a means for calculating a probability of irregular behavior based on the behavior data and reflecting the probability in a manual.

[1856] 10. The system of claim 1.

[1857] (Claim 3)

[1858] means for updating the contents of the manual in real time and notifying the user of the update;

[1859] 10. The system of claim 1. [Explanation of symbols]

[1860] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means for collecting user behavior data; means for preprocessing the collected behavioral data; a generative modeling means for analyzing the preprocessed behavioral data and learning behavioral patterns; A means for generating a manual based on the learned behavioral patterns; A means for distributing the generated manual to a user's terminal; A system including:

2. The method further includes a means for calculating a probability of irregular behavior based on the behavior data and reflecting the probability in a manual. The system of claim 1 .

3. means for updating the contents of the manual in real time and notifying the user of the update; The system of claim 1 .

4. The behavioral data collection means is an application running on the user's terminal. The system of claim 1 .

Images