System

Abstract

A system is provided.SOLUTION: A system comprising: means for collecting business-related data from an application used by an individual employee; means for analyzing the business-related data and extracting and filtering necessary information; means for storing the extracted information in a database; means for training a machine learning model based on the database and generating a business clone for each employee; and means for accessing the business clone and performing a business.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Many of the tasks that each employee is responsible for are known only to the employee, and it is common for tasks to not be adequately handed over when the employee resigns or is transferred. Under these circumstances, tasks that have become like those of a private business are left unattended, creating a high risk of disrupting the business continuity of the entire organization. Furthermore, when an employee is absent, it becomes difficult to understand existing tasks or handle new ones, resulting in business stagnation and a decline in work efficiency. The purpose of this invention is to solve these problems, facilitate employee handover procedures, and ensure business continuity. [Means for solving the problem]

[0005] The system includes a means for collecting work-related data from applications used by individual employees, a means for analyzing the work-related data, extracting and filtering necessary information, a means for storing the extracted information in a database, a means for training a machine learning model based on the database and generating work clones for each employee, and a means for accessing the work clones and performing work tasks. This system ensures smooth work handover and business continuity even when an employee is absent. Furthermore, the work-related data collected from applications used by employees includes emails, chat messages, schedule information, and created documents, enabling a comprehensive understanding of work status. Furthermore, the work clones can perform specific tasks, such as automatically replying to emails, attending meetings, and proofreading documents, when an employee is absent, preventing work stagnation and improving work efficiency.

[0006] A "data collection tool" is a device or program that collects work-related data from applications used by employees.

[0007] "Data analysis means" refers to a device or program that analyzes collected business-related data and extracts and filters necessary information.

[0008] The "database storage means" is a device or program that stores the analyzed information in a database.

[0009] The "machine learning model training means" is a device or program that trains a machine learning model based on data stored in a database and generates work clones for each employee.

[0010] A "job clone" is a virtual representative generated by a trained machine learning model that can perform an employee's job in their absence.

[0011] A "business execution means" is a device or program that accesses a business clone and executes a specific business.

[0012] "Email" is a communication medium that includes text data and attached files and is sent and received via a network such as the Internet.

[0013] "Chat messages" are text data exchanged in real time using instant messaging services.

[0014] "Schedule information" is data for managing employee schedules and tasks.

[0015] "Created documents" refers to electronic data such as text files, spreadsheets, and presentation materials created or edited by employees.

[0016] "Automatic reply" is a process of automatically replying to an email based on preset conditions.

[0017] "Meeting attendance" is a process in which a work clone participates in an online or real meeting on behalf of an employee, making comments and taking notes.

[0018] "Document proofreading" is the process of checking and correcting the content of a document to improve its quality. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention is a system in which AI automatically aggregates the work of each employee and builds the work content into a database as a clone. With this system, even when an employee is absent, the work content and response history can be understood by inquiring with the clone, enabling a smooth handover of work.

[0041] System Configuration

[0042] The system consists of the following main components:

[0043] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[0044] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[0045] 3. Database: Data storage that stores and makes accessible the analyzed data.

[0046] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[0047] Program Operation

[0048] 1. Data Collection

[0049] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from Microsoft Outlook, and also collects chat messages from Slack.

[0050] 2. Data Analysis and Storage

[0051] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unwanted data (e.g., spam emails and irrelevant chats) is also filtered out. The analyzed data is stored in a structured format in a database.

[0052] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[0053] 3. Training machine learning models and generating business clones

[0054] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[0055] For example, a clone can be created that learns employee A's past email reply patterns and the content of comments made in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[0056] 4. Access to and Use of Business Clones

[0057] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone, which retrieves the relevant information from a database and responds. The user can also request the clone to perform specific tasks.

[0058] For example, a user asks the clone of employee A, "Please tell me the date of the next project meeting," and the clone answers by looking up the schedule information.The clone also automatically replies to the specified email.

[0059] Specific examples

[0060] For example, if employee A suddenly becomes ill and cannot come to work, the user can use employee A's work clone to continue their daily work. The clone will automatically reply to emails received on employee A's behalf, attend meetings, and create minutes. In this way, it is possible to minimize work disruptions caused by employee A's absence and ensure business continuity.

[0061] This system effectively automates employee tasks and enables responses when employees are absent without human intervention, thereby improving productivity and operational efficiency across the organization.

[0062] The processing flow will be explained below.

[0063] Step 1:

[0064] The device accesses applications used by employees and collects work-related data, for example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0065] Step 2:

[0066] The server receives the collected data and temporarily stores it, allowing various data to be managed centrally.

[0067] Step 3:

[0068] The server analyzes the data it receives. It uses natural language processing (NLP) to analyze the content of emails and chat messages, extracting the necessary information, and filtering out unnecessary data (e.g., spam emails and irrelevant chats).

[0069] Step 4:

[0070] The server structures the parsed data, organizing it based on email content, sender, recipient, date, tagged keywords, etc.

[0071] Step 5:

[0072] The server stores the structured data in a database, which stores the data organized by employee for efficient access later.

[0073] Step 6:

[0074] The server reads data from the database and trains the machine learning model, using past business data and interaction history.

[0075] Step 7:

[0076] The server uses trained machine learning models to generate work clones for each employee, designed to mimic specific work patterns and behaviors.

[0077] Step 8:

[0078] Users access the work clone through a web interface or a dedicated application, and obtain information by asking the clone questions (e.g., "When is the next meeting scheduled?").

[0079] Step 9:

[0080] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[0081] Step 10:

[0082] Users can also assign tasks to the work clone, such as replying to a specific email or requesting attendance at a meeting.

[0083] Step 11:

[0084] The work clone performs the requested tasks (e.g., replying to emails, attending meetings, proofreading documents, etc.), which prevents work from stalling when an employee is absent.

[0085] In this way, each step works in coordination to realize a system that smoothly takes over employees' work and ensures business continuity throughout the organization.

[0086] Example 1

[0087] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0088] When an employee is absent, a means is needed to smoothly transfer work and prevent business stagnation. With conventional systems, it is difficult to grasp the content of employees' work and their response history, and work often does not proceed smoothly when employees are absent. This causes problems such as a decline in productivity and work efficiency throughout the organization. There is a need to solve this problem and provide a means to effectively continue work even when employees are absent.

[0089] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0090] In this invention, the server includes means for collecting work-related data from software used by each employee, means for analyzing the work-related data and extracting and filtering necessary information using natural language processing, means for storing the extracted information in a structured format in data storage, means for training a machine learning model based on the data in the data storage and generating a virtual representative for each employee, and means for accessing the virtual representative and performing work. This allows the virtual representative to automatically take over work even when the employee is absent, and to respond to emails, attend meetings, etc.

[0091] "Employee" refers to an individual who performs work within a company or organization.

[0092] "Software" refers to applications and programs that run on a computer.

[0093] "Business-related data" refers to information related to business, such as emails, messages, schedule data, and files created.

[0094] "Collection" refers to the act of gathering specific data or information.

[0095] "Natural language processing" refers to the technology that allows computers to analyze and understand human language.

[0096] "Extraction" refers to the act of isolating and extracting specific data or information.

[0097] "Filtering" refers to the act of removing unnecessary data and leaving only the necessary data.

[0098] "Data storage" refers to devices and systems for storing data.

[0099] A "structured format" refers to data that is organized according to rules and put into a form that can be easily handled.

[0100] A "machine learning model" refers to an algorithm or system that learns from data and performs pattern recognition and prediction.

[0101] A "virtual representative" refers to a software agent generated based on a machine learning model that mimics and performs the work of a specific employee.

[0102] "Access" refers to the act of connecting to, operating, or using data or a system.

[0103] "Performance" refers to the act of carrying out a specific task or job.

[0104] The system of this invention automatically aggregates employee tasks and builds a database of their tasks as virtual representatives. The system consists of the following main components:

[0105] 1. Devices: Computers and mobile devices used by employees on a daily basis. Each device has installed an email client, chat tool, schedule management application, etc. Examples include Microsoft Outlook, Slack, Google Calendar, and Microsoft Word.

[0106] 2. Server: A central device for storing and analyzing collected data and training machine learning models. This server receives, analyzes, filters, and stores data, trains machine learning models, and generates virtual representatives.

[0107] 3. Data storage: A database that stores and makes accessible the analyzed data. This data storage stores the data in a structured format.

[0108] 4. Virtual Representative: A virtual representative for each employee, generated based on machine learning, that performs tasks on behalf of the employee when they are not available.

[0109] As an example, consider the case where employee D suddenly falls ill and is unable to come to work. Employee D's device collects emails sent and received via Microsoft Outlook, which he or she uses daily, and Slack chat messages. The device transfers this data to a server. The server analyzes the transferred data using natural language processing technology, extracting and filtering important work-related information. The analyzed data is then stored in a structured format in data storage. The server then trains a machine learning model based on the data stored in the data storage. The trained model learns employee D's work patterns and response history and generates a virtual representative. Users can access employee D's virtual representative through a web interface or a dedicated application to obtain necessary information or request tasks.

[0110] Specifically, a user can ask the virtual agent for employee D, "What is the date of the next project meeting?", and the virtual agent will respond by referencing the schedule information in the data storage. The user can also have the virtual agent automatically reply to emails that they receive.

[0111] This system makes it possible to minimize disruption to work and ensure business continuity even when an employee is absent. An example of a specific prompt sentence is, "Please confirm the date of the next project meeting with the work clone of employee D."

[0112] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0113] Step 1: Data collection

[0114] The device collects work-related data from the software used by employees (e.g., Microsoft Outlook, Slack, Google Calendar, Microsoft Word). Specifically, the device connects to the API of each application to obtain emails, chat messages, schedule information, documents created, and more.

[0115] Input: Work-related data (emails, messages, etc.) entered by employees into each application

[0116] Data processing: The device obtains this data through the API and converts it into a unified format.

[0117] Output: Transfer the collected data (in a unified format) to the server.

[0118] Step 2: Data analysis and storage

[0119] The server receives business-related data sent from the device and analyzes it using natural language processing (NLP) technology, extracting necessary information and filtering out spam and irrelevant messages.

[0120] Input: Business-related data in a unified format sent from the terminal

[0121] Data processing: The server uses NLP to extract important information and filter out unnecessary data.

[0122] Output: Save the structured parsed data to data storage

[0123] Step 3: Train the machine learning model

[0124] The server trains a machine learning model based on the analyzed data stored in the data storage, learning the work patterns and response history of employees and generating a virtual representative based on that.

[0125] Input: Structured, parsed data stored in data storage

[0126] Data Computing: The server uses machine learning algorithms to learn patterns from the data and train the model.

[0127] Output: A trained machine learning model and a generated virtual representative

[0128] Step 4: Access and use the business clone

[0129] The user accesses the virtual agent through a web interface or a dedicated application. The user sends questions or tasks to the virtual agent, which then retrieves relevant information from data storage and responds.

[0130] Input: Questions and task requests that users enter into a web interface or dedicated application.

[0131] Data Computing: A virtual agent searches for relevant information from data storage and generates the best answer

[0132] Output: Response to the user (e.g., next meeting date) or task execution (e.g., email auto-reply)

[0133] Specific examples of operation

[0134] For example, if a user asks the virtual agent, "What is the date of the next project meeting?", the virtual agent will refer to the schedule information in the data storage and respond to the user, "The next project meeting is the day after tomorrow at 2:00 PM." The virtual agent can also automatically reply to configured emails, ensuring business continuity even when employees are absent.

[0135] (Application example 1)

[0136] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0137] It is important to prevent business interruptions and ensure business continuity even when employees are absent. Companies are also required to respond to security incidents quickly and accurately. However, employee absence or mistakes can delay response, which can lead to serious problems. Therefore, a system is needed that can take over employee work and streamline incident response.

[0138] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0139] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for users to access the work clones and perform their work via a web interface or a dedicated application, and means for analyzing data related to security incidents and proposing countermeasures. This ensures business continuity even when employees are absent, and enables quick and accurate responses to security incidents.

[0140] "Work-related data" is data collected from the applications and tools employees use to perform their jobs.

[0141] A "means for collecting data" is a method or device for collecting the necessary data from an application or tool.

[0142] "Means for analyzing data" refers to methods or devices that analyze collected data and extract useful information.

[0143] "Means for filtering information" refers to a method or device that eliminates unnecessary information from analyzed data and selects only the necessary information.

[0144] A "database" is a system for organizing and storing analyzed information.

[0145] A "machine learning model" is an algorithm or system that learns specific patterns or rules based on collected data.

[0146] A "work clone" is a program or agent that is generated based on a machine learning model and virtually performs the work of an employee.

[0147] A "web interface" is a screen that allows users to access and operate the system via the Internet.

[0148] A "specialized application" is software developed for a specific function or purpose.

[0149] A "security incident" is a security-related problem such as unauthorized access to or attacks on a system or network.

[0150] The "means for proposing countermeasures" refers to a method or device for providing appropriate countermeasures based on the analysis results.

[0151] "Means of carrying out business" refers to the methods and devices that business clones and users use to carry out business using the system.

[0152] This invention is a system for improving the efficiency of employee work and responding quickly and accurately to security incidents. This system collects work-related data from applications used by employees, analyzes it to extract necessary information, filters it, and stores it in a database. Furthermore, this database is used to train a machine learning model, which generates a work clone for each employee. Users can then access this work clone through a web interface or a dedicated application to perform their work.

[0153] System Configuration

[0154] 1. Hardware and Software Configuration

[0155] Device: An employee's computer or mobile device that serves as the data collection and user interface.

[0156] Server: A central device that performs data analysis and trains machine learning models.

[0157] Database: Data storage that stores and makes accessible analyzed information.

[0158] 2. Program Processing

[0159] Data collection

[0160] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from email clients and chat messages from chat tools.

[0161] Data analysis and storage

[0162] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data (such as spam emails and irrelevant chats) is also filtered out. The analyzed data is then structured and stored in a database.

[0163] Training machine learning models and generating business clones

[0164] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work on behalf of the employee when they are absent.

[0165] Accessing and using business clones

[0166] Users access the work clone through a web interface or a dedicated application. For example, when a user asks the work clone, "When is the next meeting scheduled?", the clone retrieves the relevant information from the database and responds.

[0167] Security Incident Response

[0168] The server analyzes data related to security incidents and uses natural language processing to understand the characteristics of the incidents. Based on the machine learning model, the business clone proposes appropriate countermeasures.

[0169] 3. Specific Examples

[0170] For example, if an employee is suddenly absent due to illness, the user can use a work clone to continue their daily work. The clone automatically replies to emails received on behalf of the employee, attends meetings, and creates minutes. In addition, if a security incident occurs, the clone can quickly propose countermeasures based on past response history and provide them to the user.

[0171] 4. Example prompts for generative AI models

[0172] "When is the next meeting scheduled?"

[0173] "What should we do to protect ourselves from the DDoS attack that occurred last night?"

[0174] "Can you give me more details about the security alert you received today?"

[0175] This system ensures business continuity even when employees are absent, and enables a swift and accurate response to security incidents.

[0176] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0177] Step 1:

[0178] The device connects to various applications used by employees (email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. At this time, the device obtains emails sent and received from each application, chat messages, schedule information, and created documents. For example, it collects the "contents of received emails" from email clients and the "chat history" from chat tools.

[0179] Input: Emails, chat messages, schedule information, various documents

[0180] Output: Collected work-related data

[0181] Step 2:

[0182] The server receives and analyzes the collected data, using natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information, and filters out unnecessary data (such as spam emails and irrelevant chats).

[0183] Input: Collected work-related data

[0184] Output: useful parsed information

[0185] Specific operation: Analyzes text data using NLP algorithms and extracts important keywords.

[0186] Step 3:

[0187] The server stores the analyzed information in a database, where the analysis results are stored in a structured format and made available for later processing.

[0188] Input: Parsed useful information

[0189] Output: Structured data stored in a database

[0190] Specific operation: Analysis results are stored using a database management system.

[0191] Step 4:

[0192] The server trains a machine learning model based on the data in the database. The trained model learns the work patterns and response history of employees and generates a virtual work clone.

[0193] Input: Structured data stored in a database

[0194] Output: Trained machine learning model and business clone

[0195] Specific operation: Train a model using a machine learning algorithm (e.g., K-nearest neighbors).

[0196] Step 5:

[0197] Users access the clone through a web interface or a dedicated application. When a user enters a query, the clone retrieves relevant information from a database and responds. Users can also request specific tasks.

[0198] Input: User query

[0199] Output: Answers and tasks performed by the business clone

[0200] What it does: Takes input from the user interface and generates an answer by searching a database.

[0201] Step 6:

[0202] The server analyzes data related to security incidents and proposes countermeasures. When a security incident occurs, the server provides appropriate countermeasures based on the analyzed data.

[0203] Input: Data about the security incident

[0204] Output: Incident response proposal

[0205] What it does: Analyzes incident data and derives response strategies using natural language processing and machine learning algorithms.

[0206] Step 7:

[0207] The business clone performs automatic email replies, meeting attendance, document proofreading, and security incident response proposals when employees are absent.

[0208] Input: A specific task or situation

[0209] Output: Business tasks and actions taken

[0210] Specific behavior: Based on trained behavioral patterns, the clone automatically performs the assigned task.

[0211] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0212] This invention combines a system in which AI automatically aggregates the work of individual employees and builds a database of their work content as clones with an emotion engine that recognizes user emotions. With this system, even when an employee is absent, the clone can be contacted to understand the work content and response history, and a response can be made according to the user's emotions. This not only enables smooth handover of work, but also enables effective communication that takes into account the user's emotions.

[0213] System Configuration

[0214] The system consists of the following main components:

[0215] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[0216] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[0217] 3. Database: Data storage that stores and makes accessible the analyzed data.

[0218] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[0219] 5. Emotion engine: A device or software that recognizes the user's emotions and adjusts the clone's response and response.

[0220] Program Operation

[0221] 1. Data Collection

[0222] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0223] 2. Data Analysis and Storage

[0224] The server receives the collected data and temporarily stores it. Next, it uses natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information. It also filters out unnecessary data (e.g., spam emails and irrelevant chats). The analyzed data is stored in a structured format in a database.

[0225] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[0226] 3. Training machine learning models and generating business clones

[0227] The server trains a machine learning model based on the data in the database. Past business data and interaction history are used for training. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[0228] For example, a clone can be created that learns employee A's past email reply patterns and what he or she has said in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[0229] 4. Integrating and using emotion engines

[0230] The emotion engine uses natural language processing and speech analysis to recognize the user's emotions in real time during a conversation, understanding the user's emotional state when asking a question and dynamically adjusting the response style and content of the business clone.

[0231] For example, if the emotion engine recognizes that the user is stressed from their tone of voice or text, the business clone will provide a more polite and reassuring response.

[0232] 5. Access to and Use of Business Clones

[0233] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone to obtain information. The user can also request the clone to perform specific tasks.

[0234] For example, a user can ask the clone of employee A, "What is the date of the next project meeting?", and the clone can respond by looking up the schedule information. The clone can also automatically reply to a specified email. In this case, the emotion engine recognizes the user's emotional state and adjusts the response accordingly.

[0235] This system effectively automates employee work and handles absences while taking into consideration the user's feelings, thereby improving the productivity and work efficiency of the entire organization, as well as the quality of communication between employees and users.

[0236] The processing flow will be explained below.

[0237] Step 1:

[0238] The device accesses various applications used by employees and collects work-related data. For example, the device collects email data sent and received from Microsoft Outlook and chat messages from Slack.

[0239] Step 2:

[0240] The data collected by the device is temporarily stored and periodically sent to the server, which starts the centralized management of data.

[0241] Step 3:

[0242] The server analyzes the data received. It uses natural language processing (NLP) to analyze the content of emails and chat messages and extracts the necessary information. For example, it can extract meeting details or task progress from the body of an email.

[0243] Step 4:

[0244] The server filters out unwanted data: spam and irrelevant chat messages are removed at this stage.

[0245] Step 5:

[0246] The server structures the extracted data and stores it in a database, organizing information such as email content, sender, recipient, date, and associated tags.

[0247] Step 6:

[0248] The server trains the machine learning model based on the database, using past work data and interaction history to learn employee work patterns and response histories.

[0249] Step 7:

[0250] The server uses the trained model to generate a clone of each employee's work, which is designed to perform the work even when the employee is absent based on the learned work patterns.

[0251] Step 8:

[0252] The server integrates an emotion engine, which uses natural language processing and speech analysis to recognize the user's emotions in real time.

[0253] Step 9:

[0254] Users access the business clone through a web interface or a dedicated application. When the user asks a question, the emotion engine recognizes the user's emotional state and adjusts the clone's response style accordingly.

[0255] Examples:

[0256] When a user asks the work clone, "What is the date of the next project meeting?", the emotion engine recognizes from the user's tone of voice that they are stressed, and the clone provides a more polite and reassuring response.

[0257] Step 10:

[0258] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[0259] Step 11:

[0260] The user can request a specific task from the work clone, such as replying to a specific email or requesting to attend a meeting.

[0261] Step 12:

[0262] The work clone performs the requested task (e.g., replying to emails, attending meetings, proofreading documents, etc.). By responding according to the user's emotional state through the emotion engine, work continuity is ensured and the quality of communication with the user is improved.

[0263] In this way, by having the above steps work in conjunction with each other, a system is realized that can smoothly take over the work of employees while responding to the user's emotions.

[0264] Example 2

[0265] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0266] With conventional systems, it was difficult to grasp the details of each employee's work and automate appropriate handovers and responses when they were absent. It was also difficult to communicate efficiently and appropriately in situations where consideration for the user's feelings was required. As a result, there was a risk of a decline in work efficiency and a deterioration in the quality of communication.

[0267] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0268] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for transmitting the work-related data to the server, analyzing the received work-related data, and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for integrating an emotion engine that recognizes user emotions in real time and adjusts the responses of the work clones, and means for accessing the work clones and sending questions and tasks to perform work. This allows for smooth handover of work even when employees are absent, and also enables effective communication that takes into account user emotions.

[0269] "Work-related data" refers to data collected from applications employees use in their daily work, such as emails, chat messages, schedule information, and documents they create.

[0270] "Server" means a device for receiving, storing, and analyzing collected business-related data and for training machine learning models.

[0271] "Database" refers to data storage where information extracted and filtered by the server is stored and maintained in a format that can be accessed later.

[0272] A "machine learning model" is an algorithm trained based on data in a database, which learns employees' work patterns and response history and generates virtual work clones.

[0273] A "work clone" is a virtual representative generated by a machine learning model that can perform work on behalf of an employee when they are absent.

[0274] The "emotion engine" refers to a function that uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user and adjusts the response of the business clone.

[0275] A "user" refers to a user who accesses a business clone to send a question or a task, obtain information, or perform a business.

[0276] This invention is a system that automatically aggregates the work of each employee and builds a database of clones of that work content, combined with an emotion engine that recognizes the user's emotions. Specifically, it includes the following components and processing steps:

[0277] 1. Hardware and software used

[0278] This system mainly consists of five main components: terminals, servers, databases, task clones, and emotion engines. Terminals are computers or mobile devices that employees use daily, and are equipped with email clients (e.g., Microsoft Outlook), chat tools (e.g., Slack), schedule management tools (e.g., Google Calendar), etc. The server is a central device that receives, stores, and analyzes collected data and trains machine learning models. The database is data storage that stores and makes accessible the analyzed data in a structured format. Task clones are generated based on machine learning models and are virtual representatives for each employee. The emotion engine is software that recognizes user emotions and adjusts the task clones' responses.

[0279] 2. Program Processing

[0280] Data collection

[0281] The device connects to various applications used by employees in their daily work and collects work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0282] Data transmission and reception

[0283] The device sends the collected data to the server, where it is encrypted before being sent.

[0284] Data Analysis and Filtering

[0285] The server temporarily stores the data it receives and uses natural language processing to analyze the content of emails and chat messages, extracting necessary information and filtering out unnecessary data.

[0286] Saving to a database

[0287] The server converts the parsed data into a structured format and stores it in a database.

[0288] Training a machine learning model

[0289] The server uses the data in the database to train a machine learning model, which then learns the work patterns and response history of employees and generates work clones.

[0290] Creating a business clone

[0291] The server uses a trained machine learning model to generate a work clone, a virtual representative of the employee.

[0292] Running the Emotion Engine

[0293] The server runs an emotion engine and uses natural language processing and voice analysis to recognize the user's emotions in real time during a dialogue with the user, thereby adjusting the response content and style of the business clone according to the user's emotional state.

[0294] Accessing the clone

[0295] Users access the work clone through a web interface or a dedicated application to ask questions or request tasks. For example, they can send a prompt such as "Please tell me the date of the next project meeting" to the clone, which then references the schedule information and responds.

[0296] 3. Specific Examples

[0297] When a user sends a prompt such as "Please tell me the date of the next project meeting," the work clone retrieves the relevant schedule information from the database and responds with "The next project meeting is next Tuesday at 3:00 p.m." At this time, the emotion engine analyzes the user's emotions from their tone of voice and text and provides an appropriate response style.

[0298] This system allows for smooth handover of work even when employees are absent and responds to users' emotions, improving productivity and work efficiency throughout the organization and improving the quality of communication between employees and users.

[0299] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0300] Step 1: Data collection

[0301] The device connects to various applications (e.g., email clients, chat tools, schedule management tools) used by employees in their daily work to collect work-related data. The input data is emails, chat messages, schedule information, and documents, and the output data is the collected data set. The device collects this data using APIs. For example, the device may use the Microsoft Outlook API to collect emails from the inbox, or the Slack API to collect chat messages.

[0302] Step 2: Sending and Receiving Data

[0303] The terminal sends the data it has collected to the server. The input data is business-related data collected by the terminal, and the output data is data transferred to the server. At this time, the data is encrypted and sent using the TLS protocol or similar. As a specific example, the terminal uses the TLS protocol to send collected email data and chat messages to the server.

[0304] Step 3: Data analysis and filtering

[0305] The server temporarily stores the data it receives and uses natural language processing (NLP) to analyze the content of emails and chat messages. The input data is the business-related data received by the server, and the output data is the filtered, necessary information. Specifically, the server uses the Python library NLTK to analyze the email content and extract important business-related keywords. It also uses SpamAssassin to filter spam emails.

[0306] Step 4: Saving to the Database

[0307] The server converts the parsed data into a structured format and stores it in a database. The input data is the parsed and filtered information, and the output data is the structured information stored in the database. A concrete example is when a server uses Parsey McParseface to parse email content and extract data such as project tasks, scheduled meetings, and action items and store it in a database.

[0308] Step 5: Train the machine learning model

[0309] The server uses the data in the database to train a machine learning model. The input data is the business data stored in the database, and the output data is the trained machine learning model. The server uses the Scikit-learn library to train a random forest model based on past business data.

[0310] Step 6: Creating a business clone

[0311] The server generates a job clone for each employee using the trained machine learning model. The input data is the trained machine learning model, and the output data is the generated job clone. For example, the server trains a neural network using the Keras framework to generate a job clone for employee A.

[0312] Step 7: Run the Emotion Engine

[0313] The server runs an emotion engine and uses natural language processing and speech analysis to recognize the user's emotions in real time during a dialogue with the user. The input data is the user's dialogue text and voice, and the output data is the analyzed user's emotional information. For example, the server uses the Python library TextBlob to analyze emotions from the user's text input, and if the user is feeling stressed, the work clone will provide a polite response.

[0314] Step 8: Access the clone

[0315] Users access the business clone through a web interface or a dedicated application. The input data are questions or task requests from the user, and the output data are the answers or tasks executed by the clone. For example, a user sends a prompt such as "Please tell me the date of the next project meeting" to the business clone, and the clone retrieves schedule information from a database and responds.

[0316] (Application example 2)

[0317] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0318] In today's business environment, it is often difficult to smoothly transfer work or respond to requests when employees are absent. Furthermore, when users contact employees, it is difficult for them to properly recognize and respond to the user's feelings. This situation is likely to lead to a decline in business efficiency and customer satisfaction. Furthermore, the increased burden caused by labor shortages is also a problem, and a solution to this issue is needed.

[0319] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0320] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating a work clone for each employee, emotion recognition means for recognizing the user's emotions and dynamically adjusting the response style and content of the work clone, and means for accessing the work clone and performing work. This allows for smooth handover and response even when an employee is absent, and enables appropriate response according to the user's emotions.

[0321] "Applications used by individual employees" refers to software such as email clients, chat tools, and schedule management applications that employees use in their daily work.

[0322] "Work-related data" is data such as emails, chat messages, schedule information, and documents that employees create or receive through these applications.

[0323] "Extraction and filtering means" refers to a method or device that performs processing to analyze and extract necessary information from business-related data and remove unnecessary information.

[0324] A "database" is a data storage system that stores collected and analyzed business-related data in a structured format and makes it accessible as needed.

[0325] "Training a machine learning model" is the process of using collected business-related data to create a model that can make predictions or classifications using machine learning algorithms.

[0326] A "work clone" is a virtual representative generated based on a machine learning model that mimics the work patterns and response history of a specific employee.

[0327] "Emotion recognition means" is a technology that analyzes emotions from a user's voice or text communication in real time and dynamically adjusts the response content.

[0328] "Means of performing business" refers to the mechanisms and processes by which a business clone performs business on behalf of an employee.

[0329] The present invention provides a system that uses a smart glasses application in a brick-and-mortar store to smoothly take over the work of an employee and respond to customers' emotions. Specific embodiments of the system are described below.

[0330] System Configuration

[0331] The system consists of the following main components:

[0332] 1. Devices: Smart glasses, computers, and mobile devices used by employees. These devices have email clients, chat tools, and schedule management applications installed.

[0333] 2. Server: A central device for collecting and analyzing data and training machine learning models. It mainly runs a database and an emotion recognition engine.

[0334] 3. Database: Stores the analyzed data and makes it accessible when needed.

[0335] 4. Work Clone: ​​A virtual representative for each employee, based on machine learning, is created to take over the employee's work when they are absent.

[0336] 5. Emotion Recognition Engine: Recognizes customer emotions in real time and adjusts the clone's response.

[0337] Program Operation

[0338] Data collection

[0339] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management tools) to collect work-related data. For example, the device collects email data from email clients and chat messages from chat tools.

[0340] Data analysis and storage

[0341] The server analyzes the collected data and temporarily stores it. Next, it uses natural language processing to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data is filtered out and stored in a structured format in a database.

[0342] Training machine learning models and generating business clones

[0343] The server trains a machine learning model based on the data in the database. It uses past work data and interaction history to learn employees' work patterns and response histories to generate virtual work clones. For example, a clone can be generated based on a specific employee's email response patterns or the content of comments made in meetings.

[0344] Integrating and using emotion recognition engines

[0345] The emotion recognition engine uses natural language processing and voice analysis to recognize customer emotions in real time during user interaction, understanding the emotional state of the customer when they ask a question and dynamically adjusting the response style and content of the work clone.

[0346] Specific examples of operation

[0347] Consider a scenario in which an employee wearing smart glasses serves a customer in a physical store. When a customer asks an employee a question, an emotion recognition engine analyzes the customer's emotions (e.g., anger, confusion, joy), and the work clone generates an appropriate response based on this. For example, if a customer appears confused and asks about the next event, the work clone will respond with a polite explanation and reassurance.

[0348] Prompt Sentence Examples

[0349] "Customer A seemed distressed and asked the store clerk about details of the next event. The store clerk recognized Customer A's feelings and politely explained the next event to Customer A, putting him at ease."

[0350] This system allows smooth handover of work and response even when employees are absent, and also enables appropriate responses based on customer emotions.Furthermore, customer satisfaction is expected to improve and operational efficiency can be expected to increase.

[0351] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0352] Step 1:

[0353] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management applications) to collect work-related data. The input is data from each application, and the output is the collected work-related data. Specific data collection operations involve the device collecting email data from email clients and retrieving chat messages from chat tools.

[0354] Step 2:

[0355] The server temporarily stores the collected business-related data. The input is the data collected from the terminal, and the output is the temporarily stored data. In this step, the specific operation of storing data on the server is to store emails and chat messages in temporary files in the database.

[0356] Step 3:

[0357] The server uses natural language processing to analyze the collected data, extracting and filtering the necessary information. The input is temporarily stored data, and the output is useful information after analysis. Specifically, it extracts important keywords from the body of an email and lists tasks from meeting minutes.

[0358] Step 4:

[0359] The server stores the analyzed data in a database. The input is useful information after analysis, and the output is data stored in a structured format. Specifically, in this step, important project progress information and task lists are stored in the database.

[0360] Step 5:

[0361] The server trains a machine learning model based on the data in the database. The input is the structured data in the database, and the output is the trained machine learning model. The specific operation of this step is to use past work data to learn each employee's work patterns and response history.

[0362] Step 6:

[0363] The server generates a work clone based on a trained machine learning model. The input is the trained machine learning model, and the output is a virtual representative for each employee. Specifically, a virtual work clone is created based on a specific employee's email response patterns and the content of comments made in meetings.

[0364] Step 7:

[0365] The emotion recognition engine uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user. The input is the user's voice or text, and the output is the user's emotional information. Specifically, emotions (e.g., anger, confusion, joy) are analyzed from the customer's tone of voice or text.

[0366] Step 8:

[0367] The server dynamically adjusts the response style and content of the business clone based on emotional information from the emotion recognition engine. The input is emotional information from the emotion recognition engine and response data from the business clone, and the output is an adjusted response message. Specifically, if the customer is confused, the business clone changes the response content to provide a more polite and reassuring answer.

[0368] Step 9:

[0369] The terminal provides the tailored response message to the customer through the smart glasses. The input is the tailored response message, and the output is the information the customer receives through the smart glasses. Specifically, the response of the business clone is displayed on the display of the smart glasses.

[0370] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0371] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0372] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0373] [Second embodiment]

[0374] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0375] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0376] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0377] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0378] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0379] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0380] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0381] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0382] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0383] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0384] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0385] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0386] This invention is a system in which AI automatically aggregates the work of each employee and builds the work content into a database as a clone. With this system, even when an employee is absent, the work content and response history can be understood by inquiring with the clone, enabling a smooth handover of work.

[0387] System Configuration

[0388] The system consists of the following main components:

[0389] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[0390] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[0391] 3. Database: Data storage that stores and makes accessible the analyzed data.

[0392] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[0393] Program Operation

[0394] 1. Data Collection

[0395] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from Microsoft Outlook, and also collects chat messages from Slack.

[0396] 2. Data Analysis and Storage

[0397] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unwanted data (e.g., spam emails and irrelevant chats) is also filtered out. The analyzed data is stored in a structured format in a database.

[0398] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[0399] 3. Training machine learning models and generating business clones

[0400] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[0401] For example, a clone can be created that learns employee A's past email reply patterns and the content of comments made in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[0402] 4. Access to and Use of Business Clones

[0403] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone, which retrieves the relevant information from a database and responds. The user can also request the clone to perform specific tasks.

[0404] For example, a user asks the clone of employee A, "Please tell me the date of the next project meeting," and the clone answers by looking up the schedule information.The clone also automatically replies to the specified email.

[0405] Specific examples

[0406] For example, if employee A suddenly becomes ill and cannot come to work, the user can use employee A's work clone to continue their daily work. The clone will automatically reply to emails received on employee A's behalf, attend meetings, and create minutes. In this way, it is possible to minimize work disruptions caused by employee A's absence and ensure business continuity.

[0407] This system effectively automates employee tasks and enables responses when employees are absent without human intervention, thereby improving productivity and operational efficiency across the organization.

[0408] The processing flow will be explained below.

[0409] Step 1:

[0410] The device accesses applications used by employees and collects work-related data, for example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0411] Step 2:

[0412] The server receives the collected data and temporarily stores it, allowing various data to be managed centrally.

[0413] Step 3:

[0414] The server analyzes the data it receives. It uses natural language processing (NLP) to analyze the content of emails and chat messages, extracting the necessary information, and filtering out unnecessary data (e.g., spam emails and irrelevant chats).

[0415] Step 4:

[0416] The server structures the parsed data, organizing it based on email content, sender, recipient, date, tagged keywords, etc.

[0417] Step 5:

[0418] The server stores the structured data in a database, which stores the data organized by employee for efficient access later.

[0419] Step 6:

[0420] The server reads data from the database and trains the machine learning model, using past business data and interaction history.

[0421] Step 7:

[0422] The server uses trained machine learning models to generate work clones for each employee, designed to mimic specific work patterns and behaviors.

[0423] Step 8:

[0424] Users access the work clone through a web interface or a dedicated application, and obtain information by asking the clone questions (e.g., "When is the next meeting scheduled?").

[0425] Step 9:

[0426] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[0427] Step 10:

[0428] Users can also assign tasks to the work clone, such as replying to a specific email or requesting attendance at a meeting.

[0429] Step 11:

[0430] The work clone performs the requested tasks (e.g., replying to emails, attending meetings, proofreading documents, etc.), which prevents work from stalling when an employee is absent.

[0431] In this way, each step works in coordination to realize a system that smoothly takes over employees' work and ensures business continuity throughout the organization.

[0432] Example 1

[0433] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0434] When an employee is absent, a means is needed to smoothly transfer work and prevent business stagnation. With conventional systems, it is difficult to grasp the content of employees' work and their response history, and work often does not proceed smoothly when employees are absent. This causes problems such as a decline in productivity and work efficiency throughout the organization. There is a need to solve this problem and provide a means to effectively continue work even when employees are absent.

[0435] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0436] In this invention, the server includes means for collecting work-related data from software used by each employee, means for analyzing the work-related data and extracting and filtering necessary information using natural language processing, means for storing the extracted information in a structured format in data storage, means for training a machine learning model based on the data in the data storage and generating a virtual representative for each employee, and means for accessing the virtual representative and performing work. This allows the virtual representative to automatically take over work even when the employee is absent, and to respond to emails, attend meetings, etc.

[0437] "Employee" refers to an individual who performs work within a company or organization.

[0438] "Software" refers to applications and programs that run on a computer.

[0439] "Business-related data" refers to information related to business, such as emails, messages, schedule data, and files created.

[0440] "Collection" refers to the act of gathering specific data or information.

[0441] "Natural language processing" refers to the technology that allows computers to analyze and understand human language.

[0442] "Extraction" refers to the act of isolating and extracting specific data or information.

[0443] "Filtering" refers to the act of removing unnecessary data and leaving only the necessary data.

[0444] "Data storage" refers to devices and systems for storing data.

[0445] A "structured format" refers to data that is organized according to rules and put into a form that can be easily handled.

[0446] A "machine learning model" refers to an algorithm or system that learns from data and performs pattern recognition and prediction.

[0447] A "virtual representative" refers to a software agent generated based on a machine learning model that mimics and performs the work of a specific employee.

[0448] "Access" refers to the act of connecting to, operating, or using data or a system.

[0449] "Performance" refers to the act of carrying out a specific task or job.

[0450] The system of this invention automatically aggregates employee tasks and builds a database of their tasks as virtual representatives. The system consists of the following main components:

[0451] 1. Devices: Computers and mobile devices used by employees on a daily basis. Each device has installed an email client, chat tool, schedule management application, etc. Examples include Microsoft Outlook, Slack, Google Calendar, and Microsoft Word.

[0452] 2. Server: A central device for storing and analyzing collected data and training machine learning models. This server receives, analyzes, filters, and stores data, trains machine learning models, and generates virtual representatives.

[0453] 3. Data storage: A database that stores and makes accessible the analyzed data. This data storage stores the data in a structured format.

[0454] 4. Virtual Representative: A virtual representative for each employee, generated based on machine learning, that performs tasks on behalf of the employee when they are not available.

[0455] As an example, consider the case where employee D suddenly falls ill and is unable to come to work. Employee D's device collects emails sent and received via Microsoft Outlook, which he or she uses daily, and Slack chat messages. The device transfers this data to a server. The server analyzes the transferred data using natural language processing technology, extracting and filtering important work-related information. The analyzed data is then stored in a structured format in data storage. The server then trains a machine learning model based on the data stored in the data storage. The trained model learns employee D's work patterns and response history and generates a virtual representative. Users can access employee D's virtual representative through a web interface or a dedicated application to obtain necessary information or request tasks.

[0456] Specifically, a user can ask the virtual agent for employee D, "What is the date of the next project meeting?", and the virtual agent will respond by referencing the schedule information in the data storage. The user can also have the virtual agent automatically reply to emails that they receive.

[0457] This system makes it possible to minimize disruption to work and ensure business continuity even when an employee is absent. An example of a specific prompt sentence is, "Please confirm the date of the next project meeting with the work clone of employee D."

[0458] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0459] Step 1: Data collection

[0460] The device collects work-related data from the software used by employees (e.g., Microsoft Outlook, Slack, Google Calendar, Microsoft Word). Specifically, the device connects to the API of each application to obtain emails, chat messages, schedule information, documents created, and more.

[0461] Input: Work-related data (emails, messages, etc.) entered by employees into each application

[0462] Data processing: The device obtains this data through the API and converts it into a unified format.

[0463] Output: Transfer the collected data (in a unified format) to the server.

[0464] Step 2: Data analysis and storage

[0465] The server receives business-related data sent from the device and analyzes it using natural language processing (NLP) technology, extracting necessary information and filtering out spam and irrelevant messages.

[0466] Input: Business-related data in a unified format sent from the terminal

[0467] Data processing: The server uses NLP to extract important information and filter out unnecessary data.

[0468] Output: Save the structured parsed data to data storage

[0469] Step 3: Train the machine learning model

[0470] The server trains a machine learning model based on the analyzed data stored in the data storage, learning the work patterns and response history of employees and generating a virtual representative based on that.

[0471] Input: Structured, parsed data stored in data storage

[0472] Data Computing: The server uses machine learning algorithms to learn patterns from the data and train the model.

[0473] Output: A trained machine learning model and a generated virtual representative

[0474] Step 4: Access and use the business clone

[0475] The user accesses the virtual agent through a web interface or a dedicated application. The user sends questions or tasks to the virtual agent, which then retrieves relevant information from data storage and responds.

[0476] Input: Questions and task requests that users enter into a web interface or dedicated application.

[0477] Data Computing: A virtual agent searches for relevant information from data storage and generates the best answer

[0478] Output: Response to the user (e.g., next meeting date) or task execution (e.g., email auto-reply)

[0479] Specific examples of operation

[0480] For example, if a user asks the virtual agent, "What is the date of the next project meeting?", the virtual agent will refer to the schedule information in the data storage and respond to the user, "The next project meeting is the day after tomorrow at 2:00 PM." The virtual agent can also automatically reply to configured emails, ensuring business continuity even when employees are absent.

[0481] (Application example 1)

[0482] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0483] It is important to prevent business interruptions and ensure business continuity even when employees are absent. Companies are also required to respond to security incidents quickly and accurately. However, employee absence or mistakes can delay response, which can lead to serious problems. Therefore, a system is needed that can take over employee work and streamline incident response.

[0484] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0485] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for users to access the work clones and perform their work via a web interface or a dedicated application, and means for analyzing data related to security incidents and proposing countermeasures. This ensures business continuity even when employees are absent, and enables quick and accurate responses to security incidents.

[0486] "Work-related data" is data collected from the applications and tools employees use to perform their jobs.

[0487] A "means for collecting data" is a method or device for collecting the necessary data from an application or tool.

[0488] "Means for analyzing data" refers to methods or devices that analyze collected data and extract useful information.

[0489] "Means for filtering information" refers to a method or device that eliminates unnecessary information from analyzed data and selects only the necessary information.

[0490] A "database" is a system for organizing and storing analyzed information.

[0491] A "machine learning model" is an algorithm or system that learns specific patterns or rules based on collected data.

[0492] A "work clone" is a program or agent that is generated based on a machine learning model and virtually performs the work of an employee.

[0493] A "web interface" is a screen that allows users to access and operate the system via the Internet.

[0494] A "specialized application" is software developed for a specific function or purpose.

[0495] A "security incident" is a security-related problem such as unauthorized access to or attacks on a system or network.

[0496] The "means for proposing countermeasures" refers to a method or device for providing appropriate countermeasures based on the analysis results.

[0497] "Means of carrying out business" refers to the methods and devices that business clones and users use to carry out business using the system.

[0498] This invention is a system for improving the efficiency of employee work and responding quickly and accurately to security incidents. This system collects work-related data from applications used by employees, analyzes it to extract necessary information, filters it, and stores it in a database. Furthermore, this database is used to train a machine learning model, which generates a work clone for each employee. Users can then access this work clone through a web interface or a dedicated application to perform their work.

[0499] System Configuration

[0500] 1. Hardware and Software Configuration

[0501] Device: An employee's computer or mobile device that serves as the data collection and user interface.

[0502] Server: A central device that performs data analysis and trains machine learning models.

[0503] Database: Data storage that stores and makes accessible analyzed information.

[0504] 2. Program Processing

[0505] Data collection

[0506] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from email clients and chat messages from chat tools.

[0507] Data analysis and storage

[0508] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data (such as spam emails and irrelevant chats) is also filtered out. The analyzed data is then structured and stored in a database.

[0509] Training machine learning models and generating business clones

[0510] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work on behalf of the employee when they are absent.

[0511] Accessing and using business clones

[0512] Users access the work clone through a web interface or a dedicated application. For example, when a user asks the work clone, "When is the next meeting scheduled?", the clone retrieves the relevant information from the database and responds.

[0513] Security Incident Response

[0514] The server analyzes data related to security incidents and uses natural language processing to understand the characteristics of the incidents. Based on the machine learning model, the business clone proposes appropriate countermeasures.

[0515] 3. Specific Examples

[0516] For example, if an employee is suddenly absent due to illness, the user can use a work clone to continue their daily work. The clone automatically replies to emails received on behalf of the employee, attends meetings, and creates minutes. In addition, if a security incident occurs, the clone can quickly propose countermeasures based on past response history and provide them to the user.

[0517] 4. Example prompts for generative AI models

[0518] "When is the next meeting scheduled?"

[0519] "What should we do to protect ourselves from the DDoS attack that occurred last night?"

[0520] "Can you give me more details about the security alert you received today?"

[0521] This system ensures business continuity even when employees are absent, and enables a swift and accurate response to security incidents.

[0522] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0523] Step 1:

[0524] The device connects to various applications used by employees (email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. At this time, the device obtains emails sent and received from each application, chat messages, schedule information, and created documents. For example, it collects the "contents of received emails" from email clients and the "chat history" from chat tools.

[0525] Input: Emails, chat messages, schedule information, various documents

[0526] Output: Collected work-related data

[0527] Step 2:

[0528] The server receives and analyzes the collected data, using natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information, and filters out unnecessary data (such as spam emails and irrelevant chats).

[0529] Input: Collected work-related data

[0530] Output: useful parsed information

[0531] Specific operation: Analyzes text data using NLP algorithms and extracts important keywords.

[0532] Step 3:

[0533] The server stores the analyzed information in a database, where the analysis results are stored in a structured format and made available for later processing.

[0534] Input: Parsed useful information

[0535] Output: Structured data stored in a database

[0536] Specific operation: Analysis results are stored using a database management system.

[0537] Step 4:

[0538] The server trains a machine learning model based on the data in the database. The trained model learns the work patterns and response history of employees and generates a virtual work clone.

[0539] Input: Structured data stored in a database

[0540] Output: Trained machine learning model and business clone

[0541] Specific operation: Train a model using a machine learning algorithm (e.g., K-nearest neighbors).

[0542] Step 5:

[0543] Users access the clone through a web interface or a dedicated application. When a user enters a query, the clone retrieves relevant information from a database and responds. Users can also request specific tasks.

[0544] Input: User query

[0545] Output: Answers and tasks performed by the business clone

[0546] What it does: Takes input from the user interface and generates an answer by searching a database.

[0547] Step 6:

[0548] The server analyzes data related to security incidents and proposes countermeasures. When a security incident occurs, the server provides appropriate countermeasures based on the analyzed data.

[0549] Input: Data about the security incident

[0550] Output: Incident response proposal

[0551] What it does: Analyzes incident data and derives response strategies using natural language processing and machine learning algorithms.

[0552] Step 7:

[0553] The business clone performs automatic email replies, meeting attendance, document proofreading, and security incident response proposals when employees are absent.

[0554] Input: A specific task or situation

[0555] Output: Business tasks and actions taken

[0556] Specific behavior: Based on trained behavioral patterns, the clone automatically performs the assigned task.

[0557] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0558] This invention combines a system in which AI automatically aggregates the work of individual employees and builds a database of their work content as clones with an emotion engine that recognizes user emotions. With this system, even when an employee is absent, the clone can be contacted to understand the work content and response history, and a response can be made according to the user's emotions. This not only enables smooth handover of work, but also enables effective communication that takes into account the user's emotions.

[0559] System Configuration

[0560] The system consists of the following main components:

[0561] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[0562] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[0563] 3. Database: Data storage that stores and makes accessible the analyzed data.

[0564] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[0565] 5. Emotion engine: A device or software that recognizes the user's emotions and adjusts the clone's response and response.

[0566] Program Operation

[0567] 1. Data Collection

[0568] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0569] 2. Data Analysis and Storage

[0570] The server receives the collected data and temporarily stores it. Next, it uses natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information. It also filters out unnecessary data (e.g., spam emails and irrelevant chats). The analyzed data is stored in a structured format in a database.

[0571] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[0572] 3. Training machine learning models and generating business clones

[0573] The server trains a machine learning model based on the data in the database. Past business data and interaction history are used for training. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[0574] For example, a clone can be created that learns employee A's past email reply patterns and what he or she has said in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[0575] 4. Integrating and using emotion engines

[0576] The emotion engine uses natural language processing and speech analysis to recognize the user's emotions in real time during a conversation, understanding the user's emotional state when asking a question and dynamically adjusting the response style and content of the business clone.

[0577] For example, if the emotion engine recognizes that the user is stressed from their tone of voice or text, the business clone will provide a more polite and reassuring response.

[0578] 5. Access to and Use of Business Clones

[0579] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone to obtain information. The user can also request the clone to perform specific tasks.

[0580] For example, a user can ask the clone of employee A, "What is the date of the next project meeting?", and the clone can respond by looking up the schedule information. The clone can also automatically reply to a specified email. In this case, the emotion engine recognizes the user's emotional state and adjusts the response accordingly.

[0581] This system effectively automates employee work and handles absences while taking into consideration the user's feelings, thereby improving the productivity and work efficiency of the entire organization, as well as the quality of communication between employees and users.

[0582] The processing flow will be explained below.

[0583] Step 1:

[0584] The device accesses various applications used by employees and collects work-related data. For example, the device collects email data sent and received from Microsoft Outlook and chat messages from Slack.

[0585] Step 2:

[0586] The data collected by the device is temporarily stored and periodically sent to the server, which starts the centralized management of data.

[0587] Step 3:

[0588] The server analyzes the data received. It uses natural language processing (NLP) to analyze the content of emails and chat messages and extracts the necessary information. For example, it can extract meeting details or task progress from the body of an email.

[0589] Step 4:

[0590] The server filters out unwanted data: spam and irrelevant chat messages are removed at this stage.

[0591] Step 5:

[0592] The server structures the extracted data and stores it in a database, organizing information such as email content, sender, recipient, date, and associated tags.

[0593] Step 6:

[0594] The server trains the machine learning model based on the database, using past work data and interaction history to learn employee work patterns and response histories.

[0595] Step 7:

[0596] The server uses the trained model to generate a clone of each employee's work, which is designed to perform the work even when the employee is absent based on the learned work patterns.

[0597] Step 8:

[0598] The server integrates an emotion engine, which uses natural language processing and speech analysis to recognize the user's emotions in real time.

[0599] Step 9:

[0600] Users access the business clone through a web interface or a dedicated application. When the user asks a question, the emotion engine recognizes the user's emotional state and adjusts the clone's response style accordingly.

[0601] Examples:

[0602] When a user asks the work clone, "What is the date of the next project meeting?", the emotion engine recognizes from the user's tone of voice that they are stressed, and the clone provides a more polite and reassuring response.

[0603] Step 10:

[0604] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[0605] Step 11:

[0606] The user can request a specific task from the work clone, such as replying to a specific email or requesting to attend a meeting.

[0607] Step 12:

[0608] The work clone performs the requested task (e.g., replying to emails, attending meetings, proofreading documents, etc.). By responding according to the user's emotional state through the emotion engine, work continuity is ensured and the quality of communication with the user is improved.

[0609] In this way, by having the above steps work in conjunction with each other, a system is realized that can smoothly take over the work of employees while responding to the user's emotions.

[0610] Example 2

[0611] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0612] With conventional systems, it was difficult to grasp the details of each employee's work and automate appropriate handovers and responses when they were absent. It was also difficult to communicate efficiently and appropriately in situations where consideration for the user's feelings was required. As a result, there was a risk of a decline in work efficiency and a deterioration in the quality of communication.

[0613] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0614] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for transmitting the work-related data to the server, analyzing the received work-related data, and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for integrating an emotion engine that recognizes user emotions in real time and adjusts the responses of the work clones, and means for accessing the work clones and sending questions and tasks to perform work. This allows for smooth handover of work even when employees are absent, and also enables effective communication that takes into account user emotions.

[0615] "Work-related data" refers to data collected from applications employees use in their daily work, such as emails, chat messages, schedule information, and documents they create.

[0616] "Server" means a device for receiving, storing, and analyzing collected business-related data and for training machine learning models.

[0617] "Database" refers to data storage where information extracted and filtered by the server is stored and maintained in a format that can be accessed later.

[0618] A "machine learning model" is an algorithm trained based on data in a database, which learns employees' work patterns and response history and generates virtual work clones.

[0619] A "work clone" is a virtual representative generated by a machine learning model that can perform work on behalf of an employee when they are absent.

[0620] The "emotion engine" refers to a function that uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user and adjusts the response of the business clone.

[0621] A "user" refers to a user who accesses a business clone to send a question or a task, obtain information, or perform a business.

[0622] This invention is a system that automatically aggregates the work of each employee and builds a database of clones of that work content, combined with an emotion engine that recognizes the user's emotions. Specifically, it includes the following components and processing steps:

[0623] 1. Hardware and software used

[0624] This system mainly consists of five main components: terminals, servers, databases, task clones, and emotion engines. Terminals are computers or mobile devices that employees use daily, and are equipped with email clients (e.g., Microsoft Outlook), chat tools (e.g., Slack), schedule management tools (e.g., Google Calendar), etc. The server is a central device that receives, stores, and analyzes collected data and trains machine learning models. The database is data storage that stores and makes accessible the analyzed data in a structured format. Task clones are generated based on machine learning models and are virtual representatives for each employee. The emotion engine is software that recognizes user emotions and adjusts the task clones' responses.

[0625] 2. Program Processing

[0626] Data collection

[0627] The device connects to various applications used by employees in their daily work and collects work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0628] Data transmission and reception

[0629] The device sends the collected data to the server, where it is encrypted before being sent.

[0630] Data Analysis and Filtering

[0631] The server temporarily stores the data it receives and uses natural language processing to analyze the content of emails and chat messages, extracting necessary information and filtering out unnecessary data.

[0632] Saving to a database

[0633] The server converts the parsed data into a structured format and stores it in a database.

[0634] Training a machine learning model

[0635] The server uses the data in the database to train a machine learning model, which then learns the work patterns and response history of employees and generates work clones.

[0636] Creating a business clone

[0637] The server uses a trained machine learning model to generate a work clone, a virtual representative of the employee.

[0638] Running the Emotion Engine

[0639] The server runs an emotion engine and uses natural language processing and voice analysis to recognize the user's emotions in real time during a dialogue with the user, thereby adjusting the response content and style of the business clone according to the user's emotional state.

[0640] Accessing the clone

[0641] Users access the work clone through a web interface or a dedicated application to ask questions or request tasks. For example, they can send a prompt such as "Please tell me the date of the next project meeting" to the clone, which then references the schedule information and responds.

[0642] 3. Specific Examples

[0643] When a user sends a prompt such as "Please tell me the date of the next project meeting," the work clone retrieves the relevant schedule information from the database and responds with "The next project meeting is next Tuesday at 3:00 p.m." At this time, the emotion engine analyzes the user's emotions from their tone of voice and text and provides an appropriate response style.

[0644] This system allows for smooth handover of work even when employees are absent and responds to users' emotions, improving productivity and work efficiency throughout the organization and improving the quality of communication between employees and users.

[0645] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0646] Step 1: Data collection

[0647] The device connects to various applications (e.g., email clients, chat tools, schedule management tools) used by employees in their daily work to collect work-related data. The input data is emails, chat messages, schedule information, and documents, and the output data is the collected data set. The device collects this data using APIs. For example, the device may use the Microsoft Outlook API to collect emails from the inbox, or the Slack API to collect chat messages.

[0648] Step 2: Sending and Receiving Data

[0649] The terminal sends the data it has collected to the server. The input data is business-related data collected by the terminal, and the output data is data transferred to the server. At this time, the data is encrypted and sent using the TLS protocol or similar. As a specific example, the terminal uses the TLS protocol to send collected email data and chat messages to the server.

[0650] Step 3: Data analysis and filtering

[0651] The server temporarily stores the data it receives and uses natural language processing (NLP) to analyze the content of emails and chat messages. The input data is the business-related data received by the server, and the output data is the filtered, necessary information. Specifically, the server uses the Python library NLTK to analyze the email content and extract important business-related keywords. It also uses SpamAssassin to filter spam emails.

[0652] Step 4: Saving to the Database

[0653] The server converts the parsed data into a structured format and stores it in a database. The input data is the parsed and filtered information, and the output data is the structured information stored in the database. A concrete example is when a server uses Parsey McParseface to parse email content and extract data such as project tasks, scheduled meetings, and action items and store it in a database.

[0654] Step 5: Train the machine learning model

[0655] The server uses the data in the database to train a machine learning model. The input data is the business data stored in the database, and the output data is the trained machine learning model. The server uses the Scikit-learn library to train a random forest model based on past business data.

[0656] Step 6: Creating a business clone

[0657] The server generates a job clone for each employee using the trained machine learning model. The input data is the trained machine learning model, and the output data is the generated job clone. For example, the server trains a neural network using the Keras framework to generate a job clone for employee A.

[0658] Step 7: Run the Emotion Engine

[0659] The server runs an emotion engine and uses natural language processing and speech analysis to recognize the user's emotions in real time during a dialogue with the user. The input data is the user's dialogue text and voice, and the output data is the analyzed user's emotional information. For example, the server uses the Python library TextBlob to analyze emotions from the user's text input, and if the user is feeling stressed, the work clone will provide a polite response.

[0660] Step 8: Access the clone

[0661] Users access the business clone through a web interface or a dedicated application. The input data are questions or task requests from the user, and the output data are the answers or tasks executed by the clone. For example, a user sends a prompt such as "Please tell me the date of the next project meeting" to the business clone, and the clone retrieves schedule information from a database and responds.

[0662] (Application example 2)

[0663] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0664] In today's business environment, it is often difficult to smoothly transfer work or respond to requests when employees are absent. Furthermore, when users contact employees, it is difficult for them to properly recognize and respond to the user's feelings. This situation is likely to lead to a decline in business efficiency and customer satisfaction. Furthermore, the increased burden caused by labor shortages is also a problem, and a solution to this issue is needed.

[0665] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0666] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating a work clone for each employee, emotion recognition means for recognizing the user's emotions and dynamically adjusting the response style and content of the work clone, and means for accessing the work clone and performing work. This allows for smooth handover and response even when an employee is absent, and enables appropriate response according to the user's emotions.

[0667] "Applications used by individual employees" refers to software such as email clients, chat tools, and schedule management applications that employees use in their daily work.

[0668] "Work-related data" is data such as emails, chat messages, schedule information, and documents that employees create or receive through these applications.

[0669] "Extraction and filtering means" refers to a method or device that performs processing to analyze and extract necessary information from business-related data and remove unnecessary information.

[0670] A "database" is a data storage system that stores collected and analyzed business-related data in a structured format and makes it accessible as needed.

[0671] "Training a machine learning model" is the process of using collected business-related data to create a model that can make predictions or classifications using machine learning algorithms.

[0672] A "work clone" is a virtual representative generated based on a machine learning model that mimics the work patterns and response history of a specific employee.

[0673] "Emotion recognition means" is a technology that analyzes emotions from a user's voice or text communication in real time and dynamically adjusts the response content.

[0674] "Means of performing business" refers to the mechanisms and processes by which a business clone performs business on behalf of an employee.

[0675] The present invention provides a system that uses a smart glasses application in a brick-and-mortar store to smoothly take over the work of an employee and respond to customers' emotions. Specific embodiments of the system are described below.

[0676] System Configuration

[0677] The system consists of the following main components:

[0678] 1. Devices: Smart glasses, computers, and mobile devices used by employees. These devices have email clients, chat tools, and schedule management applications installed.

[0679] 2. Server: A central device for collecting and analyzing data and training machine learning models. It mainly runs a database and an emotion recognition engine.

[0680] 3. Database: Stores the analyzed data and makes it accessible when needed.

[0681] 4. Work Clone: ​​A virtual representative for each employee, based on machine learning, is created to take over the employee's work when they are absent.

[0682] 5. Emotion Recognition Engine: Recognizes customer emotions in real time and adjusts the clone's response.

[0683] Program Operation

[0684] Data collection

[0685] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management tools) to collect work-related data. For example, the device collects email data from email clients and chat messages from chat tools.

[0686] Data analysis and storage

[0687] The server analyzes the collected data and temporarily stores it. Next, it uses natural language processing to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data is filtered out and stored in a structured format in a database.

[0688] Training machine learning models and generating business clones

[0689] The server trains a machine learning model based on the data in the database. It uses past work data and interaction history to learn employees' work patterns and response histories to generate virtual work clones. For example, a clone can be generated based on a specific employee's email response patterns or the content of comments made in meetings.

[0690] Integrating and using emotion recognition engines

[0691] The emotion recognition engine uses natural language processing and voice analysis to recognize customer emotions in real time during user interaction, understanding the emotional state of the customer when they ask a question and dynamically adjusting the response style and content of the work clone.

[0692] Specific examples of operation

[0693] Consider a scenario in which an employee wearing smart glasses serves a customer in a physical store. When a customer asks an employee a question, an emotion recognition engine analyzes the customer's emotions (e.g., anger, confusion, joy), and the work clone generates an appropriate response based on this. For example, if a customer appears confused and asks about the next event, the work clone will respond with a polite explanation and reassurance.

[0694] Prompt Sentence Examples

[0695] "Customer A seemed distressed and asked the store clerk about details of the next event. The store clerk recognized Customer A's feelings and politely explained the next event to Customer A, putting him at ease."

[0696] This system allows smooth handover of work and response even when employees are absent, and also enables appropriate responses based on customer emotions.Furthermore, customer satisfaction is expected to improve and operational efficiency can be expected to increase.

[0697] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0698] Step 1:

[0699] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management applications) to collect work-related data. The input is data from each application, and the output is the collected work-related data. Specific data collection operations involve the device collecting email data from email clients and retrieving chat messages from chat tools.

[0700] Step 2:

[0701] The server temporarily stores the collected business-related data. The input is the data collected from the terminal, and the output is the temporarily stored data. In this step, the specific operation of storing data on the server is to store emails and chat messages in temporary files in the database.

[0702] Step 3:

[0703] The server uses natural language processing to analyze the collected data, extracting and filtering the necessary information. The input is temporarily stored data, and the output is useful information after analysis. Specifically, it extracts important keywords from the body of an email and lists tasks from meeting minutes.

[0704] Step 4:

[0705] The server stores the analyzed data in a database. The input is useful information after analysis, and the output is data stored in a structured format. Specifically, in this step, important project progress information and task lists are stored in the database.

[0706] Step 5:

[0707] The server trains a machine learning model based on the data in the database. The input is the structured data in the database, and the output is the trained machine learning model. The specific operation of this step is to use past work data to learn each employee's work patterns and response history.

[0708] Step 6:

[0709] The server generates a work clone based on a trained machine learning model. The input is the trained machine learning model, and the output is a virtual representative for each employee. Specifically, a virtual work clone is created based on a specific employee's email response patterns and the content of comments made in meetings.

[0710] Step 7:

[0711] The emotion recognition engine uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user. The input is the user's voice or text, and the output is the user's emotional information. Specifically, emotions (e.g., anger, confusion, joy) are analyzed from the customer's tone of voice or text.

[0712] Step 8:

[0713] The server dynamically adjusts the response style and content of the business clone based on emotional information from the emotion recognition engine. The input is emotional information from the emotion recognition engine and response data from the business clone, and the output is an adjusted response message. Specifically, if the customer is confused, the business clone changes the response content to provide a more polite and reassuring answer.

[0714] Step 9:

[0715] The terminal provides the tailored response message to the customer through the smart glasses. The input is the tailored response message, and the output is the information the customer receives through the smart glasses. Specifically, the response of the business clone is displayed on the display of the smart glasses.

[0716] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0717] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0718] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0719] [Third embodiment]

[0720] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0721] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0722] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0723] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0724] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0725] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0726] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0727] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0728] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0729] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0730] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0731] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0732] This invention is a system in which AI automatically aggregates the work of each employee and builds the work content into a database as a clone. With this system, even when an employee is absent, the work content and response history can be understood by inquiring with the clone, enabling a smooth handover of work.

[0733] System Configuration

[0734] The system consists of the following main components:

[0735] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[0736] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[0737] 3. Database: Data storage that stores and makes accessible the analyzed data.

[0738] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[0739] Program Operation

[0740] 1. Data Collection

[0741] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from Microsoft Outlook, and also collects chat messages from Slack.

[0742] 2. Data Analysis and Storage

[0743] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unwanted data (e.g., spam emails and irrelevant chats) is also filtered out. The analyzed data is stored in a structured format in a database.

[0744] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[0745] 3. Training machine learning models and generating business clones

[0746] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[0747] For example, a clone can be created that learns employee A's past email reply patterns and the content of comments made in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[0748] 4. Access to and Use of Business Clones

[0749] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone, which retrieves the relevant information from a database and responds. The user can also request the clone to perform specific tasks.

[0750] For example, a user asks the clone of employee A, "Please tell me the date of the next project meeting," and the clone answers by looking up the schedule information.The clone also automatically replies to the specified email.

[0751] Specific examples

[0752] For example, if employee A suddenly becomes ill and cannot come to work, the user can use employee A's work clone to continue their daily work. The clone will automatically reply to emails received on employee A's behalf, attend meetings, and create minutes. In this way, it is possible to minimize work disruptions caused by employee A's absence and ensure business continuity.

[0753] This system effectively automates employee tasks and enables responses when employees are absent without human intervention, thereby improving productivity and operational efficiency across the organization.

[0754] The processing flow will be explained below.

[0755] Step 1:

[0756] The device accesses applications used by employees and collects work-related data, for example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0757] Step 2:

[0758] The server receives the collected data and temporarily stores it, allowing various data to be managed centrally.

[0759] Step 3:

[0760] The server analyzes the data it receives. It uses natural language processing (NLP) to analyze the content of emails and chat messages, extracting the necessary information, and filtering out unnecessary data (e.g., spam emails and irrelevant chats).

[0761] Step 4:

[0762] The server structures the parsed data, organizing it based on email content, sender, recipient, date, tagged keywords, etc.

[0763] Step 5:

[0764] The server stores the structured data in a database, which stores the data organized by employee for efficient access later.

[0765] Step 6:

[0766] The server reads data from the database and trains the machine learning model, using past business data and interaction history.

[0767] Step 7:

[0768] The server uses trained machine learning models to generate work clones for each employee, designed to mimic specific work patterns and behaviors.

[0769] Step 8:

[0770] Users access the work clone through a web interface or a dedicated application, and obtain information by asking the clone questions (e.g., "When is the next meeting scheduled?").

[0771] Step 9:

[0772] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[0773] Step 10:

[0774] Users can also assign tasks to the work clone, such as replying to a specific email or requesting attendance at a meeting.

[0775] Step 11:

[0776] The work clone performs the requested tasks (e.g., replying to emails, attending meetings, proofreading documents, etc.), which prevents work from stalling when an employee is absent.

[0777] In this way, each step works in coordination to realize a system that smoothly takes over employees' work and ensures business continuity throughout the organization.

[0778] Example 1

[0779] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0780] When an employee is absent, a means is needed to smoothly transfer work and prevent business stagnation. With conventional systems, it is difficult to grasp the content of employees' work and their response history, and work often does not proceed smoothly when employees are absent. This causes problems such as a decline in productivity and work efficiency throughout the organization. There is a need to solve this problem and provide a means to effectively continue work even when employees are absent.

[0781] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0782] In this invention, the server includes means for collecting work-related data from software used by each employee, means for analyzing the work-related data and extracting and filtering necessary information using natural language processing, means for storing the extracted information in a structured format in data storage, means for training a machine learning model based on the data in the data storage and generating a virtual representative for each employee, and means for accessing the virtual representative and performing work. This allows the virtual representative to automatically take over work even when the employee is absent, and to respond to emails, attend meetings, etc.

[0783] "Employee" refers to an individual who performs work within a company or organization.

[0784] "Software" refers to applications and programs that run on a computer.

[0785] "Business-related data" refers to information related to business, such as emails, messages, schedule data, and files created.

[0786] "Collection" refers to the act of gathering specific data or information.

[0787] "Natural language processing" refers to the technology that allows computers to analyze and understand human language.

[0788] "Extraction" refers to the act of isolating and extracting specific data or information.

[0789] "Filtering" refers to the act of removing unnecessary data and leaving only the necessary data.

[0790] "Data storage" refers to devices and systems for storing data.

[0791] A "structured format" refers to data that is organized according to rules and put into a form that can be easily handled.

[0792] A "machine learning model" refers to an algorithm or system that learns from data and performs pattern recognition and prediction.

[0793] A "virtual representative" refers to a software agent generated based on a machine learning model that mimics and performs the work of a specific employee.

[0794] "Access" refers to the act of connecting to, operating, or using data or a system.

[0795] "Performance" refers to the act of carrying out a specific task or job.

[0796] The system of this invention automatically aggregates employee tasks and builds a database of their tasks as virtual representatives. The system consists of the following main components:

[0797] 1. Devices: Computers and mobile devices used by employees on a daily basis. Each device has installed an email client, chat tool, schedule management application, etc. Examples include Microsoft Outlook, Slack, Google Calendar, and Microsoft Word.

[0798] 2. Server: A central device for storing and analyzing collected data and training machine learning models. This server receives, analyzes, filters, and stores data, trains machine learning models, and generates virtual representatives.

[0799] 3. Data storage: A database that stores and makes accessible the analyzed data. This data storage stores the data in a structured format.

[0800] 4. Virtual Representative: A virtual representative for each employee, generated based on machine learning, that performs tasks on behalf of the employee when they are not available.

[0801] As an example, consider the case where employee D suddenly falls ill and is unable to come to work. Employee D's device collects emails sent and received via Microsoft Outlook, which he or she uses daily, and Slack chat messages. The device transfers this data to a server. The server analyzes the transferred data using natural language processing technology, extracting and filtering important work-related information. The analyzed data is then stored in a structured format in data storage. The server then trains a machine learning model based on the data stored in the data storage. The trained model learns employee D's work patterns and response history and generates a virtual representative. Users can access employee D's virtual representative through a web interface or a dedicated application to obtain necessary information or request tasks.

[0802] Specifically, a user can ask the virtual agent for employee D, "What is the date of the next project meeting?", and the virtual agent will respond by referencing the schedule information in the data storage. The user can also have the virtual agent automatically reply to emails that they receive.

[0803] This system makes it possible to minimize disruption to work and ensure business continuity even when an employee is absent. An example of a specific prompt sentence is, "Please confirm the date of the next project meeting with the work clone of employee D."

[0804] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0805] Step 1: Data collection

[0806] The device collects work-related data from the software used by employees (e.g., Microsoft Outlook, Slack, Google Calendar, Microsoft Word). Specifically, the device connects to the API of each application to obtain emails, chat messages, schedule information, documents created, and more.

[0807] Input: Work-related data (emails, messages, etc.) entered by employees into each application

[0808] Data processing: The device obtains this data through the API and converts it into a unified format.

[0809] Output: Transfer the collected data (in a unified format) to the server.

[0810] Step 2: Data analysis and storage

[0811] The server receives business-related data sent from the device and analyzes it using natural language processing (NLP) technology, extracting necessary information and filtering out spam and irrelevant messages.

[0812] Input: Business-related data in a unified format sent from the terminal

[0813] Data processing: The server uses NLP to extract important information and filter out unnecessary data.

[0814] Output: Save the structured parsed data to data storage

[0815] Step 3: Train the machine learning model

[0816] The server trains a machine learning model based on the analyzed data stored in the data storage, learning the work patterns and response history of employees and generating a virtual representative based on that.

[0817] Input: Structured, parsed data stored in data storage

[0818] Data Computing: The server uses machine learning algorithms to learn patterns from the data and train the model.

[0819] Output: A trained machine learning model and a generated virtual representative

[0820] Step 4: Access and use the business clone

[0821] The user accesses the virtual agent through a web interface or a dedicated application. The user sends questions or tasks to the virtual agent, which then retrieves relevant information from data storage and responds.

[0822] Input: Questions and task requests that users enter into a web interface or dedicated application.

[0823] Data Computing: A virtual agent searches for relevant information from data storage and generates the best answer

[0824] Output: Response to the user (e.g., next meeting date) or task execution (e.g., email auto-reply)

[0825] Specific examples of operation

[0826] For example, if a user asks the virtual agent, "What is the date of the next project meeting?", the virtual agent will refer to the schedule information in the data storage and respond to the user, "The next project meeting is the day after tomorrow at 2:00 PM." The virtual agent can also automatically reply to configured emails, ensuring business continuity even when employees are absent.

[0827] (Application example 1)

[0828] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0829] It is important to prevent business interruptions and ensure business continuity even when employees are absent. Companies are also required to respond to security incidents quickly and accurately. However, employee absence or mistakes can delay response, which can lead to serious problems. Therefore, a system is needed that can take over employee work and streamline incident response.

[0830] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0831] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for users to access the work clones and perform their work via a web interface or a dedicated application, and means for analyzing data related to security incidents and proposing countermeasures. This ensures business continuity even when employees are absent, and enables quick and accurate responses to security incidents.

[0832] "Work-related data" is data collected from the applications and tools employees use to perform their jobs.

[0833] A "means for collecting data" is a method or device for collecting the necessary data from an application or tool.

[0834] "Means for analyzing data" refers to methods or devices that analyze collected data and extract useful information.

[0835] "Means for filtering information" refers to a method or device that eliminates unnecessary information from analyzed data and selects only the necessary information.

[0836] A "database" is a system for organizing and storing analyzed information.

[0837] A "machine learning model" is an algorithm or system that learns specific patterns or rules based on collected data.

[0838] A "work clone" is a program or agent that is generated based on a machine learning model and virtually performs the work of an employee.

[0839] A "web interface" is a screen that allows users to access and operate the system via the Internet.

[0840] A "specialized application" is software developed for a specific function or purpose.

[0841] A "security incident" is a security-related problem such as unauthorized access to or attacks on a system or network.

[0842] The "means for proposing countermeasures" refers to a method or device for providing appropriate countermeasures based on the analysis results.

[0843] "Means of carrying out business" refers to the methods and devices that business clones and users use to carry out business using the system.

[0844] This invention is a system for improving the efficiency of employee work and responding quickly and accurately to security incidents. This system collects work-related data from applications used by employees, analyzes it to extract necessary information, filters it, and stores it in a database. Furthermore, this database is used to train a machine learning model, which generates a work clone for each employee. Users can then access this work clone through a web interface or a dedicated application to perform their work.

[0845] System Configuration

[0846] 1. Hardware and Software Configuration

[0847] Device: An employee's computer or mobile device that serves as the data collection and user interface.

[0848] Server: A central device that performs data analysis and trains machine learning models.

[0849] Database: Data storage that stores and makes accessible analyzed information.

[0850] 2. Program Processing

[0851] Data collection

[0852] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from email clients and chat messages from chat tools.

[0853] Data analysis and storage

[0854] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data (such as spam emails and irrelevant chats) is also filtered out. The analyzed data is then structured and stored in a database.

[0855] Training machine learning models and generating business clones

[0856] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work on behalf of the employee when they are absent.

[0857] Accessing and using business clones

[0858] Users access the work clone through a web interface or a dedicated application. For example, when a user asks the work clone, "When is the next meeting scheduled?", the clone retrieves the relevant information from the database and responds.

[0859] Security Incident Response

[0860] The server analyzes data related to security incidents and uses natural language processing to understand the characteristics of the incidents. Based on the machine learning model, the business clone proposes appropriate countermeasures.

[0861] 3. Specific Examples

[0862] For example, if an employee is suddenly absent due to illness, the user can use a work clone to continue their daily work. The clone automatically replies to emails received on behalf of the employee, attends meetings, and creates minutes. In addition, if a security incident occurs, the clone can quickly propose countermeasures based on past response history and provide them to the user.

[0863] 4. Example prompts for generative AI models

[0864] "When is the next meeting scheduled?"

[0865] "What should we do to protect ourselves from the DDoS attack that occurred last night?"

[0866] "Can you give me more details about the security alert you received today?"

[0867] This system ensures business continuity even when employees are absent, and enables a swift and accurate response to security incidents.

[0868] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0869] Step 1:

[0870] The device connects to various applications used by employees (email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. At this time, the device obtains emails sent and received from each application, chat messages, schedule information, and created documents. For example, it collects the "contents of received emails" from email clients and the "chat history" from chat tools.

[0871] Input: Emails, chat messages, schedule information, various documents

[0872] Output: Collected work-related data

[0873] Step 2:

[0874] The server receives and analyzes the collected data, using natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information, and filters out unnecessary data (such as spam emails and irrelevant chats).

[0875] Input: Collected work-related data

[0876] Output: useful parsed information

[0877] Specific operation: Analyzes text data using NLP algorithms and extracts important keywords.

[0878] Step 3:

[0879] The server stores the analyzed information in a database, where the analysis results are stored in a structured format and made available for later processing.

[0880] Input: Parsed useful information

[0881] Output: Structured data stored in a database

[0882] Specific operation: Analysis results are stored using a database management system.

[0883] Step 4:

[0884] The server trains a machine learning model based on the data in the database. The trained model learns the work patterns and response history of employees and generates a virtual work clone.

[0885] Input: Structured data stored in a database

[0886] Output: Trained machine learning model and business clone

[0887] Specific operation: Train a model using a machine learning algorithm (e.g., K-nearest neighbors).

[0888] Step 5:

[0889] Users access the clone through a web interface or a dedicated application. When a user enters a query, the clone retrieves relevant information from a database and responds. Users can also request specific tasks.

[0890] Input: User query

[0891] Output: Answers and tasks performed by the business clone

[0892] What it does: Takes input from the user interface and generates an answer by searching a database.

[0893] Step 6:

[0894] The server analyzes data related to security incidents and proposes countermeasures. When a security incident occurs, the server provides appropriate countermeasures based on the analyzed data.

[0895] Input: Data about the security incident

[0896] Output: Incident response proposal

[0897] What it does: Analyzes incident data and derives response strategies using natural language processing and machine learning algorithms.

[0898] Step 7:

[0899] The business clone performs automatic email replies, meeting attendance, document proofreading, and security incident response proposals when employees are absent.

[0900] Input: A specific task or situation

[0901] Output: Business tasks and actions taken

[0902] Specific behavior: Based on trained behavioral patterns, the clone automatically performs the assigned task.

[0903] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0904] This invention combines a system in which AI automatically aggregates the work of individual employees and builds a database of their work content as clones with an emotion engine that recognizes user emotions. With this system, even when an employee is absent, the clone can be contacted to understand the work content and response history, and a response can be made according to the user's emotions. This not only enables smooth handover of work, but also enables effective communication that takes into account the user's emotions.

[0905] System Configuration

[0906] The system consists of the following main components:

[0907] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[0908] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[0909] 3. Database: Data storage that stores and makes accessible the analyzed data.

[0910] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[0911] 5. Emotion engine: A device or software that recognizes the user's emotions and adjusts the clone's response and response.

[0912] Program Operation

[0913] 1. Data Collection

[0914] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0915] 2. Data Analysis and Storage

[0916] The server receives the collected data and temporarily stores it. Next, it uses natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information. It also filters out unnecessary data (e.g., spam emails and irrelevant chats). The analyzed data is stored in a structured format in a database.

[0917] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[0918] 3. Training machine learning models and generating business clones

[0919] The server trains a machine learning model based on the data in the database. Past business data and interaction history are used for training. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[0920] For example, a clone can be created that learns employee A's past email reply patterns and what he or she has said in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[0921] 4. Integrating and using emotion engines

[0922] The emotion engine uses natural language processing and speech analysis to recognize the user's emotions in real time during a conversation, understanding the user's emotional state when asking a question and dynamically adjusting the response style and content of the business clone.

[0923] For example, if the emotion engine recognizes that the user is stressed from their tone of voice or text, the business clone will provide a more polite and reassuring response.

[0924] 5. Access to and Use of Business Clones

[0925] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone to obtain information. The user can also request the clone to perform specific tasks.

[0926] For example, a user can ask the clone of employee A, "What is the date of the next project meeting?", and the clone can respond by looking up the schedule information. The clone can also automatically reply to a specified email. In this case, the emotion engine recognizes the user's emotional state and adjusts the response accordingly.

[0927] This system effectively automates employee work and handles absences while taking into consideration the user's feelings, thereby improving the productivity and work efficiency of the entire organization, as well as the quality of communication between employees and users.

[0928] The processing flow will be explained below.

[0929] Step 1:

[0930] The device accesses various applications used by employees and collects work-related data. For example, the device collects email data sent and received from Microsoft Outlook and chat messages from Slack.

[0931] Step 2:

[0932] The data collected by the device is temporarily stored and periodically sent to the server, which starts the centralized management of data.

[0933] Step 3:

[0934] The server analyzes the data received. It uses natural language processing (NLP) to analyze the content of emails and chat messages and extracts the necessary information. For example, it can extract meeting details or task progress from the body of an email.

[0935] Step 4:

[0936] The server filters out unwanted data: spam and irrelevant chat messages are removed at this stage.

[0937] Step 5:

[0938] The server structures the extracted data and stores it in a database, organizing information such as email content, sender, recipient, date, and associated tags.

[0939] Step 6:

[0940] The server trains the machine learning model based on the database, using past work data and interaction history to learn employee work patterns and response histories.

[0941] Step 7:

[0942] The server uses the trained model to generate a clone of each employee's work, which is designed to perform the work even when the employee is absent based on the learned work patterns.

[0943] Step 8:

[0944] The server integrates an emotion engine, which uses natural language processing and speech analysis to recognize the user's emotions in real time.

[0945] Step 9:

[0946] Users access the business clone through a web interface or a dedicated application. When the user asks a question, the emotion engine recognizes the user's emotional state and adjusts the clone's response style accordingly.

[0947] Examples:

[0948] When a user asks the work clone, "What is the date of the next project meeting?", the emotion engine recognizes from the user's tone of voice that they are stressed, and the clone provides a more polite and reassuring response.

[0949] Step 10:

[0950] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[0951] Step 11:

[0952] The user can request a specific task from the work clone, such as replying to a specific email or requesting to attend a meeting.

[0953] Step 12:

[0954] The work clone performs the requested task (e.g., replying to emails, attending meetings, proofreading documents, etc.). By responding according to the user's emotional state through the emotion engine, work continuity is ensured and the quality of communication with the user is improved.

[0955] In this way, by having the above steps work in conjunction with each other, a system is realized that can smoothly take over the work of employees while responding to the user's emotions.

[0956] Example 2

[0957] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0958] With conventional systems, it was difficult to grasp the details of each employee's work and automate appropriate handovers and responses when they were absent. It was also difficult to communicate efficiently and appropriately in situations where consideration for the user's feelings was required. As a result, there was a risk of a decline in work efficiency and a deterioration in the quality of communication.

[0959] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0960] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for transmitting the work-related data to the server, analyzing the received work-related data, and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for integrating an emotion engine that recognizes user emotions in real time and adjusts the responses of the work clones, and means for accessing the work clones and sending questions and tasks to perform work. This allows for smooth handover of work even when employees are absent, and also enables effective communication that takes into account user emotions.

[0961] "Work-related data" refers to data collected from applications employees use in their daily work, such as emails, chat messages, schedule information, and documents they create.

[0962] "Server" means a device for receiving, storing, and analyzing collected business-related data and for training machine learning models.

[0963] "Database" refers to data storage where information extracted and filtered by the server is stored and maintained in a format that can be accessed later.

[0964] A "machine learning model" is an algorithm trained based on data in a database, which learns employees' work patterns and response history and generates virtual work clones.

[0965] A "work clone" is a virtual representative generated by a machine learning model that can perform work on behalf of an employee when they are absent.

[0966] The "emotion engine" refers to a function that uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user and adjusts the response of the business clone.

[0967] A "user" refers to a user who accesses a business clone to send a question or a task, obtain information, or perform a business.

[0968] This invention is a system that automatically aggregates the work of each employee and builds a database of clones of that work content, combined with an emotion engine that recognizes the user's emotions. Specifically, it includes the following components and processing steps:

[0969] 1. Hardware and software used

[0970] This system mainly consists of five main components: terminals, servers, databases, task clones, and emotion engines. Terminals are computers or mobile devices that employees use daily, and are equipped with email clients (e.g., Microsoft Outlook), chat tools (e.g., Slack), schedule management tools (e.g., Google Calendar), etc. The server is a central device that receives, stores, and analyzes collected data and trains machine learning models. The database is data storage that stores and makes accessible the analyzed data in a structured format. Task clones are generated based on machine learning models and are virtual representatives for each employee. The emotion engine is software that recognizes user emotions and adjusts the task clones' responses.

[0971] 2. Program Processing

[0972] Data collection

[0973] The device connects to various applications used by employees in their daily work and collects work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[0974] Data transmission and reception

[0975] The device sends the collected data to the server, where it is encrypted before being sent.

[0976] Data Analysis and Filtering

[0977] The server temporarily stores the data it receives and uses natural language processing to analyze the content of emails and chat messages, extracting necessary information and filtering out unnecessary data.

[0978] Saving to a database

[0979] The server converts the parsed data into a structured format and stores it in a database.

[0980] Training a machine learning model

[0981] The server uses the data in the database to train a machine learning model, which then learns the work patterns and response history of employees and generates work clones.

[0982] Creating a business clone

[0983] The server uses a trained machine learning model to generate a work clone, a virtual representative of the employee.

[0984] Running the Emotion Engine

[0985] The server runs an emotion engine and uses natural language processing and voice analysis to recognize the user's emotions in real time during a dialogue with the user, thereby adjusting the response content and style of the business clone according to the user's emotional state.

[0986] Accessing the clone

[0987] Users access the work clone through a web interface or a dedicated application to ask questions or request tasks. For example, they can send a prompt such as "Please tell me the date of the next project meeting" to the clone, which then references the schedule information and responds.

[0988] 3. Specific Examples

[0989] When a user sends a prompt such as "Please tell me the date of the next project meeting," the work clone retrieves the relevant schedule information from the database and responds with "The next project meeting is next Tuesday at 3:00 p.m." At this time, the emotion engine analyzes the user's emotions from their tone of voice and text and provides an appropriate response style.

[0990] This system allows for smooth handover of work even when employees are absent and responds to users' emotions, improving productivity and work efficiency throughout the organization and improving the quality of communication between employees and users.

[0991] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0992] Step 1: Data collection

[0993] The device connects to various applications (e.g., email clients, chat tools, schedule management tools) used by employees in their daily work to collect work-related data. The input data is emails, chat messages, schedule information, and documents, and the output data is the collected data set. The device collects this data using APIs. For example, the device may use the Microsoft Outlook API to collect emails from the inbox, or the Slack API to collect chat messages.

[0994] Step 2: Sending and Receiving Data

[0995] The terminal sends the data it has collected to the server. The input data is business-related data collected by the terminal, and the output data is data transferred to the server. At this time, the data is encrypted and sent using the TLS protocol or similar. As a specific example, the terminal uses the TLS protocol to send collected email data and chat messages to the server.

[0996] Step 3: Data analysis and filtering

[0997] The server temporarily stores the data it receives and uses natural language processing (NLP) to analyze the content of emails and chat messages. The input data is the business-related data received by the server, and the output data is the filtered, necessary information. Specifically, the server uses the Python library NLTK to analyze the email content and extract important business-related keywords. It also uses SpamAssassin to filter spam emails.

[0998] Step 4: Saving to the Database

[0999] The server converts the parsed data into a structured format and stores it in a database. The input data is the parsed and filtered information, and the output data is the structured information stored in the database. A concrete example is when a server uses Parsey McParseface to parse email content and extract data such as project tasks, scheduled meetings, and action items and store it in a database.

[1000] Step 5: Train the machine learning model

[1001] The server uses the data in the database to train a machine learning model. The input data is the business data stored in the database, and the output data is the trained machine learning model. The server uses the Scikit-learn library to train a random forest model based on past business data.

[1002] Step 6: Creating a business clone

[1003] The server generates a job clone for each employee using the trained machine learning model. The input data is the trained machine learning model, and the output data is the generated job clone. For example, the server trains a neural network using the Keras framework to generate a job clone for employee A.

[1004] Step 7: Run the Emotion Engine

[1005] The server runs an emotion engine and uses natural language processing and speech analysis to recognize the user's emotions in real time during a dialogue with the user. The input data is the user's dialogue text and voice, and the output data is the analyzed user's emotional information. For example, the server uses the Python library TextBlob to analyze emotions from the user's text input, and if the user is feeling stressed, the work clone will provide a polite response.

[1006] Step 8: Access the clone

[1007] Users access the business clone through a web interface or a dedicated application. The input data are questions or task requests from the user, and the output data are the answers or tasks executed by the clone. For example, a user sends a prompt such as "Please tell me the date of the next project meeting" to the business clone, and the clone retrieves schedule information from a database and responds.

[1008] (Application example 2)

[1009] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1010] In today's business environment, it is often difficult to smoothly transfer work or respond to requests when employees are absent. Furthermore, when users contact employees, it is difficult for them to properly recognize and respond to the user's feelings. This situation is likely to lead to a decline in business efficiency and customer satisfaction. Furthermore, the increased burden caused by labor shortages is also a problem, and a solution to this issue is needed.

[1011] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1012] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating a work clone for each employee, emotion recognition means for recognizing the user's emotions and dynamically adjusting the response style and content of the work clone, and means for accessing the work clone and performing work. This allows for smooth handover and response even when an employee is absent, and enables appropriate response according to the user's emotions.

[1013] "Applications used by individual employees" refers to software such as email clients, chat tools, and schedule management applications that employees use in their daily work.

[1014] "Work-related data" is data such as emails, chat messages, schedule information, and documents that employees create or receive through these applications.

[1015] "Extraction and filtering means" refers to a method or device that performs processing to analyze and extract necessary information from business-related data and remove unnecessary information.

[1016] A "database" is a data storage system that stores collected and analyzed business-related data in a structured format and makes it accessible as needed.

[1017] "Training a machine learning model" is the process of using collected business-related data to create a model that can make predictions or classifications using machine learning algorithms.

[1018] A "work clone" is a virtual representative generated based on a machine learning model that mimics the work patterns and response history of a specific employee.

[1019] "Emotion recognition means" is a technology that analyzes emotions from a user's voice or text communication in real time and dynamically adjusts the response content.

[1020] "Means of performing business" refers to the mechanisms and processes by which a business clone performs business on behalf of an employee.

[1021] The present invention provides a system that uses a smart glasses application in a brick-and-mortar store to smoothly take over the work of an employee and respond to customers' emotions. Specific embodiments of the system are described below.

[1022] System Configuration

[1023] The system consists of the following main components:

[1024] 1. Devices: Smart glasses, computers, and mobile devices used by employees. These devices have email clients, chat tools, and schedule management applications installed.

[1025] 2. Server: A central device for collecting and analyzing data and training machine learning models. It mainly runs a database and an emotion recognition engine.

[1026] 3. Database: Stores the analyzed data and makes it accessible when needed.

[1027] 4. Work Clone: ​​A virtual representative for each employee, based on machine learning, is created to take over the employee's work when they are absent.

[1028] 5. Emotion Recognition Engine: Recognizes customer emotions in real time and adjusts the clone's response.

[1029] Program Operation

[1030] Data collection

[1031] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management tools) to collect work-related data. For example, the device collects email data from email clients and chat messages from chat tools.

[1032] Data analysis and storage

[1033] The server analyzes the collected data and temporarily stores it. Next, it uses natural language processing to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data is filtered out and stored in a structured format in a database.

[1034] Training machine learning models and generating business clones

[1035] The server trains a machine learning model based on the data in the database. It uses past work data and interaction history to learn employees' work patterns and response histories to generate virtual work clones. For example, a clone can be generated based on a specific employee's email response patterns or the content of comments made in meetings.

[1036] Integrating and using emotion recognition engines

[1037] The emotion recognition engine uses natural language processing and voice analysis to recognize customer emotions in real time during user interaction, understanding the emotional state of the customer when they ask a question and dynamically adjusting the response style and content of the work clone.

[1038] Specific examples of operation

[1039] Consider a scenario in which an employee wearing smart glasses serves a customer in a physical store. When a customer asks an employee a question, an emotion recognition engine analyzes the customer's emotions (e.g., anger, confusion, joy), and the work clone generates an appropriate response based on this. For example, if a customer appears confused and asks about the next event, the work clone will respond with a polite explanation and reassurance.

[1040] Prompt Sentence Examples

[1041] "Customer A seemed distressed and asked the store clerk about details of the next event. The store clerk recognized Customer A's feelings and politely explained the next event to Customer A, putting him at ease."

[1042] This system allows smooth handover of work and response even when employees are absent, and also enables appropriate responses based on customer emotions.Furthermore, customer satisfaction is expected to improve and operational efficiency can be expected to increase.

[1043] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1044] Step 1:

[1045] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management applications) to collect work-related data. The input is data from each application, and the output is the collected work-related data. Specific data collection operations involve the device collecting email data from email clients and retrieving chat messages from chat tools.

[1046] Step 2:

[1047] The server temporarily stores the collected business-related data. The input is the data collected from the terminal, and the output is the temporarily stored data. In this step, the specific operation of storing data on the server is to store emails and chat messages in temporary files in the database.

[1048] Step 3:

[1049] The server uses natural language processing to analyze the collected data, extracting and filtering the necessary information. The input is temporarily stored data, and the output is useful information after analysis. Specifically, it extracts important keywords from the body of an email and lists tasks from meeting minutes.

[1050] Step 4:

[1051] The server stores the analyzed data in a database. The input is useful information after analysis, and the output is data stored in a structured format. Specifically, in this step, important project progress information and task lists are stored in the database.

[1052] Step 5:

[1053] The server trains a machine learning model based on the data in the database. The input is the structured data in the database, and the output is the trained machine learning model. The specific operation of this step is to use past work data to learn each employee's work patterns and response history.

[1054] Step 6:

[1055] The server generates a work clone based on a trained machine learning model. The input is the trained machine learning model, and the output is a virtual representative for each employee. Specifically, a virtual work clone is created based on a specific employee's email response patterns and the content of comments made in meetings.

[1056] Step 7:

[1057] The emotion recognition engine uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user. The input is the user's voice or text, and the output is the user's emotional information. Specifically, emotions (e.g., anger, confusion, joy) are analyzed from the customer's tone of voice or text.

[1058] Step 8:

[1059] The server dynamically adjusts the response style and content of the business clone based on emotional information from the emotion recognition engine. The input is emotional information from the emotion recognition engine and response data from the business clone, and the output is an adjusted response message. Specifically, if the customer is confused, the business clone changes the response content to provide a more polite and reassuring answer.

[1060] Step 9:

[1061] The terminal provides the tailored response message to the customer through the smart glasses. The input is the tailored response message, and the output is the information the customer receives through the smart glasses. Specifically, the response of the business clone is displayed on the display of the smart glasses.

[1062] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1063] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1064] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1065] [Fourth embodiment]

[1066] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1067] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1068] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1069] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1070] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1071] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1072] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1073] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1074] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1075] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1076] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1077] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1078] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1079] This invention is a system in which AI automatically aggregates the work of each employee and builds the work content into a database as a clone. With this system, even when an employee is absent, the work content and response history can be understood by inquiring with the clone, enabling a smooth handover of work.

[1080] System Configuration

[1081] The system consists of the following main components:

[1082] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[1083] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[1084] 3. Database: Data storage that stores and makes accessible the analyzed data.

[1085] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[1086] Program Operation

[1087] 1. Data Collection

[1088] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from Microsoft Outlook, and also collects chat messages from Slack.

[1089] 2. Data Analysis and Storage

[1090] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unwanted data (e.g., spam emails and irrelevant chats) is also filtered out. The analyzed data is stored in a structured format in a database.

[1091] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[1092] 3. Training machine learning models and generating business clones

[1093] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[1094] For example, a clone can be created that learns employee A's past email reply patterns and the content of comments made in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[1095] 4. Access to and Use of Business Clones

[1096] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone, which retrieves the relevant information from a database and responds. The user can also request the clone to perform specific tasks.

[1097] For example, a user asks the clone of employee A, "Please tell me the date of the next project meeting," and the clone answers by looking up the schedule information.The clone also automatically replies to the specified email.

[1098] Specific examples

[1099] For example, if employee A suddenly becomes ill and cannot come to work, the user can use employee A's work clone to continue their daily work. The clone will automatically reply to emails received on employee A's behalf, attend meetings, and create minutes. In this way, it is possible to minimize work disruptions caused by employee A's absence and ensure business continuity.

[1100] This system effectively automates employee tasks and enables responses when employees are absent without human intervention, thereby improving productivity and operational efficiency across the organization.

[1101] The processing flow will be explained below.

[1102] Step 1:

[1103] The device accesses applications used by employees and collects work-related data, for example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[1104] Step 2:

[1105] The server receives the collected data and temporarily stores it, allowing various data to be managed centrally.

[1106] Step 3:

[1107] The server analyzes the data it receives. It uses natural language processing (NLP) to analyze the content of emails and chat messages, extracting the necessary information, and filtering out unnecessary data (e.g., spam emails and irrelevant chats).

[1108] Step 4:

[1109] The server structures the parsed data, organizing it based on email content, sender, recipient, date, tagged keywords, etc.

[1110] Step 5:

[1111] The server stores the structured data in a database, which stores the data organized by employee for efficient access later.

[1112] Step 6:

[1113] The server reads data from the database and trains the machine learning model, using past business data and interaction history.

[1114] Step 7:

[1115] The server uses trained machine learning models to generate work clones for each employee, designed to mimic specific work patterns and behaviors.

[1116] Step 8:

[1117] Users access the work clone through a web interface or a dedicated application, and obtain information by asking the clone questions (e.g., "When is the next meeting scheduled?").

[1118] Step 9:

[1119] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[1120] Step 10:

[1121] Users can also assign tasks to the work clone, such as replying to a specific email or requesting attendance at a meeting.

[1122] Step 11:

[1123] The work clone performs the requested tasks (e.g., replying to emails, attending meetings, proofreading documents, etc.), which prevents work from stalling when an employee is absent.

[1124] In this way, each step works in coordination to realize a system that smoothly takes over employees' work and ensures business continuity throughout the organization.

[1125] Example 1

[1126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1127] When an employee is absent, a means is needed to smoothly transfer work and prevent business stagnation. With conventional systems, it is difficult to grasp the content of employees' work and their response history, and work often does not proceed smoothly when employees are absent. This causes problems such as a decline in productivity and work efficiency throughout the organization. There is a need to solve this problem and provide a means to effectively continue work even when employees are absent.

[1128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1129] In this invention, the server includes means for collecting work-related data from software used by each employee, means for analyzing the work-related data and extracting and filtering necessary information using natural language processing, means for storing the extracted information in a structured format in data storage, means for training a machine learning model based on the data in the data storage and generating a virtual representative for each employee, and means for accessing the virtual representative and performing work. This allows the virtual representative to automatically take over work even when the employee is absent, and to respond to emails, attend meetings, etc.

[1130] "Employee" refers to an individual who performs work within a company or organization.

[1131] "Software" refers to applications and programs that run on a computer.

[1132] "Business-related data" refers to information related to business, such as emails, messages, schedule data, and files created.

[1133] "Collection" refers to the act of gathering specific data or information.

[1134] "Natural language processing" refers to the technology that allows computers to analyze and understand human language.

[1135] "Extraction" refers to the act of isolating and extracting specific data or information.

[1136] "Filtering" refers to the act of removing unnecessary data and leaving only the necessary data.

[1137] "Data storage" refers to devices and systems for storing data.

[1138] A "structured format" refers to data that is organized according to rules and put into a form that can be easily handled.

[1139] A "machine learning model" refers to an algorithm or system that learns from data and performs pattern recognition and prediction.

[1140] A "virtual representative" refers to a software agent generated based on a machine learning model that mimics and performs the work of a specific employee.

[1141] "Access" refers to the act of connecting to, operating, or using data or a system.

[1142] "Performance" refers to the act of carrying out a specific task or job.

[1143] The system of this invention automatically aggregates employee tasks and builds a database of their tasks as virtual representatives. The system consists of the following main components:

[1144] 1. Devices: Computers and mobile devices used by employees on a daily basis. Each device has installed an email client, chat tool, schedule management application, etc. Examples include Microsoft Outlook, Slack, Google Calendar, and Microsoft Word.

[1145] 2. Server: A central device for storing and analyzing collected data and training machine learning models. This server receives, analyzes, filters, and stores data, trains machine learning models, and generates virtual representatives.

[1146] 3. Data storage: A database that stores and makes accessible the analyzed data. This data storage stores the data in a structured format.

[1147] 4. Virtual Representative: A virtual representative for each employee, generated based on machine learning, that performs tasks on behalf of the employee when they are not available.

[1148] As an example, consider the case where employee D suddenly falls ill and is unable to come to work. Employee D's device collects emails sent and received via Microsoft Outlook, which he or she uses daily, and Slack chat messages. The device transfers this data to a server. The server analyzes the transferred data using natural language processing technology, extracting and filtering important work-related information. The analyzed data is then stored in a structured format in data storage. The server then trains a machine learning model based on the data stored in the data storage. The trained model learns employee D's work patterns and response history and generates a virtual representative. Users can access employee D's virtual representative through a web interface or a dedicated application to obtain necessary information or request tasks.

[1149] Specifically, a user can ask the virtual agent for employee D, "What is the date of the next project meeting?", and the virtual agent will respond by referencing the schedule information in the data storage. The user can also have the virtual agent automatically reply to emails that they receive.

[1150] This system makes it possible to minimize disruption to work and ensure business continuity even when an employee is absent. An example of a specific prompt sentence is, "Please confirm the date of the next project meeting with the work clone of employee D."

[1151] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1152] Step 1: Data collection

[1153] The device collects work-related data from the software used by employees (e.g., Microsoft Outlook, Slack, Google Calendar, Microsoft Word). Specifically, the device connects to the API of each application to obtain emails, chat messages, schedule information, documents created, and more.

[1154] Input: Work-related data (emails, messages, etc.) entered by employees into each application

[1155] Data processing: The device obtains this data through the API and converts it into a unified format.

[1156] Output: Transfer the collected data (in a unified format) to the server.

[1157] Step 2: Data analysis and storage

[1158] The server receives business-related data sent from the device and analyzes it using natural language processing (NLP) technology, extracting necessary information and filtering out spam and irrelevant messages.

[1159] Input: Business-related data in a unified format sent from the terminal

[1160] Data processing: The server uses NLP to extract important information and filter out unnecessary data.

[1161] Output: Save the structured parsed data to data storage

[1162] Step 3: Train the machine learning model

[1163] The server trains a machine learning model based on the analyzed data stored in the data storage, learning the work patterns and response history of employees and generating a virtual representative based on that.

[1164] Input: Structured, parsed data stored in data storage

[1165] Data Computing: The server uses machine learning algorithms to learn patterns from the data and train the model.

[1166] Output: A trained machine learning model and a generated virtual representative

[1167] Step 4: Access and use the business clone

[1168] The user accesses the virtual agent through a web interface or a dedicated application. The user sends questions or tasks to the virtual agent, which then retrieves relevant information from data storage and responds.

[1169] Input: Questions and task requests that users enter into a web interface or dedicated application.

[1170] Data Computing: A virtual agent searches for relevant information from data storage and generates the best answer

[1171] Output: Response to the user (e.g., next meeting date) or task execution (e.g., email auto-reply)

[1172] Specific examples of operation

[1173] For example, if a user asks the virtual agent, "What is the date of the next project meeting?", the virtual agent will refer to the schedule information in the data storage and respond to the user, "The next project meeting is the day after tomorrow at 2:00 PM." The virtual agent can also automatically reply to configured emails, ensuring business continuity even when employees are absent.

[1174] (Application example 1)

[1175] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1176] It is important to prevent business interruptions and ensure business continuity even when employees are absent. Companies are also required to respond to security incidents quickly and accurately. However, employee absence or mistakes can delay response, which can lead to serious problems. Therefore, a system is needed that can take over employee work and streamline incident response.

[1177] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1178] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for users to access the work clones and perform their work via a web interface or a dedicated application, and means for analyzing data related to security incidents and proposing countermeasures. This ensures business continuity even when employees are absent, and enables quick and accurate responses to security incidents.

[1179] "Work-related data" is data collected from the applications and tools employees use to perform their jobs.

[1180] A "means for collecting data" is a method or device for collecting the necessary data from an application or tool.

[1181] "Means for analyzing data" refers to methods or devices that analyze collected data and extract useful information.

[1182] "Means for filtering information" refers to a method or device that eliminates unnecessary information from analyzed data and selects only the necessary information.

[1183] A "database" is a system for organizing and storing analyzed information.

[1184] A "machine learning model" is an algorithm or system that learns specific patterns or rules based on collected data.

[1185] A "work clone" is a program or agent that is generated based on a machine learning model and virtually performs the work of an employee.

[1186] A "web interface" is a screen that allows users to access and operate the system via the Internet.

[1187] A "specialized application" is software developed for a specific function or purpose.

[1188] A "security incident" is a security-related problem such as unauthorized access to or attacks on a system or network.

[1189] The "means for proposing countermeasures" refers to a method or device for providing appropriate countermeasures based on the analysis results.

[1190] "Means of carrying out business" refers to the methods and devices that business clones and users use to carry out business using the system.

[1191] This invention is a system for improving the efficiency of employee work and responding quickly and accurately to security incidents. This system collects work-related data from applications used by employees, analyzes it to extract necessary information, filters it, and stores it in a database. Furthermore, this database is used to train a machine learning model, which generates a work clone for each employee. Users can then access this work clone through a web interface or a dedicated application to perform their work.

[1192] System Configuration

[1193] 1. Hardware and Software Configuration

[1194] Device: An employee's computer or mobile device that serves as the data collection and user interface.

[1195] Server: A central device that performs data analysis and trains machine learning models.

[1196] Database: Data storage that stores and makes accessible analyzed information.

[1197] 2. Program Processing

[1198] Data collection

[1199] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data sent and received from email clients and chat messages from chat tools.

[1200] Data analysis and storage

[1201] The server receives and analyzes the collected data. Natural language processing (NLP) is used to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data (such as spam emails and irrelevant chats) is also filtered out. The analyzed data is then structured and stored in a database.

[1202] Training machine learning models and generating business clones

[1203] The server trains a machine learning model based on the data in the database. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work on behalf of the employee when they are absent.

[1204] Accessing and using business clones

[1205] Users access the work clone through a web interface or a dedicated application. For example, when a user asks the work clone, "When is the next meeting scheduled?", the clone retrieves the relevant information from the database and responds.

[1206] Security Incident Response

[1207] The server analyzes data related to security incidents and uses natural language processing to understand the characteristics of the incidents. Based on the machine learning model, the business clone proposes appropriate countermeasures.

[1208] 3. Specific Examples

[1209] For example, if an employee is suddenly absent due to illness, the user can use a work clone to continue their daily work. The clone automatically replies to emails received on behalf of the employee, attends meetings, and creates minutes. In addition, if a security incident occurs, the clone can quickly propose countermeasures based on past response history and provide them to the user.

[1210] 4. Example prompts for generative AI models

[1211] "When is the next meeting scheduled?"

[1212] "What should we do to protect ourselves from the DDoS attack that occurred last night?"

[1213] "Can you give me more details about the security alert you received today?"

[1214] This system ensures business continuity even when employees are absent, and enables a swift and accurate response to security incidents.

[1215] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1216] Step 1:

[1217] The device connects to various applications used by employees (email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. At this time, the device obtains emails sent and received from each application, chat messages, schedule information, and created documents. For example, it collects the "contents of received emails" from email clients and the "chat history" from chat tools.

[1218] Input: Emails, chat messages, schedule information, various documents

[1219] Output: Collected work-related data

[1220] Step 2:

[1221] The server receives and analyzes the collected data, using natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information, and filters out unnecessary data (such as spam emails and irrelevant chats).

[1222] Input: Collected work-related data

[1223] Output: useful parsed information

[1224] Specific operation: Analyzes text data using NLP algorithms and extracts important keywords.

[1225] Step 3:

[1226] The server stores the analyzed information in a database, where the analysis results are stored in a structured format and made available for later processing.

[1227] Input: Parsed useful information

[1228] Output: Structured data stored in a database

[1229] Specific operation: Analysis results are stored using a database management system.

[1230] Step 4:

[1231] The server trains a machine learning model based on the data in the database. The trained model learns the work patterns and response history of employees and generates a virtual work clone.

[1232] Input: Structured data stored in a database

[1233] Output: Trained machine learning model and business clone

[1234] Specific operation: Train a model using a machine learning algorithm (e.g., K-nearest neighbors).

[1235] Step 5:

[1236] Users access the clone through a web interface or a dedicated application. When a user enters a query, the clone retrieves relevant information from a database and responds. Users can also request specific tasks.

[1237] Input: User query

[1238] Output: Answers and tasks performed by the business clone

[1239] What it does: Takes input from the user interface and generates an answer by searching a database.

[1240] Step 6:

[1241] The server analyzes data related to security incidents and proposes countermeasures. When a security incident occurs, the server provides appropriate countermeasures based on the analyzed data.

[1242] Input: Data about the security incident

[1243] Output: Incident response proposal

[1244] What it does: Analyzes incident data and derives response strategies using natural language processing and machine learning algorithms.

[1245] Step 7:

[1246] The business clone performs automatic email replies, meeting attendance, document proofreading, and security incident response proposals when employees are absent.

[1247] Input: A specific task or situation

[1248] Output: Business tasks and actions taken

[1249] Specific behavior: Based on trained behavioral patterns, the clone automatically performs the assigned task.

[1250] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1251] This invention combines a system in which AI automatically aggregates the work of individual employees and builds a database of their work content as clones with an emotion engine that recognizes user emotions. With this system, even when an employee is absent, the clone can be contacted to understand the work content and response history, and a response can be made according to the user's emotions. This not only enables smooth handover of work, but also enables effective communication that takes into account the user's emotions.

[1252] System Configuration

[1253] The system consists of the following main components:

[1254] 1. Devices: Computers and mobile devices that employees use on a daily basis, with email clients, chat tools, schedule management applications, etc. installed.

[1255] 2. Server: A central device for storing and analyzing collected data and training machine learning models.

[1256] 3. Database: Data storage that stores and makes accessible the analyzed data.

[1257] 4. Work Clone: ​​A virtual representative for each employee, generated based on machine learning.

[1258] 5. Emotion engine: A device or software that recognizes the user's emotions and adjusts the clone's response and response.

[1259] Program Operation

[1260] 1. Data Collection

[1261] The device connects to various applications used by employees (e.g., email clients, chat tools, schedule management tools, document creation tools, etc.) to collect work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[1262] 2. Data Analysis and Storage

[1263] The server receives the collected data and temporarily stores it. Next, it uses natural language processing (NLP) to analyze the content of emails and chat messages to extract the necessary information. It also filters out unnecessary data (e.g., spam emails and irrelevant chats). The analyzed data is stored in a structured format in a database.

[1264] For example, the server analyzes employee A's email data, extracts important work-related emails, and creates a list of project progress and tasks from them. Similarly, it extracts action items from meeting minutes and stores them in a database.

[1265] 3. Training machine learning models and generating business clones

[1266] The server trains a machine learning model based on the data in the database. Past business data and interaction history are used for training. The trained model learns the employee's work patterns and response history and generates a virtual work clone. This work clone can perform work in place of the employee when they are absent.

[1267] For example, a clone can be created that learns employee A's past email reply patterns and what he or she has said in meetings, and when employee A is absent, the clone will attend project meetings and make appropriate comments and take notes.

[1268] 4. Integrating and using emotion engines

[1269] The emotion engine uses natural language processing and speech analysis to recognize the user's emotions in real time during a conversation, understanding the user's emotional state when asking a question and dynamically adjusting the response style and content of the business clone.

[1270] For example, if the emotion engine recognizes that the user is stressed from their tone of voice or text, the business clone will provide a more polite and reassuring response.

[1271] 5. Access to and Use of Business Clones

[1272] When a user needs to handle business tasks while away from home, the user accesses the work clone through a web interface or a dedicated application. The user sends a question (for example, "When is the next meeting scheduled?") to the clone to obtain information. The user can also request the clone to perform specific tasks.

[1273] For example, a user can ask the clone of employee A, "What is the date of the next project meeting?", and the clone can respond by looking up the schedule information. The clone can also automatically reply to a specified email. In this case, the emotion engine recognizes the user's emotional state and adjusts the response accordingly.

[1274] This system effectively automates employee work and handles absences while taking into consideration the user's feelings, thereby improving the productivity and work efficiency of the entire organization, as well as the quality of communication between employees and users.

[1275] The processing flow will be explained below.

[1276] Step 1:

[1277] The device accesses various applications used by employees and collects work-related data. For example, the device collects email data sent and received from Microsoft Outlook and chat messages from Slack.

[1278] Step 2:

[1279] The data collected by the device is temporarily stored and periodically sent to the server, which starts the centralized management of data.

[1280] Step 3:

[1281] The server analyzes the data received. It uses natural language processing (NLP) to analyze the content of emails and chat messages and extracts the necessary information. For example, it can extract meeting details or task progress from the body of an email.

[1282] Step 4:

[1283] The server filters out unwanted data: spam and irrelevant chat messages are removed at this stage.

[1284] Step 5:

[1285] The server structures the extracted data and stores it in a database, organizing information such as email content, sender, recipient, date, and associated tags.

[1286] Step 6:

[1287] The server trains the machine learning model based on the database, using past work data and interaction history to learn employee work patterns and response histories.

[1288] Step 7:

[1289] The server uses the trained model to generate a clone of each employee's work, which is designed to perform the work even when the employee is absent based on the learned work patterns.

[1290] Step 8:

[1291] The server integrates an emotion engine, which uses natural language processing and speech analysis to recognize the user's emotions in real time.

[1292] Step 9:

[1293] Users access the business clone through a web interface or a dedicated application. When the user asks a question, the emotion engine recognizes the user's emotional state and adjusts the clone's response style accordingly.

[1294] Examples:

[1295] When a user asks the work clone, "What is the date of the next project meeting?", the emotion engine recognizes from the user's tone of voice that they are stressed, and the clone provides a more polite and reassuring response.

[1296] Step 10:

[1297] The task clone retrieves relevant information from the database and responds to the user, for example, providing meeting schedule information.

[1298] Step 11:

[1299] The user can request a specific task from the work clone, such as replying to a specific email or requesting to attend a meeting.

[1300] Step 12:

[1301] The work clone performs the requested task (e.g., replying to emails, attending meetings, proofreading documents, etc.). By responding according to the user's emotional state through the emotion engine, work continuity is ensured and the quality of communication with the user is improved.

[1302] In this way, by having the above steps work in conjunction with each other, a system is realized that can smoothly take over the work of employees while responding to the user's emotions.

[1303] Example 2

[1304] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1305] With conventional systems, it was difficult to grasp the details of each employee's work and automate appropriate handovers and responses when they were absent. It was also difficult to communicate efficiently and appropriately in situations where consideration for the user's feelings was required. As a result, there was a risk of a decline in work efficiency and a deterioration in the quality of communication.

[1306] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1307] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for transmitting the work-related data to the server, analyzing the received work-related data, and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating work clones for each employee, means for integrating an emotion engine that recognizes user emotions in real time and adjusts the responses of the work clones, and means for accessing the work clones and sending questions and tasks to perform work. This allows for smooth handover of work even when employees are absent, and also enables effective communication that takes into account user emotions.

[1308] "Work-related data" refers to data collected from applications employees use in their daily work, such as emails, chat messages, schedule information, and documents they create.

[1309] "Server" means a device for receiving, storing, and analyzing collected business-related data and for training machine learning models.

[1310] "Database" refers to data storage where information extracted and filtered by the server is stored and maintained in a format that can be accessed later.

[1311] A "machine learning model" is an algorithm trained based on data in a database, which learns employees' work patterns and response history and generates virtual work clones.

[1312] A "work clone" is a virtual representative generated by a machine learning model that can perform work on behalf of an employee when they are absent.

[1313] The "emotion engine" refers to a function that uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user and adjusts the response of the business clone.

[1314] A "user" refers to a user who accesses a business clone to send a question or a task, obtain information, or perform a business.

[1315] This invention is a system that automatically aggregates the work of each employee and builds a database of clones of that work content, combined with an emotion engine that recognizes the user's emotions. Specifically, it includes the following components and processing steps:

[1316] 1. Hardware and software used

[1317] This system mainly consists of five main components: terminals, servers, databases, task clones, and emotion engines. Terminals are computers or mobile devices that employees use daily, and are equipped with email clients (e.g., Microsoft Outlook), chat tools (e.g., Slack), schedule management tools (e.g., Google Calendar), etc. The server is a central device that receives, stores, and analyzes collected data and trains machine learning models. The database is data storage that stores and makes accessible the analyzed data in a structured format. Task clones are generated based on machine learning models and are virtual representatives for each employee. The emotion engine is software that recognizes user emotions and adjusts the task clones' responses.

[1318] 2. Program Processing

[1319] Data collection

[1320] The device connects to various applications used by employees in their daily work and collects work-related data. For example, the device collects email data from Microsoft Outlook and chat messages from Slack.

[1321] Data transmission and reception

[1322] The device sends the collected data to the server, where it is encrypted before being sent.

[1323] Data Analysis and Filtering

[1324] The server temporarily stores the data it receives and uses natural language processing to analyze the content of emails and chat messages, extracting necessary information and filtering out unnecessary data.

[1325] Saving to a database

[1326] The server converts the parsed data into a structured format and stores it in a database.

[1327] Training a machine learning model

[1328] The server uses the data in the database to train a machine learning model, which then learns the work patterns and response history of employees and generates work clones.

[1329] Creating a business clone

[1330] The server uses a trained machine learning model to generate a work clone, a virtual representative of the employee.

[1331] Running the Emotion Engine

[1332] The server runs an emotion engine and uses natural language processing and voice analysis to recognize the user's emotions in real time during a dialogue with the user, thereby adjusting the response content and style of the business clone according to the user's emotional state.

[1333] Accessing the clone

[1334] Users access the work clone through a web interface or a dedicated application to ask questions or request tasks. For example, they can send a prompt such as "Please tell me the date of the next project meeting" to the clone, which then references the schedule information and responds.

[1335] 3. Specific Examples

[1336] When a user sends a prompt such as "Please tell me the date of the next project meeting," the work clone retrieves the relevant schedule information from the database and responds with "The next project meeting is next Tuesday at 3:00 p.m." At this time, the emotion engine analyzes the user's emotions from their tone of voice and text and provides an appropriate response style.

[1337] This system allows for smooth handover of work even when employees are absent and responds to users' emotions, improving productivity and work efficiency throughout the organization and improving the quality of communication between employees and users.

[1338] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1339] Step 1: Data collection

[1340] The device connects to various applications (e.g., email clients, chat tools, schedule management tools) used by employees in their daily work to collect work-related data. The input data is emails, chat messages, schedule information, and documents, and the output data is the collected data set. The device collects this data using APIs. For example, the device may use the Microsoft Outlook API to collect emails from the inbox, or the Slack API to collect chat messages.

[1341] Step 2: Sending and Receiving Data

[1342] The terminal sends the data it has collected to the server. The input data is business-related data collected by the terminal, and the output data is data transferred to the server. At this time, the data is encrypted and sent using the TLS protocol or similar. As a specific example, the terminal uses the TLS protocol to send collected email data and chat messages to the server.

[1343] Step 3: Data analysis and filtering

[1344] The server temporarily stores the data it receives and uses natural language processing (NLP) to analyze the content of emails and chat messages. The input data is the business-related data received by the server, and the output data is the filtered, necessary information. Specifically, the server uses the Python library NLTK to analyze the email content and extract important business-related keywords. It also uses SpamAssassin to filter spam emails.

[1345] Step 4: Saving to the Database

[1346] The server converts the parsed data into a structured format and stores it in a database. The input data is the parsed and filtered information, and the output data is the structured information stored in the database. A concrete example is when a server uses Parsey McParseface to parse email content and extract data such as project tasks, scheduled meetings, and action items and store it in a database.

[1347] Step 5: Train the machine learning model

[1348] The server uses the data in the database to train a machine learning model. The input data is the business data stored in the database, and the output data is the trained machine learning model. The server uses the Scikit-learn library to train a random forest model based on past business data.

[1349] Step 6: Creating a business clone

[1350] The server generates a job clone for each employee using the trained machine learning model. The input data is the trained machine learning model, and the output data is the generated job clone. For example, the server trains a neural network using the Keras framework to generate a job clone for employee A.

[1351] Step 7: Run the Emotion Engine

[1352] The server runs an emotion engine and uses natural language processing and speech analysis to recognize the user's emotions in real time during a dialogue with the user. The input data is the user's dialogue text and voice, and the output data is the analyzed user's emotional information. For example, the server uses the Python library TextBlob to analyze emotions from the user's text input, and if the user is feeling stressed, the work clone will provide a polite response.

[1353] Step 8: Access the clone

[1354] Users access the business clone through a web interface or a dedicated application. The input data are questions or task requests from the user, and the output data are the answers or tasks executed by the clone. For example, a user sends a prompt such as "Please tell me the date of the next project meeting" to the business clone, and the clone retrieves schedule information from a database and responds.

[1355] (Application example 2)

[1356] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1357] In today's business environment, it is often difficult to smoothly transfer work or respond to requests when employees are absent. Furthermore, when users contact employees, it is difficult for them to properly recognize and respond to the user's feelings. This situation is likely to lead to a decline in business efficiency and customer satisfaction. Furthermore, the increased burden caused by labor shortages is also a problem, and a solution to this issue is needed.

[1358] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1359] In this invention, the server includes means for collecting work-related data from applications used by individual employees, means for analyzing the work-related data and extracting and filtering necessary information, means for storing the extracted information in a database, means for training a machine learning model based on the database and generating a work clone for each employee, emotion recognition means for recognizing the user's emotions and dynamically adjusting the response style and content of the work clone, and means for accessing the work clone and performing work. This allows for smooth handover and response even when an employee is absent, and enables appropriate response according to the user's emotions.

[1360] "Applications used by individual employees" refers to software such as email clients, chat tools, and schedule management applications that employees use in their daily work.

[1361] "Work-related data" is data such as emails, chat messages, schedule information, and documents that employees create or receive through these applications.

[1362] "Extraction and filtering means" refers to a method or device that performs processing to analyze and extract necessary information from business-related data and remove unnecessary information.

[1363] A "database" is a data storage system that stores collected and analyzed business-related data in a structured format and makes it accessible as needed.

[1364] "Training a machine learning model" is the process of using collected business-related data to create a model that can make predictions or classifications using machine learning algorithms.

[1365] A "work clone" is a virtual representative generated based on a machine learning model that mimics the work patterns and response history of a specific employee.

[1366] "Emotion recognition means" is a technology that analyzes emotions from a user's voice or text communication in real time and dynamically adjusts the response content.

[1367] "Means of performing business" refers to the mechanisms and processes by which a business clone performs business on behalf of an employee.

[1368] The present invention provides a system that uses a smart glasses application in a brick-and-mortar store to smoothly take over the work of an employee and respond to customers' emotions. Specific embodiments of the system are described below.

[1369] System Configuration

[1370] The system consists of the following main components:

[1371] 1. Devices: Smart glasses, computers, and mobile devices used by employees. These devices have email clients, chat tools, and schedule management applications installed.

[1372] 2. Server: A central device for collecting and analyzing data and training machine learning models. It mainly runs a database and an emotion recognition engine.

[1373] 3. Database: Stores the analyzed data and makes it accessible when needed.

[1374] 4. Work Clone: ​​A virtual representative for each employee, based on machine learning, is created to take over the employee's work when they are absent.

[1375] 5. Emotion Recognition Engine: Recognizes customer emotions in real time and adjusts the clone's response.

[1376] Program Operation

[1377] Data collection

[1378] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management tools) to collect work-related data. For example, the device collects email data from email clients and chat messages from chat tools.

[1379] Data analysis and storage

[1380] The server analyzes the collected data and temporarily stores it. Next, it uses natural language processing to analyze the content of emails and chat messages to extract the necessary information. Unnecessary data is filtered out and stored in a structured format in a database.

[1381] Training machine learning models and generating business clones

[1382] The server trains a machine learning model based on the data in the database. It uses past work data and interaction history to learn employees' work patterns and response histories to generate virtual work clones. For example, a clone can be generated based on a specific employee's email response patterns or the content of comments made in meetings.

[1383] Integrating and using emotion recognition engines

[1384] The emotion recognition engine uses natural language processing and voice analysis to recognize customer emotions in real time during user interaction, understanding the emotional state of the customer when they ask a question and dynamically adjusting the response style and content of the work clone.

[1385] Specific examples of operation

[1386] Consider a scenario in which an employee wearing smart glasses serves a customer in a physical store. When a customer asks an employee a question, an emotion recognition engine analyzes the customer's emotions (e.g., anger, confusion, joy), and the work clone generates an appropriate response based on this. For example, if a customer appears confused and asks about the next event, the work clone will respond with a polite explanation and reassurance.

[1387] Prompt Sentence Examples

[1388] "Customer A seemed distressed and asked the store clerk about details of the next event. The store clerk recognized Customer A's feelings and politely explained the next event to Customer A, putting him at ease."

[1389] This system allows smooth handover of work and response even when employees are absent, and also enables appropriate responses based on customer emotions.Furthermore, customer satisfaction is expected to improve and operational efficiency can be expected to increase.

[1390] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1391] Step 1:

[1392] The device connects to applications used by employees in their daily work (e.g., email clients, chat tools, schedule management applications) to collect work-related data. The input is data from each application, and the output is the collected work-related data. Specific data collection operations involve the device collecting email data from email clients and retrieving chat messages from chat tools.

[1393] Step 2:

[1394] The server temporarily stores the collected business-related data. The input is the data collected from the terminal, and the output is the temporarily stored data. In this step, the specific operation of storing data on the server is to store emails and chat messages in temporary files in the database.

[1395] Step 3:

[1396] The server uses natural language processing to analyze the collected data, extracting and filtering the necessary information. The input is temporarily stored data, and the output is useful information after analysis. Specifically, it extracts important keywords from the body of an email and lists tasks from meeting minutes.

[1397] Step 4:

[1398] The server stores the analyzed data in a database. The input is useful information after analysis, and the output is data stored in a structured format. Specifically, in this step, important project progress information and task lists are stored in the database.

[1399] Step 5:

[1400] The server trains a machine learning model based on the data in the database. The input is the structured data in the database, and the output is the trained machine learning model. The specific operation of this step is to use past work data to learn each employee's work patterns and response history.

[1401] Step 6:

[1402] The server generates a work clone based on a trained machine learning model. The input is the trained machine learning model, and the output is a virtual representative for each employee. Specifically, a virtual work clone is created based on a specific employee's email response patterns and the content of comments made in meetings.

[1403] Step 7:

[1404] The emotion recognition engine uses natural language processing and voice analysis to recognize the user's emotions in real time during a conversation with the user. The input is the user's voice or text, and the output is the user's emotional information. Specifically, emotions (e.g., anger, confusion, joy) are analyzed from the customer's tone of voice or text.

[1405] Step 8:

[1406] The server dynamically adjusts the response style and content of the business clone based on emotional information from the emotion recognition engine. The input is emotional information from the emotion recognition engine and response data from the business clone, and the output is an adjusted response message. Specifically, if the customer is confused, the business clone changes the response content to provide a more polite and reassuring answer.

[1407] Step 9:

[1408] The terminal provides the tailored response message to the customer through the smart glasses. The input is the tailored response message, and the output is the information the customer receives through the smart glasses. Specifically, the response of the business clone is displayed on the display of the smart glasses.

[1409] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1410] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1411] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1412] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1413] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1414] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1415] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1416] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1417] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1418] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1419] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1420] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1421] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1422] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1423] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1424] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1425] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1426] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1427] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1428] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1429] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1430] The following is further disclosed regarding the above embodiment.

[1431] (Claim 1)

[1432] A means of collecting work-related data from applications used by individual employees;

[1433] means for analyzing the business-related data and extracting and filtering necessary information;

[1434] means for storing the extracted information in a database;

[1435] A means for training a machine learning model based on the database and generating a job clone for each employee;

[1436] a means for accessing the business clone and executing the business;

[1437] A system including:

[1438] (Claim 2)

[1439] 10. The system of claim 1, wherein the work-related data includes emails, chat messages, schedule information, and created documents.

[1440] (Claim 3)

[1441] 2. The system according to claim 1, wherein the business clone performs automatic email replies, meeting attendance, and document proofreading when an employee is absent.

[1442] "Example 1"

[1443] (Claim 1)

[1444] A means of collecting work-related data from software used by individual employees;

[1445] means for analyzing the business-related data and extracting and filtering necessary information using natural language processing;

[1446] means for storing the extracted information in a structured format in a data storage;

[1447] means for training a machine learning model based on the data in the data storage to generate a virtual representative for each employee;

[1448] means for accessing and conducting business with said virtual agent;

[1449] A system including:

[1450] (Claim 2)

[1451] 2. The system of claim 1, wherein the work-related data includes emails, messages, schedule data, and created files.

[1452] (Claim 3)

[1453] 10. The system of claim 1, wherein the virtual agent performs email auto-replies, meeting attendance, and document generation in an employee's absence.

[1454] "Application Example 1"

[1455] (Claim 1)

[1456] A means of collecting work-related data from applications used by individual employees;

[1457] means for analyzing the business-related data and extracting and filtering necessary information;

[1458] means for storing the extracted information in a database;

[1459] A means for training a machine learning model based on the database and generating a job clone for each employee;

[1460] A means for a user to access the business clone and perform a business through a web interface or a dedicated application;

[1461] A means for analyzing data relating to security incidents and proposing countermeasures;

[1462] A system including:

[1463] (Claim 2)

[1464] 10. The system of claim 1, wherein the work-related data and security incident data includes emails, chat messages, schedule information, and created documents.

[1465] (Claim 3)

[1466] 2. The system according to claim 1, wherein the business clone performs automatic email replies, meeting attendance, document proofreading, and security incident response proposals when an employee is absent.

[1467] "Example 2: Combining Emotion Engines"

[1468] (Claim 1)

[1469] A means of collecting work-related data from applications used by individual employees;

[1470] means for transmitting the business-related data to a server, analyzing the received business-related data, and extracting and filtering necessary information;

[1471] means for storing the extracted information in a database;

[1472] A means for training a machine learning model based on the database and generating a job clone for each employee;

[1473] means for integrating an emotion engine that recognizes user emotions in real time and adjusts the responses of said business clone;

[1474] A means for accessing the business clone and sending a question or task to the business clone to perform the business;

[1475] A system including:

[1476] (Claim 2)

[1477] 10. The system of claim 1, wherein the work-related data includes emails, chat messages, schedule information, and created documents.

[1478] (Claim 3)

[1479] 2. The system of claim 1, wherein the business clone performs automatic email replies, meeting attendance, and document confirmation when an employee is absent.

[1480] "Application example 2 when combining emotion engines"

[1481] (Claim 1)

[1482] A means of collecting work-related data from applications used by individual employees;

[1483] means for analyzing the business-related data and extracting and filtering necessary information;

[1484] means for storing the extracted information in a database;

[1485] A means for training a machine learning model based on the database and generating a job clone for each employee;

[1486] an emotion recognition means for recognizing the emotion of the user and dynamically adjusting the response style and content of the business clone;

[1487] a means for accessing the business clone and executing the business;

[1488] A system including:

[1489] (Claim 2)

[1490] 10. The system of claim 1, wherein the work-related data includes emails, chat messages, schedule information, and created documents.

[1491] (Claim 3)

[1492] 2. The system according to claim 1, wherein the business clone performs automatic email replies, meeting attendance, and document proofreading when an employee is absent.

[1493] (Claim 4)

[1494] 2. The system according to claim 1, wherein said emotion recognition means recognizes the user's emotions in real time and adjusts the response of the work clone. [Explanation of symbols]

[1495] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means of collecting work-related data from applications used by individual employees; means for analyzing the business-related data and extracting and filtering necessary information; means for storing the extracted information in a database; A means for training a machine learning model based on the database and generating a job clone for each employee; a means for accessing the business clone and executing the business; A system including:

2. 2. The system of claim 1, wherein the work-related data includes emails, chat messages, schedule information, and created documents.

3. 2. The system according to claim 1, wherein the business clone performs automatic email replies, meeting attendance, and document proofreading when an employee is absent.

Images