system

Abstract

We provide the system. [Solution] The information processing device provides a means for acquiring data related to past operations, A means of preprocessing the acquired data to remove noise and unnecessary information, A means for training a generative artificial intelligence model using preprocessed data, A means for generating answers to natural language questions from users based on a generative artificial intelligence model, A system that includes means for presenting a generated response to a user via a user interface.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional handover of work for new employees and transferred employees, information transmission between the persons in charge is often incomplete. As a result, there are frequent problems such as handover omissions and troubles due to insufficient information. This problem is particularly prominent when there is a complex work environment and a large amount of work data, and it is a factor that reduces work efficiency. In addition, it becomes difficult to ask questions after the person in charge retires or is transferred, and accumulation of experience and sharing of knowledge are often not properly carried out.

Means for Solving the Problems

[0005] This invention relates to a system in which an information processing device acquires data related to past business operations, preprocesses the acquired data to remove noise and unnecessary information, trains a generative artificial intelligence model using the preprocessed data, and generates answers to user questions in natural language based on the generative artificial intelligence model. Furthermore, it includes means for presenting the generated answers to the user via a user interface. This makes it possible to avoid problems caused by handover omissions and insufficient information, and to improve business efficiency. In addition, since the user interface is provided as a web application or desktop application, users can ask questions anytime, anywhere. Furthermore, the data preprocessing means includes a filtering function to mask personal and confidential information, thus ensuring security.

[0006] An "information processing device" is a device that acquires data, preprocesses it, trains artificial intelligence models, and generates answers to questions.

[0007] "Data acquisition" refers to the process of automatically collecting files and messages related to past business operations.

[0008] "Preprocessing" is the process of removing noise and unnecessary information from acquired data and converting it into a format suitable for analysis and learning.

[0009] A "generative artificial intelligence model" is a type of artificial intelligence that has the ability to learn from large amounts of data and generate appropriate responses to natural language input.

[0010] A "user" is an individual or group that uses this system to ask questions in natural language and receive answers.

[0011] A "question" refers to a user entering questions about their work in natural language.

[0012] "Answer generation" is the process by which a generative artificial intelligence model generates an appropriate answer based on the user's question.

[0013] A "user interface" is the interface through which a user accesses a system, inputs questions, and receives answers.

[0014] A "web application" is a software application that users access and use via the internet.

[0015] A "desktop application" is a software application that is installed on a user's computer and runs in a local environment.

[0016] A "filtering function" is a function that identifies personal and confidential information and removes or masks it.

[0017] "Ensuring security" refers to the measures and functions taken to protect the safety of data handled within a system. [Brief explanation of the drawing]

[0018] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8]It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0019] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0020] First, the language used in the following description will be explained.

[0021] In the following embodiments, a processor with a reference number (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of a plurality of arithmetic units. Further, the processor may be a single type of arithmetic unit or a combination of a plurality of types of arithmetic units. Examples of the arithmetic unit include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0022] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0023] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0024] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0026] [First Embodiment]

[0027] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0028] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0031] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0034] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0038] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0039] This invention is a system that utilizes generative artificial intelligence to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, thereby improving work efficiency. A detailed embodiment of this system will be described below.

[0040] Data collection

[0041] The server automatically retrieves historical work data for each department. Specifically, it collects relevant files and messages from the company's email server, chat server, file server, or cloud storage. For example, the server can download emails, chat history, and accounting-related documents from the finance department for the past three years via an API.

[0042] Data preprocessing

[0043] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, removes noise and unnecessary information (e.g., email signatures and advertisements), and masks personal and confidential information. This step generates a clean dataset.

[0044] AI model training and fine-tuning

[0045] Using preprocessed data, the server fine-tunes a generative artificial intelligence model (e.g., GPT-4®). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0046] Providing a user interface

[0047] After the generative artificial intelligence model is trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can enter questions through it. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[0048] Question submission and answer generation

[0049] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model, which generates an answer based on its trained knowledge. For example, the model might generate an answer such as, "At yesterday's meeting, we discussed a new sales strategy."

[0050] Providing a response

[0051] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[0052] Specific example

[0053] For example, suppose a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and a generative artificial intelligence model, trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view the answer.

[0054] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[0055] The following describes the processing flow.

[0056] Step 1:

[0057] The server retrieves past work data. Specifically, it collects data from mail servers, chat servers, file servers, or cloud storage. For example, the server uses APIs to download emails, chat history, and documents that are tagged or stored in specific folders.

[0058] Step 2:

[0059] The server converts the acquired data into text format. PDFs and scanned documents are converted into text data using OCR (Optical Character Recognition) technology.

[0060] Step 3:

[0061] The server preprocesses the converted text data. Specifically, it filters out noise and unnecessary information (e.g., email signatures, ad blockers) and cleans the data.

[0062] Step 4:

[0063] The server masks personal and confidential information during the data preprocessing process. Personal information such as names and email addresses are replaced with placeholders.

[0064] Step 5:

[0065] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using a pre-processed, clean dataset. It monitors the learning progress and adjusts hyperparameters as needed.

[0066] Step 6:

[0067] The server provides a user interface for users to enter questions. This interface is designed as either a web application or a desktop application.

[0068] Step 7:

[0069] The user logs into the user interface and enters their question in natural language. For example, they might type, "Please tell me the steps for preparing the latest quarterly financial report."

[0070] Step 8:

[0071] The terminal sends the entered question to the server. The text entered in the user interface is sent to the server as an HTTP POST request.

[0072] Step 9:

[0073] The server tokenizes the received question and queries a generative artificial intelligence model. The model generates an appropriate answer based on the training data.

[0074] Step 10:

[0075] The server restores the generated response to text format and formats it. For example, it might generate a response such as, "The procedure for preparing the latest quarterly financial report is to collect income and expenditure reports from each department, integrate them, and create the final financial statement."

[0076] Step 11:

[0077] The server sends the formatted response to the user interface.

[0078] Step 12:

[0079] The terminal displays the response received from the server on the user interface. The user can view this response and use it for their work.

[0080] (Example 1)

[0081] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0082] Traditional job handover processes frequently resulted in insufficient information and omissions, placing a significant burden on new and transferred employees in learning their new roles. Furthermore, manual information gathering and analysis were time-consuming and labor-intensive, leading to decreased work efficiency. Therefore, a system was needed to efficiently collect and analyze past work data and provide new and transferred employees with timely and accurate information.

[0083] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0084] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, means for converting the data into text format, means for masking personal and confidential information, means for dynamically adjusting the hyperparameters of the generative artificial intelligence model, means for sending the user's questions to the server as HTTP requests, and means for formatting the generated answers into a format that is easy for the user to understand. This enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and handover omissions, and improve work efficiency.

[0085] An "information processing device" is a system that includes hardware and software for collecting, processing, analyzing, and providing data results.

[0086] "Data related to past work" refers to records of work performed by a specific department within a company in the past, and includes emails, chat history, documents, etc.

[0087] "Preprocessing" is the process of converting collected data into an analyzable format by removing noise and unnecessary information, and masking personal and confidential information.

[0088] A "generative artificial intelligence model" is an AI model trained to perform natural language processing tasks using large amounts of text data, and models like GPT-4 fall into this category.

[0089] "Natural language questions from users" are inquiries or questions written by users in human language, specifically those entered as sentences or phrases.

[0090] A "means for generating answers" refers to a system that uses a generative artificial intelligence model to perform processing to generate appropriate answers to user questions.

[0091] A "user interface" is the part that provides the visual and manipulative means for a user to interact with a system, and is implemented as a web application or desktop application.

[0092] "Means of converting to text format" refers to the process of converting collected data into text data, and may involve using technologies such as OCR (Optical Character Recognition).

[0093] "Methods for masking personal and confidential information" refer to systems that process collected data to conceal information that could identify a specific individual or confidential company information.

[0094] "Methods for dynamically adjusting hyperparameters" refers to the process of dynamically changing parameters such as learning rate and batch size in order to optimize the performance of a generative artificial intelligence model.

[0095] "Means of sending to a server as an HTTP request" refers to the means by which a user enters a question from their terminal and sends it to a server via the Internet Protocol.

[0096] "Means of formatting into a user-friendly format" refers to a system that processes generated responses into an appropriate format so that users can easily understand them.

[0097] This invention is an information processing system that utilizes generative artificial intelligence to prevent information deficiencies and omissions during the handover of duties between new employees and transferred employees, and to improve work efficiency. The embodiments for carrying out this invention will be described in detail below.

[0098] First, the server automatically collects data related to past business operations. This data is retrieved from the company's email server, chat server, file server, or cloud storage. For example, the server uses an API to download emails, chat history, and accounting-related documents from the finance department for the past three years.

[0099] The collected data is preprocessed by the server. Preprocessing includes converting the data to text format, removing noise and unwanted information, and masking personal and sensitive information. For example, the server uses Python libraries (such as pdfminer and beautifulsoup) to convert documents to text format and removes unwanted information using regular expressions. It also uses NER (Named Entity Recognition) to identify and appropriately mask sensitive information.

[0100] Next, using the preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). The server sets up the training environment (e.g., using TENSORFLOW® or PyTorch), sets the hyperparameters appropriately, and trains the model. During training, the server monitors the progress and dynamically adjusts the hyperparameters as needed. For example, it might set the learning rate to 0.001 and the batch size to 32, and train for 500 epochs.

[0101] Once the training is complete, the generative artificial intelligence model becomes available through a user interface. The server provides the user interface as a web application or desktop application. Users can log in to the system through this interface and enter questions. For example, a new employee could ask, "What were the key topics from yesterday's meeting?"

[0102] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and generates an appropriate answer. The generated answer is then formatted by the server into a user-friendly format and sent back to the terminal. The terminal displays this answer on its user interface, allowing the user to instantly obtain an answer to their question.

[0103] As a concrete example, consider a scenario where a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" This question is sent from the terminal to the server, and the server, using a generative artificial intelligence model trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view it.

[0104] As a result, the system of the present invention enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and omissions in handover, and improve work efficiency.

[0105] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0106] Step 1:

[0107] Data collection

[0108] The server automatically retrieves data related to past business operations. First, the server connects to the company's mail server, chat server, file server, or cloud storage via API calls. Then, the server downloads emails, chat history, and documents from each department for the past three years. For example, the server retrieves documents from the finance department from AWS® S3 and downloads email history using the Gmail API.

[0109] Input: Data from the company's email server, chat server, and file server.

[0110] Output: Email, chat history, and documents for each department over the past 3 years.

[0111] Step 2:

[0112] Data preprocessing

[0113] The server preprocesses the collected data. First, it converts the collected data into text format. It uses Python libraries (e.g., pdfminer or beautifulsoup) to convert documents to text format. Next, it uses regular expressions to remove unwanted information and noise (e.g., email signatures and advertisements). Finally, it uses NER (Named Entity Recognition) to mask personal and sensitive information.

[0114] Input: Email and chat history for each department over the past three years, and documents.

[0115] Output: Data converted to text format, with unnecessary information removed and personal and confidential information masked.

[0116] Step 3:

[0117] AI model training and fine-tuning

[0118] The server trains and fine-tunes generative artificial intelligence models using preprocessed data. First, it prepares the training dataset using Python and sets up the training environment using TensorFlow or PyTorch. The server sets hyperparameters such as the learning rate and batch size and starts training the model. During training, the server monitors the learning progress and dynamically adjusts the hyperparameters as needed.

[0119] Input: Data converted to text format, with personal and confidential information masked.

[0120] Output: Trained and fine-tuned generative artificial intelligence model

[0121] Step 4:

[0122] Providing a user interface

[0123] The server provides a user interface for using generative artificial intelligence models. First, the user interface for the web application is designed and implemented using React.js, allowing users to access the system. If provided as a desktop application, the interface is built using Electron or similar technologies. The server also manages user authentication information in a database (e.g., MySQL®, PostgreSQL) and integrates it into the user interface.

[0124] Input: Generative artificial intelligence model

[0125] Output: User interface provided as a web application or desktop application.

[0126] Step 5:

[0127] Question submission and answer generation

[0128] When a user enters a question, it is sent to the server via the terminal. The terminal sends the question to the server as an HTTP request, and the server poses the question to a generative artificial intelligence model. The model generates an appropriate answer based on its trained knowledge. For example, in response to the question, "What were the main topics of discussion at yesterday's meeting?", the model might generate the answer, "At yesterday's meeting, a new sales strategy was discussed."

[0129] Input: Questions entered via the user interface

[0130] Output: Answers generated by a generative artificial intelligence model

[0131] Step 6:

[0132] Providing a response

[0133] The server formats the generated response and returns it to the terminal via the user interface. The terminal receives this response and displays it on the user interface. Specifically, the response is displayed on the interface of a web application or desktop application, allowing the user to instantly obtain an answer to their question.

[0134] Input: Answer generated by a generative artificial intelligence model

[0135] Output: Answer displayed on the user interface

[0136] (Application Example 1)

[0137] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0138] New employees and transferred employees are required to quickly and efficiently understand their duties in the factory environment and perform their work smoothly. However, traditional handover processes often result in insufficient information or omissions, preventing new employees from immediately adapting to on-site situations. This raises concerns about decreased productivity and increased errors. Therefore, a system is needed that can effectively facilitate job handover in a factory environment.

[0139] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0140] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, and a user interface related to smart glasses that provide visual work instructions and explanations, particularly in a factory environment. This enables new employees and transferred employees to quickly become proficient in their work within the factory, improving productivity and reducing errors.

[0141] An "information processing device" is a device that performs tasks such as data collection, processing, analysis, and output.

[0142] "Data related to past work" includes all information related to a specific task, such as work processes, work history, meeting records, and message history.

[0143] "Preprocessing" is the process of removing noise and unnecessary information from collected data to make it suitable for useful analysis.

[0144] A "generative artificial intelligence model" is a machine learning model specialized in natural language processing and data generation. It learns from large amounts of data and is capable of answering questions and generating text.

[0145] A "user interface" is a collection of screens, input devices, and interaction methods that a user uses when operating a system.

[0146] "Smart glasses" are glasses-type devices worn by the wearer that have the function of displaying information visually.

[0147] "Visual work instructions" refer to instructions that visually indicate work procedures and precautions using text, images, videos, etc.

[0148] "Training methods" refer to the techniques and processes used to train a generative artificial intelligence model using collected and pre-processed data.

[0149] "Means of generating answers" refers to the process by which a generative artificial intelligence model creates an appropriate answer to a question from a user.

[0150] This invention provides a system that supports new employees and transferred employees in a factory environment to quickly and efficiently understand their tasks and perform their work smoothly. Specific embodiments of this system are described below.

[0151] System Overview

[0152] The server collects data from the factory department to retrieve data related to past operations. Specifically, it collects relevant files and messages from the company's mail server, chat server, file server, or cloud storage as means of data acquisition. For example, the server can download past work data and records via an API.

[0153] Since the acquired data cannot be used as is, the server preprocesses the data. As a preprocessing step, the data is converted to text format, noise and unnecessary information (e.g., email signatures and advertisements) are removed, and personal and confidential information is masked. This step generates a clean dataset.

[0154] Training of generative artificial intelligence models

[0155] Using a clean dataset, the server trains a generative artificial intelligence model (e.g., GPT-4). The model learns key information and patterns related to specific tasks as part of the training process. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0156] The server uses a generative artificial intelligence model to generate answers to natural language questions from users. As a means of generating answers, users input questions through smart glasses, and these questions are sent to the server. The server then poses the question to a trained AI model and generates an answer. For example, a new employee might ask, "What is the next step?", and the AI ​​might answer, "Attach part A to part B."

[0157] User Interface

[0158] The server presents the generated answers to the user via a user interface. In particular, in factory environments, smart glasses are used to provide visual work instructions and explanations. This allows new employees and transferred employees to resolve questions in real time while performing their actual work.

[0159] Hardware and software to be used

[0160] Hardware: Smart glasses (e.g., Vuzix Blade)

[0161] Software: OpenAI® GPT-4 API, data preprocessing tools

[0162] As a concrete example, a new employee wearing smart glasses stands on a manufacturing line and asks, "What is the next step?" This question is sent to a server, and a generative artificial intelligence model generates the answer, "Attach part A to part B." This answer is displayed on the smart glasses' screen, allowing the new employee to follow the instructions and proceed with the task.

[0163] Example of a prompt

[0164] "Manufacturing Line Work Instruction Prompt: Generate detailed instructions to help new employees understand the steps on the manufacturing line. For example, write instructions for attaching part A to part B."

[0165] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[0166] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0167] Step 1:

[0168] Data collection

[0169] The server uses means to retrieve data related to past work. Specifically, the server collects data from the company's mail server, chat server, file server, or cloud storage. It downloads past work data and records via API and saves them to local storage. Inputs include emails, chat history, and work manuals, and output is raw data stored in local storage.

[0170] Step 2:

[0171] Data preprocessing

[0172] The server uses means to preprocess the acquired data. Specifically, it removes noise and unnecessary information (e.g., email signatures, advertisements) from the collected data and converts it to text format. It also filters and masks personal and confidential information. Raw data is given as input, and a clean dataset is generated as output. Specific data preprocessing steps include text cleaning using natural language processing tools.

[0173] Step 3:

[0174] Training of artificial intelligence models

[0175] The server performs a means of training a generative artificial intelligence model using preprocessed data. Specifically, it fine-tunes the model (e.g., GPT-4) based on a large amount of text data. The server monitors the model's performance and dynamically adjusts hyperparameters as needed. A clean dataset is used as input, and the output is a trained generative artificial intelligence model.

[0176] Step 4:

[0177] Accepting questions

[0178] The device (smart glasses) accepts questions from the user in natural language. The user inputs the question through the smart glasses' interface, and this is sent to the server. The input is the question from the user, and the output is the question data sent to the server.

[0179] Step 5:

[0180] Answer generation

[0181] The server uses a means of generating answers to user questions based on a generative artificial intelligence model. Specifically, the server feeds question data to the generative AI model and generates an appropriate answer. The question data is used as input, and the generated answer is obtained as output.

[0182] Step 6:

[0183] Providing an answer

[0184] The server performs a means of presenting the answer via a user interface. The generated answer is visually displayed to the user through smart glasses. The input is the generated answer, and the output is the answer displayed on the terminal (smart glasses).

[0185] This processing flow allows new employees and transferred employees to receive timely and relevant information, improving factory work efficiency.

[0186] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0187] This invention is a system that utilizes generative artificial intelligence and an emotion engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency. A detailed embodiment of this system will be described below.

[0188] Data collection

[0189] The server automatically retrieves historical work data for each department. Specifically, it collects data from email servers, chat servers, file servers, or cloud storage. For example, the server can use APIs to download emails, chat histories, and documents that are tagged or stored in specific folders.

[0190] Data preprocessing

[0191] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, filters out noise and unnecessary information (e.g., email signatures, ad blockers), and masks personal and confidential information. This step generates a clean dataset.

[0192] AI model training and fine-tuning

[0193] Using preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0194] Emotional engine integration

[0195] Furthermore, the server combines a generative artificial intelligence model with an emotion engine. The emotion engine can recognize and evaluate emotions from user input and interactions. It detects user emotions using text analysis, speech analysis, or facial recognition. Based on this emotion information, the tone and content of the generated responses can be adjusted.

[0196] Providing a user interface

[0197] After the generative AI model and emotion engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can input questions through it and receive emotion-based responses. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[0198] Question submission and answer generation

[0199] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and an emotion engine, which generates an appropriate answer based on its trained knowledge and the user's emotional state. For example, it might generate an answer that takes the user's emotions into consideration, such as, "At yesterday's meeting, we discussed a new sales strategy."

[0200] Providing a response

[0201] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[0202] Specific example

[0203] For example, suppose a new employee is assigned to the finance department. The new employee, as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and the emotion engine senses anxiety or tension from the user's input. The generative artificial intelligence model takes this emotional information into consideration and generates a response in a gentle tone, such as, "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement. If you have any problems, please feel free to ask." The server returns this response to the terminal, and the user can see the answer.

[0204] In this way, the system of the present invention utilizes past business data to respond quickly and accurately to questions from new employees and employees transferring to other departments, and by integrating an emotion engine, it provides responses that take into account the user's emotions, thereby enabling a smooth handover of duties.

[0205] The following describes the processing flow.

[0206] Step 1:

[0207] The server automatically retrieves historical business data from email servers, chat servers, file servers, or cloud storage. For example, the server uses an API to download email and chat history from the finance department for the past three years.

[0208] Step 2:

[0209] The server converts the acquired data into text format. For example, PDF files and scanned documents are converted into text data using OCR (Optical Character Recognition). Excel files and Word documents are also converted into text format.

[0210] Step 3:

[0211] The server preprocesses the retrieved text data to remove noise and unwanted information (e.g., email signatures and ad blockers). For example, it might use regular expressions to filter out footer text such as "Best regards" or advertisements.

[0212] Step 4:

[0213] The server masks personal and confidential information during the preprocessing stage. For example, names and email addresses are replaced with placeholders such as "[Name]" and "[Email Address]".

[0214] Step 5:

[0215] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using pre-processed data. During this process, the server uses a learning framework (e.g., PyTorch) to ensure the model learns properly.

[0216] Step 6:

[0217] The server integrates an emotion engine into the generative artificial intelligence model. For example, the emotion engine incorporates algorithms that detect and evaluate the user's emotions using text analysis, speech analysis, or facial recognition.

[0218] Step 7:

[0219] The user logs into the user interface and enters a question. This interface is provided as a web application or desktop application and includes a form for the user to enter the question in natural language.

[0220] Step 8:

[0221] The terminal sends the user-entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[0222] Step 9:

[0223] The server inputs the received question into a generative artificial intelligence model and an emotion engine. At this time, the server tokenizes the question and passes it to the model as input data.

[0224] Step 10:

[0225] The emotion engine analyzes emotional information from the user's questions. For example, the emotion engine extracts emotional tags such as "tension," "anxiety," and "joy" from the text of the question.

[0226] Step 11:

[0227] Generative artificial intelligence models generate answers to questions while taking emotional information into account. For example, if the user is nervous, the model will soften the tone of the answer and use gentle expressions such as, "I'll explain it simply, so don't worry."

[0228] Step 12:

[0229] The server restores the generated response to text format and formats it. For example, it might generate a response like, "The procedure for preparing the most recent quarterly financial report is to collect income and expenditure reports from each department, integrate them, and then create the final financial statement. Please feel free to ask questions if you have any problems."

[0230] Step 13:

[0231] The server sends the formatted response to the user interface. Specifically, it returns the response data as an HTTP response.

[0232] Step 14:

[0233] The terminal displays the response received from the server on the user interface. Users can view this response and use it to help with their work. For example, they can gain a concrete understanding of "the procedure for preparing the most recent quarterly financial report."

[0234] In this way, by using a generative artificial intelligence model that integrates an emotion engine, appropriate responses can be obtained that are tailored to the user's emotional state. This not only facilitates smooth handover of tasks but also reduces the psychological burden on the user.

[0235] (Example 2)

[0236] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0237] In companies and organizations, the handover of duties for new employees and employees transferring to other departments can sometimes be unsuccessful. In such cases, important information may be leaked, and knowledge related to the work may not be properly shared, leading to decreased work efficiency and an increased likelihood of errors. Furthermore, new employees and transferring employees often experience anxiety and stress during the handover process, which further reduces work efficiency. Traditional handover methods are insufficient to adequately address these problems.

[0238] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information and mask personal and confidential information, means for training a generative artificial intelligence model using the preprocessed data, means for integrating an emotion recognition engine into the generative artificial intelligence model and evaluating emotions from user input, means for generating answers to natural language questions from the user based on the integrated generative artificial intelligence model and adjusting the tone and content of the answers based on the user's emotions, and means for presenting the generated answers to the user via a user interface. As a result, it is possible to obtain responses that take into account the emotions the user feels during the handover of operations, preventing information handover omissions and incomplete information sharing, and enabling increased operational efficiency and reduced troubles.

[0239] An "information processing device" is a combination of hardware and software for collecting, preprocessing, analyzing, generating, and outputting data.

[0240] "Data related to past business operations" refers to documents and communication data that contain information about business activities carried out in the past.

[0241] "Preprocessing" refers to the process of removing noise and unwanted information, converting formats, and masking personal and confidential information in order to transform collected data into a usable format.

[0242] A "generative artificial intelligence model" refers to a machine learning algorithm that can find patterns in large amounts of data and generate new data.

[0243] An "emotion recognition engine" is an algorithm or software that detects emotions from user input and actions and provides feedback based on those emotions.

[0244] "User" refers to an individual or member of an organization that uses this system.

[0245] "Means for generating answers to questions" refers to a combination of software and hardware that analyzes natural language questions from users and generates appropriate answers.

[0246] A "user interface" is an application that has a display screen and input mechanism for a user to interact with the system.

[0247] "Noise and unnecessary information" refers to irrelevant data and information that does not affect the analysis or generation process.

[0248] "Personal information and confidential information" refers to information that includes the identification of an individual or highly confidential content, and is subject to confidentiality obligations.

[0249] "Adjusting the tone and content of responses" refers to generating responses that are not cruel, taking into account the user's emotional state.

[0250] "Real-time" refers to the time range in which data is processed and outputted with only a slight delay after it is input.

[0251] An "API" refers to an interface used to access the functions of different software programs.

[0252] A "web application" refers to software that can be accessed through an internet browser.

[0253] A "desktop application" refers to software that runs directly on a personal computer.

[0254] This invention is a system that utilizes generative artificial intelligence and an emotion recognition engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency.

[0255] The server first automatically retrieves data related to past work for each department. Data collection utilizes services such as email servers (e.g., Microsoft® Exchange), chat servers (Slack), file servers (NAS), and cloud storage (Google® Drive, Dropbox). The server uses APIs provided by these services (e.g., Google Drive API and Slack API) to download emails, chat histories, and documents associated with specific folders or tags. This allows users to gain a comprehensive overview of the information they need.

[0256] The collected data undergoes preprocessing. The server converts the data to text format using Python's BeautifulSoup or NLP libraries (e.g., NLTK, SpaCy), removes noise and unnecessary information (such as email signatures and ad blockers), and masks personal and sensitive information. This step generates a clean and usable dataset.

[0257] Next, the server trains a generative artificial intelligence model (e.g., GPT-4) using the preprocessed data. Machine learning libraries such as PyTorch and TensorFlow are used for training, the training progress is monitored, and hyperparameters (such as learning rate and batch size) are dynamically adjusted as needed. This allows the model to learn important information and patterns related to specific tasks.

[0258] Furthermore, the emotion recognition engine is integrated into the generative artificial intelligence model. The server uses the emotion recognition engine to evaluate emotions from user input and interactions. For this purpose, TextBlob is used for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. Based on the emotional information, the tone and content of the generated responses will be adjusted. For example, if a user enters a question while feeling anxious, a response in a gentle tone that takes that emotion into account will be provided.

[0259] After the generative AI model and emotion recognition engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, using React or Vue.js for the frontend and Node.js or Django for the backend. Users can input questions through the interface and receive answers in real time.

[0260] For example, if a new employee, who is a user, enters a question such as "Please tell me the procedure for preparing the most recent quarterly financial report," that question will be sent to the server via the terminal.

[0261] The server poses this question to a generative artificial intelligence model and an emotion recognition engine, which analyze the data and generate a response tailored to the user's emotional state. For example, if the emotion recognition engine detects anxiety from the user's input, it will generate a gentle response such as, "The procedure for preparing the most recent quarterly financial report involves first collecting income and expenditure reports from each department, and then integrating them to create the final financial statement. Please feel free to ask if you have any problems."

[0262] The generated answers are sent from the server to the terminal, and the user can display the answers on the screen. In this way, new employees and employees transferring to other departments can get quick and accurate answers to their work-related questions, leading to increased work efficiency.

[0263] This system provides responses that take into account the emotions users feel during the handover of tasks, preventing information from being missed or incomplete, and enabling increased work efficiency and a reduction in problems.

[0264] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0265] Step 1: Data Collection

[0266] The server collects data related to past business operations from cloud storage, mail servers, chat servers, and file servers. Specifically, the server uses APIs (e.g., Google Drive API, Slack API) to download emails, chat histories, and documents tagged with specific folders from each service. It accepts API keys and folder paths from each service as input and saves the downloaded data as a file as output.

[0267] Step 2: Data Preprocessing

[0268] The server preprocesses the collected data. It converts the data to plain text and removes noise and unnecessary information. Specifically, it uses Python libraries (e.g., BeautifulSoup, NLTK, SpaCy) to convert HTML emails to plain text and remove advertisements and personal information. It receives downloaded raw data as input and generates clean text data as output.

[0269] Step 3: Training the AI ​​model

[0270] The server trains a generative artificial intelligence model (e.g., GPT-4) using pre-processed text data. Specifically, it uses PyTorch or TensorFlow to feed a large amount of data into the model and train it. It takes clean text data as input and produces a trained model as output.

[0271] Step 4: Integrating the emotion recognition engine

[0272] The server integrates an emotion recognition engine with a trained generative artificial intelligence model. It analyzes user emotions using TextBlob for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. It receives user interaction data (text, audio, images) as input and outputs analyzed emotion information.

[0273] Step 5: Provide User Interface

[0274] The server provides the user interface. It is designed as a web application (React, Node.js) or a desktop application (Electron). Through this interface, users can input questions and receive answers in real time. It receives user questions as input and displays generated answers as output.

[0275] Step 6: Accepting Questions

[0276] The user enters a question into the user interface, and this is sent to the server via the terminal. Specifically, the entered question is sent to the API endpoint and received as data for processing. It receives the user's question as input and obtains data for processing as output.

[0277] Step 7: Generate Response

[0278] The server uses a generative artificial intelligence model and an emotion recognition engine to generate answers to user questions. For example, if it detects anxiety from the user's input, the model will generate an answer in a gentle tone. It receives user questions and emotion data as input and obtains the generated answer as output.

[0279] Step 8: Provide your answer

[0280] The server formats the generated answer and returns it to the terminal. The terminal interprets this answer and displays it on the user interface. Specifically, it displays the JSON-formatted answer data sent through the API endpoint in HTML format. It receives the generated answer as input and provides the answer presented to the user as output.

[0281] With this specific processing step, an environment is established where new employees and transferred employees in each department can smoothly take over the work, and it is possible to improve work efficiency.

[0282] (Application Example 2)

[0283] Next, Application Example 2 will be described. In the following description, the data processing device 12 is referred to as the "server", and the smart device 14 is referred to as the "terminal".

[0284] In the conventional work handover system, there were problems such that it was difficult for new employees and transferred employees to quickly and accurately acquire information related to their work. Also, inefficiencies and mistakes in work due to insufficient information and handover omissions were likely to occur, and in addition, responses considering the feelings of users were often insufficient. Furthermore, at sites such as logistics centers, support was required to make it easier for staff to understand complex work processes.

[0285] The specific processing by the specific processing unit 290 of the data processing apparatus 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for including an emotion engine that analyzes the user's emotion and adjusts the tone of the response based on the result, and means for presenting the response generated by the user via a smartphone application. As a result, new employees and transferred employees can quickly and accurately answer questions related to their work, and can provide responses that take emotions into account. In addition, support is provided to make it easier for the staff at the logistics center to immediately understand the work procedures and processes.

[0286] An "information processing apparatus" is an apparatus that acquires past business data, performs preprocessing of the data, training of a generative artificial intelligence model, and generation of responses to questions in natural language from the user.

[0287] A "generative artificial intelligence model" is an artificial intelligence model that is trained using the collected data, learns information and patterns related to specific operations, and generates responses to the user's questions.

[0288] An "emotion engine" is a system that recognizes and evaluates emotions from the user's input and interactions, and is used to adjust the tone and content of the responses generated by the generative artificial intelligence model.

[0289] "Preprocessing" refers to a series of processes that convert the collected data into text format, remove noise and unnecessary information, and mask personal information and confidential information.

[0290] A "user interface" refers to an interface through which the user accesses this system, inputs questions, or checks the generated responses.

[0291] A "smartphone application" is an application that runs on a smartphone and allows users to input work-related questions and receive answers.

[0292] A "logistics center" refers to a facility or area where logistics operations such as the storage, management, and shipping of goods are carried out.

[0293] "Tone" refers to the manner in which the generated response is spoken and expressed, and it is adjusted to provide a user-friendly response in a kind and gentle tone.

[0294] The system for implementing this invention is centered around an information processing device and consists of the following steps.

[0295] First, the server automatically retrieves data related to past operations from various sources. These sources include email servers, chat servers, file servers, and cloud storage. Specifically, the server uses APIs to download emails, chat history, documents, and other files that are tagged or stored in specific folders.

[0296] Next, the server preprocesses the acquired data. This preprocessing includes converting the data to text format, filtering out noise and unnecessary information (such as email signatures and ad blockers), and masking personal and sensitive information. This generates a clean dataset.

[0297] Next, the server uses the preprocessed data to train and fine-tune a generative artificial intelligence model (e.g., GPT-4). This allows the model to learn important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0298] Subsequently, the server integrates an emotion engine into the generative artificial intelligence model. The emotion engine has the ability to recognize and evaluate emotions from user input and interactions. This utilizes text analysis, speech analysis, or facial recognition technology. Based on the emotional information, the tone and content of the generated responses are adjusted.

[0299] The user interface is provided in the form of a smartphone application. Through this application, users can input work-related questions in text format and receive immediate answers. Once a question is entered, it is sent via the device to a server, which generates an answer based on a generative artificial intelligence model and an emotion engine. This answer is generated in an appropriate tone, taking into account the user's emotional state.

[0300] For example, if a user asks, "Could you please explain the procedure for taking inventory?", the emotion engine senses tension and anxiety from the user's input. The generative artificial intelligence model takes this emotional information into account and generates a response in a gentle tone, such as, "The procedure for taking inventory is as follows: First, check the quantity of each product, and then enter it into the database. Please let us know if you have any problems." This response is then provided to the user through the smartphone application interface.

[0301] Examples of prompt statements include the following:

[0302] Question to the AI: "Please explain the procedure for taking inventory."

[0303] Emotional status: "Nervous"

[0304] Answer: "The procedure for inventory counting is as follows: First, confirm the quantity of each product, and then enter it into the database. Please let us know if you have any problems."

[0305] In this way, new employees and transferred employees can quickly and accurately solve questions related to their work, and the staff in the logistics center is also provided with support to immediately understand the work procedures and processes.

[0306] The flow of the specific process in Application Example 2 will be described using FIG. 14.

[0307] Step 1:

[0308] The server acquires data related to past work. Specifically, it downloads emails, chat histories, and documents with specific folders or tags from a mail server, a chat server, a file server, or cloud storage using an API. The input for this step is the API request, and the output is the downloaded original data.

[0309] Step 2:

[0310] The server preprocesses the acquired data. First, it converts the data into text format, and then filters out noise and unnecessary information (e.g., email signatures and advertising blocks). It also masks personal information and confidential information. The input for this step is the downloaded original data, and the output is clean text data.

[0311] Step 3:

[0312] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using the preprocessed data. Clean text data is used as training data to let the model learn important information and patterns related to specific work. The input for this step is clean text data, and the output is the trained AI model.

[0313] Step 4:

[0314] The server integrates an emotion engine into the generative artificial intelligence model. The emotion engine recognizes and evaluates emotions from user input and interactions, primarily through text analysis. The input for this step is a trained AI model and a dataset for emotion analysis, while the output is a turnkey system that generates emotion-sensitive responses.

[0315] Step 5:

[0316] The user uses a smartphone application to input questions related to their work. For example, they might input, "Please explain the procedure for taking inventory." The input in this step is text input by the user, and the output is that text question.

[0317] Step 6:

[0318] The terminal sends the user's question to the server. The server receives the question, performs sentiment analysis, and generates an answer using a generative artificial intelligence model. The input for this step is the question text sent by the user, and the output is the generated answer.

[0319] Step 7:

[0320] The server formats the generated response and sends it back to the user via a smartphone application. The input for this step is the generated response, and the output is the final formatted response text.

[0321] Step 8:

[0322] The user reviews the responses generated through a smartphone application. For example, they might receive a response such as, "The inventory count procedure is as follows: First, check the quantity of each product, then enter it into the database. Please let us know if you have any problems." The input in this step is the formatted response returned from the server, and the output is the user's understanding and actions.

[0323] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0324] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0325] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0326] [Second Embodiment]

[0327] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0328] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0329] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0330] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0331] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0332] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0333] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0334] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0335] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0336] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0337] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0338] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0339] This invention is a system that utilizes generative artificial intelligence to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, thereby improving work efficiency. A detailed embodiment of this system will be described below.

[0340] Data collection

[0341] The server automatically retrieves historical work data for each department. Specifically, it collects relevant files and messages from the company's email server, chat server, file server, or cloud storage. For example, the server can download emails, chat history, and accounting-related documents from the finance department for the past three years via an API.

[0342] Data preprocessing

[0343] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, removes noise and unnecessary information (e.g., email signatures and advertisements), and masks personal and confidential information. This step generates a clean dataset.

[0344] AI model training and fine-tuning

[0345] Using pre-processed data, the server fine-tunes a generative artificial intelligence model (e.g., GPT-4). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0346] Providing a user interface

[0347] After the generative artificial intelligence model is trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can enter questions through it. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[0348] Question submission and answer generation

[0349] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model, which generates an answer based on its trained knowledge. For example, the model might generate an answer such as, "At yesterday's meeting, we discussed a new sales strategy."

[0350] Providing a response

[0351] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[0352] Specific example

[0353] For example, suppose a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and a generative artificial intelligence model, trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view the answer.

[0354] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[0355] The following describes the processing flow.

[0356] Step 1:

[0357] The server retrieves past work data. Specifically, it collects data from mail servers, chat servers, file servers, or cloud storage. For example, the server uses APIs to download emails, chat history, and documents that are tagged or stored in specific folders.

[0358] Step 2:

[0359] The server converts the acquired data into text format. PDFs and scanned documents are converted into text data using OCR (Optical Character Recognition) technology.

[0360] Step 3:

[0361] The server preprocesses the converted text data. Specifically, it filters out noise and unnecessary information (e.g., email signatures, ad blockers) and cleans the data.

[0362] Step 4:

[0363] The server masks personal and confidential information during the data preprocessing process. Personal information such as names and email addresses are replaced with placeholders.

[0364] Step 5:

[0365] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using a pre-processed, clean dataset. It monitors the learning progress and adjusts hyperparameters as needed.

[0366] Step 6:

[0367] The server provides a user interface for users to enter questions. This interface is designed as either a web application or a desktop application.

[0368] Step 7:

[0369] The user logs into the user interface and enters their question in natural language. For example, they might type, "Please tell me the steps for preparing the latest quarterly financial report."

[0370] Step 8:

[0371] The terminal sends the entered question to the server. The text entered in the user interface is sent to the server as an HTTP POST request.

[0372] Step 9:

[0373] The server tokenizes the received question and queries a generative artificial intelligence model. The model generates an appropriate answer based on the training data.

[0374] Step 10:

[0375] The server restores the generated response to text format and formats it. For example, it might generate a response such as, "The procedure for preparing the latest quarterly financial report is to collect income and expenditure reports from each department, integrate them, and create the final financial statement."

[0376] Step 11:

[0377] The server sends the formatted response to the user interface.

[0378] Step 12:

[0379] The terminal displays the response received from the server on the user interface. The user can view this response and use it for their work.

[0380] (Example 1)

[0381] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0382] Traditional job handover processes frequently resulted in insufficient information and omissions, placing a significant burden on new and transferred employees in learning their new roles. Furthermore, manual information gathering and analysis were time-consuming and labor-intensive, leading to decreased work efficiency. Therefore, a system was needed to efficiently collect and analyze past work data and provide new and transferred employees with timely and accurate information.

[0383] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0384] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, means for converting the data into text format, means for masking personal and confidential information, means for dynamically adjusting the hyperparameters of the generative artificial intelligence model, means for sending the user's questions to the server as HTTP requests, and means for formatting the generated answers into a format that is easy for the user to understand. This enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and handover omissions, and improve work efficiency.

[0385] An "information processing device" is a system that includes hardware and software for collecting, processing, analyzing, and providing data results.

[0386] "Data related to past work" refers to records of work performed by a specific department within a company in the past, and includes emails, chat history, documents, etc.

[0387] "Preprocessing" is the process of converting collected data into an analyzable format by removing noise and unnecessary information, and masking personal and confidential information.

[0388] A "generative artificial intelligence model" is an AI model trained to perform natural language processing tasks using large amounts of text data, and models like GPT-4 fall into this category.

[0389] "Natural language questions from users" are inquiries or questions written by users in human language, specifically those entered as sentences or phrases.

[0390] A "means for generating answers" refers to a system that uses a generative artificial intelligence model to perform processing to generate appropriate answers to user questions.

[0391] A "user interface" is the part that provides the visual and manipulative means for a user to interact with a system, and is implemented as a web application or desktop application.

[0392] "Means of converting to text format" refers to the process of converting collected data into text data, and may involve using technologies such as OCR (Optical Character Recognition).

[0393] "Methods for masking personal and confidential information" refer to systems that process collected data to conceal information that could identify a specific individual or confidential company information.

[0394] "Methods for dynamically adjusting hyperparameters" refers to the process of dynamically changing parameters such as learning rate and batch size in order to optimize the performance of a generative artificial intelligence model.

[0395] "Means of sending to a server as an HTTP request" refers to the means by which a user enters a question from their terminal and sends it to a server via the Internet Protocol.

[0396] "Means of formatting into a user-friendly format" refers to a system that processes generated responses into an appropriate format so that users can easily understand them.

[0397] This invention is an information processing system that utilizes generative artificial intelligence to prevent information deficiencies and omissions during the handover of duties between new employees and transferred employees, and to improve work efficiency. The embodiments for carrying out this invention will be described in detail below.

[0398] First, the server automatically collects data related to past business operations. This data is retrieved from the company's email server, chat server, file server, or cloud storage. For example, the server uses an API to download emails, chat history, and accounting-related documents from the finance department for the past three years.

[0399] The collected data is preprocessed by the server. Preprocessing includes converting the data to text format, removing noise and unwanted information, and masking personal and sensitive information. For example, the server uses Python libraries (such as pdfminer and beautifulsoup) to convert documents to text format and removes unwanted information using regular expressions. It also uses NER (Named Entity Recognition) to identify and appropriately mask sensitive information.

[0400] Next, using the preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). The server sets up the training environment (e.g., using TensorFlow or PyTorch), sets the hyperparameters appropriately, and trains the model. During training, the server monitors the progress and dynamically adjusts the hyperparameters as needed. For example, it might set the learning rate to 0.001 and the batch size to 32, and train for 500 epochs.

[0401] Once the training is complete, the generative artificial intelligence model becomes available through a user interface. The server provides the user interface as a web application or desktop application. Users can log in to the system through this interface and enter questions. For example, a new employee could ask, "What were the key topics from yesterday's meeting?"

[0402] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and generates an appropriate answer. The generated answer is then formatted by the server into a user-friendly format and sent back to the terminal. The terminal displays this answer on its user interface, allowing the user to instantly obtain an answer to their question.

[0403] As a concrete example, consider a scenario where a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" This question is sent from the terminal to the server, and the server, using a generative artificial intelligence model trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view it.

[0404] As a result, the system of the present invention enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and omissions in handover, and improve work efficiency.

[0405] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0406] Step 1:

[0407] Data collection

[0408] The server automatically retrieves data related to past business operations. First, the server connects to the company's mail server, chat server, file server, or cloud storage via API calls. Then, the server downloads emails, chat history, and documents from each department for the past three years. For example, the server retrieves documents from the finance department from AWS S3 and downloads email history using the Gmail API.

[0409] Input: Data from the company's email server, chat server, and file server.

[0410] Output: Email, chat history, and documents for each department over the past 3 years.

[0411] Step 2:

[0412] Data preprocessing

[0413] The server preprocesses the collected data. First, it converts the collected data into text format. It uses Python libraries (e.g., pdfminer or beautifulsoup) to convert documents to text format. Next, it uses regular expressions to remove unwanted information and noise (e.g., email signatures and advertisements). Finally, it uses NER (Named Entity Recognition) to mask personal and sensitive information.

[0414] Input: Email and chat history for each department over the past three years, and documents.

[0415] Output: Data converted to text format, with unnecessary information removed and personal and confidential information masked.

[0416] Step 3:

[0417] AI model training and fine-tuning

[0418] The server trains and fine-tunes generative artificial intelligence models using preprocessed data. First, it prepares the training dataset using Python and sets up the training environment using TensorFlow or PyTorch. The server sets hyperparameters such as the learning rate and batch size and starts training the model. During training, the server monitors the learning progress and dynamically adjusts the hyperparameters as needed.

[0419] Input: Data converted to text format, with personal and confidential information masked.

[0420] Output: Trained and fine-tuned generative artificial intelligence model

[0421] Step 4:

[0422] Providing a user interface

[0423] The server provides a user interface for using generative artificial intelligence models. First, the user interface for the web application is designed and implemented using React.js, allowing users to access the system. If it is provided as a desktop application, the interface is built using Electron or similar. The server also manages user authentication information in a database (e.g., MySQL, PostgreSQL) and integrates it into the user interface.

[0424] Input: Generative artificial intelligence model

[0425] Output: User interface provided as a web application or desktop application.

[0426] Step 5:

[0427] Question submission and answer generation

[0428] When a user enters a question, it is sent to the server via the terminal. The terminal sends the question to the server as an HTTP request, and the server poses the question to a generative artificial intelligence model. The model generates an appropriate answer based on its trained knowledge. For example, in response to the question, "What were the main topics of discussion at yesterday's meeting?", the model might generate the answer, "At yesterday's meeting, a new sales strategy was discussed."

[0429] Input: Questions entered via the user interface

[0430] Output: Answers generated by a generative artificial intelligence model

[0431] Step 6:

[0432] Providing a response

[0433] The server formats the generated response and returns it to the terminal via the user interface. The terminal receives this response and displays it on the user interface. Specifically, the response is displayed on the interface of a web application or desktop application, allowing the user to instantly obtain an answer to their question.

[0434] Input: Answer generated by a generative artificial intelligence model

[0435] Output: Answer displayed on the user interface

[0436] (Application Example 1)

[0437] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0438] New employees and transferred employees are required to quickly and efficiently understand their duties in the factory environment and perform their work smoothly. However, traditional handover processes often result in insufficient information or omissions, preventing new employees from immediately adapting to on-site situations. This raises concerns about decreased productivity and increased errors. Therefore, a system is needed that can effectively facilitate job handover in a factory environment.

[0439] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0440] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, and a user interface related to smart glasses that provide visual work instructions and explanations, particularly in a factory environment. This enables new employees and transferred employees to quickly become proficient in their work within the factory, improving productivity and reducing errors.

[0441] An "information processing device" is a device that performs tasks such as data collection, processing, analysis, and output.

[0442] "Data related to past work" includes all information related to a specific task, such as work processes, work history, meeting records, and message history.

[0443] "Preprocessing" is the process of removing noise and unnecessary information from collected data to make it suitable for useful analysis.

[0444] A "generative artificial intelligence model" is a machine learning model specialized in natural language processing and data generation. It learns from large amounts of data and is capable of answering questions and generating text.

[0445] A "user interface" is a collection of screens, input devices, and interaction methods that a user uses when operating a system.

[0446] "Smart glasses" are glasses-type devices worn by the wearer that have the function of displaying information visually.

[0447] "Visual work instructions" refer to instructions that visually indicate work procedures and precautions using text, images, videos, etc.

[0448] "Training methods" refer to the techniques and processes used to train a generative artificial intelligence model using collected and pre-processed data.

[0449] "Means of generating answers" refers to the process by which a generative artificial intelligence model creates an appropriate answer to a question from a user.

[0450] This invention provides a system that supports new employees and transferred employees in a factory environment to quickly and efficiently understand their tasks and perform their work smoothly. Specific embodiments of this system are described below.

[0451] System Overview

[0452] The server collects data from the factory department to retrieve data related to past operations. Specifically, it collects relevant files and messages from the company's mail server, chat server, file server, or cloud storage as means of data acquisition. For example, the server can download past work data and records via an API.

[0453] Since the acquired data cannot be used as is, the server preprocesses the data. As a preprocessing step, the data is converted to text format, noise and unnecessary information (e.g., email signatures and advertisements) are removed, and personal and confidential information is masked. This step generates a clean dataset.

[0454] Training of generative artificial intelligence models

[0455] Using a clean dataset, the server trains a generative artificial intelligence model (e.g., GPT-4). The model learns key information and patterns related to specific tasks as part of the training process. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0456] The server uses a generative artificial intelligence model to generate answers to natural language questions from users. As a means of generating answers, users input questions through smart glasses, and these questions are sent to the server. The server then poses the question to a trained AI model and generates an answer. For example, a new employee might ask, "What is the next step?", and the AI ​​might answer, "Attach part A to part B."

[0457] User Interface

[0458] The server presents the generated answers to the user via a user interface. In particular, in factory environments, smart glasses are used to provide visual work instructions and explanations. This allows new employees and transferred employees to resolve questions in real time while performing their actual work.

[0459] Hardware and software to be used

[0460] Hardware: Smart glasses (e.g., Vuzix Blade)

[0461] Software: OpenAI GPT-4 API, data preprocessing tools

[0462] As a concrete example, a new employee wearing smart glasses stands on a manufacturing line and asks, "What is the next step?" This question is sent to a server, and a generative artificial intelligence model generates the answer, "Attach part A to part B." This answer is displayed on the smart glasses' screen, allowing the new employee to follow the instructions and proceed with the task.

[0463] Example of a prompt

[0464] "Manufacturing Line Work Instruction Prompt: Generate detailed instructions to help new employees understand the steps on the manufacturing line. For example, write instructions for attaching part A to part B."

[0465] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[0466] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0467] Step 1:

[0468] Data collection

[0469] The server uses means to retrieve data related to past work. Specifically, the server collects data from the company's mail server, chat server, file server, or cloud storage. It downloads past work data and records via API and saves them to local storage. Inputs include emails, chat history, and work manuals, and output is raw data stored in local storage.

[0470] Step 2:

[0471] Data preprocessing

[0472] The server uses means to preprocess the acquired data. Specifically, it removes noise and unnecessary information (e.g., email signatures, advertisements) from the collected data and converts it to text format. It also filters and masks personal and confidential information. Raw data is given as input, and a clean dataset is generated as output. Specific data preprocessing steps include text cleaning using natural language processing tools.

[0473] Step 3:

[0474] Training of artificial intelligence models

[0475] The server performs a means of training a generative artificial intelligence model using preprocessed data. Specifically, it fine-tunes the model (e.g., GPT-4) based on a large amount of text data. The server monitors the model's performance and dynamically adjusts hyperparameters as needed. A clean dataset is used as input, and the output is a trained generative artificial intelligence model.

[0476] Step 4:

[0477] Accepting questions

[0478] The device (smart glasses) accepts questions from the user in natural language. The user inputs the question through the smart glasses' interface, and this is sent to the server. The input is the question from the user, and the output is the question data sent to the server.

[0479] Step 5:

[0480] Answer generation

[0481] The server uses a means of generating answers to user questions based on a generative artificial intelligence model. Specifically, the server feeds question data to the generative AI model and generates an appropriate answer. The question data is used as input, and the generated answer is obtained as output.

[0482] Step 6:

[0483] Providing an answer

[0484] The server performs a means of presenting the answer via a user interface. The generated answer is visually displayed to the user through smart glasses. The input is the generated answer, and the output is the answer displayed on the terminal (smart glasses).

[0485] This processing flow allows new employees and transferred employees to receive timely and relevant information, improving factory work efficiency.

[0486] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0487] This invention is a system that utilizes generative artificial intelligence and an emotion engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency. A detailed embodiment of this system will be described below.

[0488] Data collection

[0489] The server automatically retrieves historical work data for each department. Specifically, it collects data from email servers, chat servers, file servers, or cloud storage. For example, the server can use APIs to download emails, chat histories, and documents that are tagged or stored in specific folders.

[0490] Data preprocessing

[0491] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, filters out noise and unnecessary information (e.g., email signatures, ad blockers), and masks personal and confidential information. This step generates a clean dataset.

[0492] AI model training and fine-tuning

[0493] Using preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0494] Emotional engine integration

[0495] Furthermore, the server combines a generative artificial intelligence model with an emotion engine. The emotion engine can recognize and evaluate emotions from user input and interactions. It detects user emotions using text analysis, speech analysis, or facial recognition. Based on this emotion information, the tone and content of the generated responses can be adjusted.

[0496] Providing a user interface

[0497] After the generative AI model and emotion engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can input questions through it and receive emotion-based responses. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[0498] Question submission and answer generation

[0499] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and an emotion engine, which generates an appropriate answer based on its trained knowledge and the user's emotional state. For example, it might generate an answer that takes the user's emotions into consideration, such as, "At yesterday's meeting, we discussed a new sales strategy."

[0500] Providing a response

[0501] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[0502] Specific example

[0503] For example, suppose a new employee is assigned to the finance department. The new employee, as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and the emotion engine senses anxiety or tension from the user's input. The generative artificial intelligence model takes this emotional information into consideration and generates a response in a gentle tone, such as, "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement. If you have any problems, please feel free to ask." The server returns this response to the terminal, and the user can see the answer.

[0504] In this way, the system of the present invention utilizes past business data to respond quickly and accurately to questions from new employees and employees transferring to other departments, and by integrating an emotion engine, it provides responses that take into account the user's emotions, thereby enabling a smooth handover of duties.

[0505] The following describes the processing flow.

[0506] Step 1:

[0507] The server automatically retrieves historical business data from email servers, chat servers, file servers, or cloud storage. For example, the server uses an API to download email and chat history from the finance department for the past three years.

[0508] Step 2:

[0509] The server converts the acquired data into text format. For example, PDF files and scanned documents are converted into text data using OCR (Optical Character Recognition). Excel files and Word documents are also converted into text format.

[0510] Step 3:

[0511] The server preprocesses the retrieved text data to remove noise and unwanted information (e.g., email signatures and ad blockers). For example, it might use regular expressions to filter out footer text such as "Best regards" or advertisements.

[0512] Step 4:

[0513] The server masks personal and confidential information during the preprocessing stage. For example, names and email addresses are replaced with placeholders such as "[Name]" and "[Email Address]".

[0514] Step 5:

[0515] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using pre-processed data. During this process, the server uses a learning framework (e.g., PyTorch) to ensure the model learns properly.

[0516] Step 6:

[0517] The server integrates an emotion engine into the generative artificial intelligence model. For example, the emotion engine incorporates algorithms that detect and evaluate the user's emotions using text analysis, speech analysis, or facial recognition.

[0518] Step 7:

[0519] The user logs into the user interface and enters a question. This interface is provided as a web application or desktop application and includes a form for the user to enter the question in natural language.

[0520] Step 8:

[0521] The terminal sends the user-entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[0522] Step 9:

[0523] The server inputs the received question into a generative artificial intelligence model and an emotion engine. At this time, the server tokenizes the question and passes it to the model as input data.

[0524] Step 10:

[0525] The emotion engine analyzes emotional information from the user's questions. For example, the emotion engine extracts emotional tags such as "tension," "anxiety," and "joy" from the text of the question.

[0526] Step 11:

[0527] Generative artificial intelligence models generate answers to questions while taking emotional information into account. For example, if the user is nervous, the model will soften the tone of the answer and use gentle expressions such as, "I'll explain it simply, so don't worry."

[0528] Step 12:

[0529] The server restores the generated response to text format and formats it. For example, it might generate a response like, "The procedure for preparing the most recent quarterly financial report is to collect income and expenditure reports from each department, integrate them, and then create the final financial statement. Please feel free to ask questions if you have any problems."

[0530] Step 13:

[0531] The server sends the formatted response to the user interface. Specifically, it returns the response data as an HTTP response.

[0532] Step 14:

[0533] The terminal displays the response received from the server on the user interface. Users can view this response and use it to help with their work. For example, they can gain a concrete understanding of "the procedure for preparing the most recent quarterly financial report."

[0534] In this way, by using a generative artificial intelligence model that integrates an emotion engine, appropriate responses can be obtained that are tailored to the user's emotional state. This not only facilitates smooth handover of tasks but also reduces the psychological burden on the user.

[0535] (Example 2)

[0536] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0537] In companies and organizations, the handover of duties for new employees and employees transferring to other departments can sometimes be unsuccessful. In such cases, important information may be leaked, and knowledge related to the work may not be properly shared, leading to decreased work efficiency and an increased likelihood of errors. Furthermore, new employees and transferring employees often experience anxiety and stress during the handover process, which further reduces work efficiency. Traditional handover methods are insufficient to adequately address these problems.

[0538] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information and mask personal and confidential information, means for training a generative artificial intelligence model using the preprocessed data, means for integrating an emotion recognition engine into the generative artificial intelligence model and evaluating emotions from user input, means for generating answers to natural language questions from the user based on the integrated generative artificial intelligence model and adjusting the tone and content of the answers based on the user's emotions, and means for presenting the generated answers to the user via a user interface. As a result, it is possible to obtain responses that take into account the emotions the user feels during the handover of operations, preventing information handover omissions and incomplete information sharing, and enabling increased operational efficiency and reduced troubles.

[0539] An "information processing device" is a combination of hardware and software for collecting, preprocessing, analyzing, generating, and outputting data.

[0540] "Data related to past business operations" refers to documents and communication data that contain information about business activities carried out in the past.

[0541] "Preprocessing" refers to the process of removing noise and unwanted information, converting formats, and masking personal and confidential information in order to transform collected data into a usable format.

[0542] A "generative artificial intelligence model" refers to a machine learning algorithm that can find patterns in large amounts of data and generate new data.

[0543] An "emotion recognition engine" is an algorithm or software that detects emotions from user input and actions and provides feedback based on those emotions.

[0544] "User" refers to an individual or member of an organization that uses this system.

[0545] "Means for generating answers to questions" refers to a combination of software and hardware that analyzes natural language questions from users and generates appropriate answers.

[0546] A "user interface" is an application that has a display screen and input mechanism for a user to interact with the system.

[0547] "Noise and unnecessary information" refers to irrelevant data and information that does not affect the analysis or generation process.

[0548] "Personal information and confidential information" refers to information that includes the identification of an individual or highly confidential content, and is subject to confidentiality obligations.

[0549] "Adjusting the tone and content of responses" refers to generating responses that are not cruel, taking into account the user's emotional state.

[0550] "Real-time" refers to the time range in which data is processed and outputted with only a slight delay after it is input.

[0551] An "API" refers to an interface used to access the functions of different software programs.

[0552] A "web application" refers to software that can be accessed through an internet browser.

[0553] A "desktop application" refers to software that runs directly on a personal computer.

[0554] This invention is a system that utilizes generative artificial intelligence and an emotion recognition engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency.

[0555] The server first automatically retrieves data related to past work for each department. Data collection utilizes services such as email servers (e.g., Microsoft Exchange), chat servers (Slack), file servers (NAS), and cloud storage (Google Drive, Dropbox). The server uses APIs provided by these services (e.g., Google Drive API and Slack API) to download emails, chat histories, and documents associated with specific folders or tags. This allows the server to grasp a comprehensive overview of the information users need.

[0556] The collected data undergoes preprocessing. The server converts the data to text format using Python's BeautifulSoup or NLP libraries (e.g., NLTK, SpaCy), removes noise and unnecessary information (such as email signatures and ad blockers), and masks personal and sensitive information. This step generates a clean and usable dataset.

[0557] Next, the server trains a generative artificial intelligence model (e.g., GPT-4) using the preprocessed data. Machine learning libraries such as PyTorch and TensorFlow are used for training, the training progress is monitored, and hyperparameters (such as learning rate and batch size) are dynamically adjusted as needed. This allows the model to learn important information and patterns related to specific tasks.

[0558] Furthermore, the emotion recognition engine is integrated into the generative artificial intelligence model. The server uses the emotion recognition engine to evaluate emotions from user input and interactions. For this purpose, TextBlob is used for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. Based on the emotional information, the tone and content of the generated responses will be adjusted. For example, if a user enters a question while feeling anxious, a response in a gentle tone that takes that emotion into account will be provided.

[0559] After the generative AI model and emotion recognition engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, using React or Vue.js for the frontend and Node.js or Django for the backend. Users can input questions through the interface and receive answers in real time.

[0560] For example, if a new employee, who is a user, enters a question such as "Please tell me the procedure for preparing the most recent quarterly financial report," that question will be sent to the server via the terminal.

[0561] The server poses this question to a generative artificial intelligence model and an emotion recognition engine, which analyze the data and generate a response tailored to the user's emotional state. For example, if the emotion recognition engine detects anxiety from the user's input, it will generate a gentle response such as, "The procedure for preparing the most recent quarterly financial report involves first collecting income and expenditure reports from each department, and then integrating them to create the final financial statement. Please feel free to ask if you have any problems."

[0562] The generated answers are sent from the server to the terminal, and the user can display the answers on the screen. In this way, new employees and employees transferring to other departments can get quick and accurate answers to their work-related questions, leading to increased work efficiency.

[0563] This system provides responses that take into account the emotions users feel during the handover of tasks, preventing information from being missed or incomplete, and enabling increased work efficiency and a reduction in problems.

[0564] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0565] Step 1: Data Collection

[0566] The server collects data related to past business operations from cloud storage, mail servers, chat servers, and file servers. Specifically, the server uses APIs (e.g., Google Drive API, Slack API) to download emails, chat histories, and documents tagged with specific folders from each service. It accepts API keys and folder paths from each service as input and saves the downloaded data as a file as output.

[0567] Step 2: Data Preprocessing

[0568] The server preprocesses the collected data. It converts the data to plain text and removes noise and unnecessary information. Specifically, it uses Python libraries (e.g., BeautifulSoup, NLTK, SpaCy) to convert HTML emails to plain text and remove advertisements and personal information. It receives downloaded raw data as input and generates clean text data as output.

[0569] Step 3: Training the AI ​​model

[0570] The server trains a generative artificial intelligence model (e.g., GPT-4) using pre-processed text data. Specifically, it uses PyTorch or TensorFlow to feed a large amount of data into the model and train it. It takes clean text data as input and produces a trained model as output.

[0571] Step 4: Integrating the emotion recognition engine

[0572] The server integrates an emotion recognition engine with a trained generative artificial intelligence model. It analyzes user emotions using TextBlob for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. It receives user interaction data (text, audio, images) as input and outputs analyzed emotion information.

[0573] Step 5: Provide User Interface

[0574] The server provides the user interface. It is designed as a web application (React, Node.js) or a desktop application (Electron). Through this interface, users can input questions and receive answers in real time. It receives user questions as input and displays generated answers as output.

[0575] Step 6: Accepting Questions

[0576] The user enters a question into the user interface, and this is sent to the server via the terminal. Specifically, the entered question is sent to the API endpoint and received as data for processing. It receives the user's question as input and obtains data for processing as output.

[0577] Step 7: Generate Response

[0578] The server uses a generative artificial intelligence model and an emotion recognition engine to generate answers to user questions. For example, if it detects anxiety from the user's input, the model will generate an answer in a gentle tone. It receives user questions and emotion data as input and obtains the generated answer as output.

[0579] Step 8: Provide your answer

[0580] The server formats the generated response and returns it to the terminal. The terminal interprets this response and displays it on the user interface. Specifically, it displays the JSON-formatted response data sent via the API endpoint in HTML format. It receives the generated response as input and provides the response presented to the user as output.

[0581] These specific processing steps create an environment where new employees and transferred employees in each department can smoothly take over their duties, thereby improving work efficiency.

[0582] (Application Example 2)

[0583] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0584] Traditional handover systems had problems that made it difficult for new employees and employees transferring to other departments to quickly and accurately acquire information about their work. Furthermore, inefficiencies and errors were common due to insufficient information and missed handovers, and the systems often lacked consideration for the feelings of users. In addition, in places like logistics centers, there was a need for support to help staff understand complex work processes more easily.

[0585] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means including an emotion engine that analyzes the user's emotions and adjusts the tone of the response based on the results, and means for presenting the generated response to the user via a smartphone application. This enables new employees and transferred employees to answer questions about their work quickly and accurately, and allows for responses that are sensitive to emotions. It also provides support to help logistics center staff quickly understand work procedures and processes.

[0586] An "information processing device" is a device that acquires past business data, performs data preprocessing, trains generative artificial intelligence models, and generates answers to user questions in natural language.

[0587] A "generative artificial intelligence model" is an artificial intelligence model that is trained using collected data, learns information and patterns related to specific tasks, and generates answers to user questions.

[0588] An "emotion engine" is a system that recognizes and evaluates emotions from user input and interactions, and is used to adjust the tone and content of responses generated by generative artificial intelligence models.

[0589] "Preprocessing" refers to a series of processes that convert collected data into text format, remove noise and unnecessary information, and mask personal and confidential information.

[0590] The "user interface" refers to the interface through which users access this system, input questions, and review generated answers.

[0591] A "smartphone application" is an application that runs on a smartphone and allows users to input work-related questions and receive answers.

[0592] A "logistics center" refers to a facility or area where logistics operations such as the storage, management, and shipping of goods are carried out.

[0593] "Tone" refers to the manner in which the generated response is spoken and expressed, and it is adjusted to provide a user-friendly response in a kind and gentle tone.

[0594] The system for implementing this invention is centered around an information processing device and consists of the following steps.

[0595] First, the server automatically retrieves data related to past operations from various sources. These sources include email servers, chat servers, file servers, and cloud storage. Specifically, the server uses APIs to download emails, chat history, documents, and other files that are tagged or stored in specific folders.

[0596] Next, the server preprocesses the acquired data. This preprocessing includes converting the data to text format, filtering out noise and unnecessary information (such as email signatures and ad blockers), and masking personal and sensitive information. This generates a clean dataset.

[0597] Next, the server uses the preprocessed data to train and fine-tune a generative artificial intelligence model (e.g., GPT-4). This allows the model to learn important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0598] Subsequently, the server integrates an emotion engine into the generative artificial intelligence model. The emotion engine has the ability to recognize and evaluate emotions from user input and interactions. This utilizes text analysis, speech analysis, or facial recognition technology. Based on the emotional information, the tone and content of the generated responses are adjusted.

[0599] The user interface is provided in the form of a smartphone application. Through this application, users can input work-related questions in text format and receive immediate answers. Once a question is entered, it is sent via the device to a server, which generates an answer based on a generative artificial intelligence model and an emotion engine. This answer is generated in an appropriate tone, taking into account the user's emotional state.

[0600] For example, if a user asks, "Could you please explain the procedure for taking inventory?", the emotion engine senses tension and anxiety from the user's input. The generative artificial intelligence model takes this emotional information into account and generates a response in a gentle tone, such as, "The procedure for taking inventory is as follows: First, check the quantity of each product, and then enter it into the database. Please let us know if you have any problems." This response is then provided to the user through the smartphone application interface.

[0601] Examples of prompt statements include the following:

[0602] Question to the AI: "Please explain the procedure for taking inventory."

[0603] Emotional status: "Nervous"

[0604] Answer: "The procedure for inventory counting is as follows: First, confirm the quantity of each product, and then enter it into the database. Please let us know if you have any problems."

[0605] In this way, new employees and transferred employees can quickly and accurately resolve work-related questions, and logistics center staff are provided with support to immediately understand work procedures and processes.

[0606] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0607] Step 1:

[0608] The server retrieves data related to past operations. Specifically, it uses APIs to download emails, chat histories, and documents with specific folders or tags from mail servers, chat servers, file servers, or cloud storage. The input for this step is an API request, and the output is the downloaded raw data.

[0609] Step 2:

[0610] The server preprocesses the acquired data. First, it converts the data to text format, then filters out noise and unnecessary information (e.g., email signatures and ad blockers). It also masks personal and confidential information. The input for this step is the downloaded original data, and the output is clean text data.

[0611] Step 3:

[0612] The server uses preprocessed data to train and fine-tune a generative artificial intelligence model (e.g., GPT-4). Clean text data is used as training data, and the model learns important information and patterns related to specific tasks. The input for this step is clean text data, and the output is the trained AI model.

[0613] Step 4:

[0614] The server integrates an emotion engine into the generative artificial intelligence model. The emotion engine recognizes and evaluates emotions from user input and interactions, primarily through text analysis. The input for this step is a trained AI model and a dataset for emotion analysis, while the output is a turnkey system that generates emotion-sensitive responses.

[0615] Step 5:

[0616] The user uses a smartphone application to input questions related to their work. For example, they might input, "Please explain the procedure for taking inventory." The input in this step is text input by the user, and the output is that text question.

[0617] Step 6:

[0618] The terminal sends the user's question to the server. The server receives the question, performs sentiment analysis, and generates an answer using a generative artificial intelligence model. The input for this step is the question text sent by the user, and the output is the generated answer.

[0619] Step 7:

[0620] The server formats the generated response and sends it back to the user via a smartphone application. The input for this step is the generated response, and the output is the final formatted response text.

[0621] Step 8:

[0622] The user reviews the responses generated through a smartphone application. For example, they might receive a response such as, "The inventory count procedure is as follows: First, check the quantity of each product, then enter it into the database. Please let us know if you have any problems." The input in this step is the formatted response returned from the server, and the output is the user's understanding and actions.

[0623] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0624] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0625] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0626] [Third Embodiment]

[0627] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0628] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0629] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0630] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0631] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0632] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0633] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0634] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0635] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0636] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0637] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0638] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0639] This invention is a system that utilizes generative artificial intelligence to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, thereby improving work efficiency. A detailed embodiment of this system will be described below.

[0640] Data collection

[0641] The server automatically retrieves historical work data for each department. Specifically, it collects relevant files and messages from the company's email server, chat server, file server, or cloud storage. For example, the server can download emails, chat history, and accounting-related documents from the finance department for the past three years via an API.

[0642] Data preprocessing

[0643] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, removes noise and unnecessary information (e.g., email signatures and advertisements), and masks personal and confidential information. This step generates a clean dataset.

[0644] AI model training and fine-tuning

[0645] Using pre-processed data, the server fine-tunes a generative artificial intelligence model (e.g., GPT-4). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0646] Providing a user interface

[0647] After the generative artificial intelligence model is trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can enter questions through it. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[0648] Question submission and answer generation

[0649] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model, which generates an answer based on its trained knowledge. For example, the model might generate an answer such as, "At yesterday's meeting, we discussed a new sales strategy."

[0650] Providing a response

[0651] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[0652] Specific example

[0653] For example, suppose a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and a generative artificial intelligence model, trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view the answer.

[0654] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[0655] The following describes the processing flow.

[0656] Step 1:

[0657] The server retrieves past work data. Specifically, it collects data from mail servers, chat servers, file servers, or cloud storage. For example, the server uses APIs to download emails, chat history, and documents that are tagged or stored in specific folders.

[0658] Step 2:

[0659] The server converts the acquired data into text format. PDFs and scanned documents are converted into text data using OCR (Optical Character Recognition) technology.

[0660] Step 3:

[0661] The server preprocesses the converted text data. Specifically, it filters out noise and unnecessary information (e.g., email signatures, ad blockers) and cleans the data.

[0662] Step 4:

[0663] The server masks personal and confidential information during the data preprocessing process. Personal information such as names and email addresses are replaced with placeholders.

[0664] Step 5:

[0665] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using a pre-processed, clean dataset. It monitors the learning progress and adjusts hyperparameters as needed.

[0666] Step 6:

[0667] The server provides a user interface for users to enter questions. This interface is designed as either a web application or a desktop application.

[0668] Step 7:

[0669] The user logs into the user interface and enters their question in natural language. For example, they might type, "Please tell me the steps for preparing the latest quarterly financial report."

[0670] Step 8:

[0671] The terminal sends the entered question to the server. The text entered in the user interface is sent to the server as an HTTP POST request.

[0672] Step 9:

[0673] The server tokenizes the received question and queries a generative artificial intelligence model. The model generates an appropriate answer based on the training data.

[0674] Step 10:

[0675] The server restores the generated response to text format and formats it. For example, it might generate a response such as, "The procedure for preparing the latest quarterly financial report is to collect income and expenditure reports from each department, integrate them, and create the final financial statement."

[0676] Step 11:

[0677] The server sends the formatted response to the user interface.

[0678] Step 12:

[0679] The terminal displays the response received from the server on the user interface. The user can view this response and use it for their work.

[0680] (Example 1)

[0681] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0682] Traditional job handover processes frequently resulted in insufficient information and omissions, placing a significant burden on new and transferred employees in learning their new roles. Furthermore, manual information gathering and analysis were time-consuming and labor-intensive, leading to decreased work efficiency. Therefore, a system was needed to efficiently collect and analyze past work data and provide new and transferred employees with timely and accurate information.

[0683] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0684] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, means for converting the data into text format, means for masking personal and confidential information, means for dynamically adjusting the hyperparameters of the generative artificial intelligence model, means for sending the user's questions to the server as HTTP requests, and means for formatting the generated answers into a format that is easy for the user to understand. This enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and handover omissions, and improve work efficiency.

[0685] An "information processing device" is a system that includes hardware and software for collecting, processing, analyzing, and providing data results.

[0686] "Data related to past work" refers to records of work performed by a specific department within a company in the past, and includes emails, chat history, documents, etc.

[0687] "Preprocessing" is the process of converting collected data into an analyzable format by removing noise and unnecessary information, and masking personal and confidential information.

[0688] A "generative artificial intelligence model" is an AI model trained to perform natural language processing tasks using large amounts of text data, and models like GPT-4 fall into this category.

[0689] "Natural language questions from users" are inquiries or questions written by users in human language, specifically those entered as sentences or phrases.

[0690] "Means for generating answers" refers to a system that uses a generative artificial intelligence model to perform processing to generate appropriate answers to user questions.

[0691] A "user interface" is the part that provides the visual and manipulative means for a user to interact with a system, and is implemented as a web application or desktop application.

[0692] "Means of converting to text format" refers to the process of converting collected data into text data, and may involve using technologies such as OCR (Optical Character Recognition).

[0693] "Methods for masking personal and confidential information" refer to systems that process collected data to conceal information that could identify a specific individual or confidential company information.

[0694] "Methods for dynamically adjusting hyperparameters" refers to the process of dynamically changing parameters such as learning rate and batch size in order to optimize the performance of a generative artificial intelligence model.

[0695] "Means of sending to a server as an HTTP request" refers to the means by which a user enters a question from their terminal and sends it to a server via the Internet Protocol.

[0696] "Means of formatting into a user-friendly format" refers to a system that processes generated responses into an appropriate format so that users can easily understand them.

[0697] This invention is an information processing system that utilizes generative artificial intelligence to prevent information deficiencies and omissions during the handover of duties between new employees and transferred employees, and to improve work efficiency. The embodiments for carrying out this invention will be described in detail below.

[0698] First, the server automatically collects data related to past business operations. This data is retrieved from the company's email server, chat server, file server, or cloud storage. For example, the server uses an API to download emails, chat history, and accounting-related documents from the finance department for the past three years.

[0699] The collected data is preprocessed by the server. Preprocessing includes converting the data to text format, removing noise and unwanted information, and masking personal and sensitive information. For example, the server uses Python libraries (such as pdfminer and beautifulsoup) to convert documents to text format and removes unwanted information using regular expressions. It also uses NER (Named Entity Recognition) to identify and appropriately mask sensitive information.

[0700] Next, using the preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). The server sets up the training environment (e.g., using TensorFlow or PyTorch), sets the hyperparameters appropriately, and trains the model. During training, the server monitors the progress and dynamically adjusts the hyperparameters as needed. For example, it might set the learning rate to 0.001 and the batch size to 32, and train for 500 epochs.

[0701] Once the training is complete, the generative artificial intelligence model becomes available through a user interface. The server provides the user interface as a web application or desktop application. Users can log in to the system through this interface and enter questions. For example, a new employee could ask, "What were the key topics from yesterday's meeting?"

[0702] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and generates an appropriate answer. The generated answer is then formatted by the server into a user-friendly format and sent back to the terminal. The terminal displays this answer on its user interface, allowing the user to instantly obtain an answer to their question.

[0703] As a concrete example, consider a scenario where a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" This question is sent from the terminal to the server, and the server, using a generative artificial intelligence model trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view it.

[0704] As a result, the system of the present invention enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and omissions in handover, and improve work efficiency.

[0705] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0706] Step 1:

[0707] Data collection

[0708] The server automatically retrieves data related to past business operations. First, the server connects to the company's mail server, chat server, file server, or cloud storage via API calls. Then, the server downloads emails, chat history, and documents from each department for the past three years. For example, the server retrieves documents from the finance department from AWS S3 and downloads email history using the Gmail API.

[0709] Input: Data from the company's email server, chat server, and file server.

[0710] Output: Email, chat history, and documents for each department over the past 3 years.

[0711] Step 2:

[0712] Data preprocessing

[0713] The server preprocesses the collected data. First, it converts the collected data into text format. It uses Python libraries (e.g., pdfminer or beautifulsoup) to convert documents to text format. Next, it uses regular expressions to remove unwanted information and noise (e.g., email signatures and advertisements). Finally, it uses NER (Named Entity Recognition) to mask personal and sensitive information.

[0714] Input: Email and chat history for each department over the past three years, and documents.

[0715] Output: Data converted to text format, with unnecessary information removed and personal and confidential information masked.

[0716] Step 3:

[0717] AI model training and fine-tuning

[0718] The server trains and fine-tunes generative artificial intelligence models using preprocessed data. First, it prepares the training dataset using Python and sets up the training environment using TensorFlow or PyTorch. The server sets hyperparameters such as the learning rate and batch size and starts training the model. During training, the server monitors the learning progress and dynamically adjusts the hyperparameters as needed.

[0719] Input: Data converted to text format, with personal and confidential information masked.

[0720] Output: Trained and fine-tuned generative artificial intelligence model

[0721] Step 4:

[0722] Providing a user interface

[0723] The server provides a user interface for using generative artificial intelligence models. First, the user interface for the web application is designed and implemented using React.js, allowing users to access the system. If it is provided as a desktop application, the interface is built using Electron or similar. The server also manages user authentication information in a database (e.g., MySQL, PostgreSQL) and integrates it into the user interface.

[0724] Input: Generative artificial intelligence model

[0725] Output: User interface provided as a web application or desktop application.

[0726] Step 5:

[0727] Question submission and answer generation

[0728] When a user enters a question, it is sent to the server via the terminal. The terminal sends the question to the server as an HTTP request, and the server poses the question to a generative artificial intelligence model. The model generates an appropriate answer based on its trained knowledge. For example, in response to the question, "What were the main topics of discussion at yesterday's meeting?", the model might generate the answer, "At yesterday's meeting, a new sales strategy was discussed."

[0729] Input: Questions entered via the user interface

[0730] Output: Answers generated by a generative artificial intelligence model

[0731] Step 6:

[0732] Providing a response

[0733] The server formats the generated response and returns it to the terminal via the user interface. The terminal receives this response and displays it on the user interface. Specifically, the response is displayed on the interface of a web application or desktop application, allowing the user to instantly obtain an answer to their question.

[0734] Input: Answer generated by a generative artificial intelligence model

[0735] Output: Answer displayed on the user interface

[0736] (Application Example 1)

[0737] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0738] New employees and transferred employees are required to quickly and efficiently understand their duties in the factory environment and perform their work smoothly. However, traditional handover processes often result in insufficient information or omissions, preventing new employees from immediately adapting to on-site situations. This raises concerns about decreased productivity and increased errors. Therefore, a system is needed that can effectively facilitate job handover in a factory environment.

[0739] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0740] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, and a user interface related to smart glasses that provide visual work instructions and explanations, particularly in a factory environment. This enables new employees and transferred employees to quickly become proficient in their work within the factory, improving productivity and reducing errors.

[0741] An "information processing device" is a device that performs tasks such as data collection, processing, analysis, and output.

[0742] "Data related to past work" includes all information related to a specific task, such as work processes, work history, meeting records, and message history.

[0743] "Preprocessing" is the process of removing noise and unnecessary information from collected data to make it suitable for useful analysis.

[0744] A "generative artificial intelligence model" is a machine learning model specialized in natural language processing and data generation. It learns from large amounts of data and is capable of answering questions and generating text.

[0745] A "user interface" is a collection of screens, input devices, and interaction methods that a user uses when operating a system.

[0746] "Smart glasses" are glasses-type devices worn by the wearer that have the function of displaying information visually.

[0747] "Visual work instructions" refer to instructions that visually indicate work procedures and precautions using text, images, videos, etc.

[0748] "Training methods" refer to the techniques and processes used to train a generative artificial intelligence model using collected and pre-processed data.

[0749] "Means of generating answers" refers to the process by which a generative artificial intelligence model creates an appropriate answer to a question from a user.

[0750] This invention provides a system that supports new employees and transferred employees in a factory environment to quickly and efficiently understand their tasks and perform their work smoothly. Specific embodiments of this system are described below.

[0751] System Overview

[0752] The server collects data from the factory department to retrieve data related to past operations. Specifically, it collects relevant files and messages from the company's mail server, chat server, file server, or cloud storage as means of data acquisition. For example, the server can download past work data and records via an API.

[0753] Since the acquired data cannot be used as is, the server preprocesses the data. As a preprocessing step, the data is converted to text format, noise and unnecessary information (e.g., email signatures and advertisements) are removed, and personal and confidential information is masked. This step generates a clean dataset.

[0754] Training of generative artificial intelligence models

[0755] Using a clean dataset, the server trains a generative artificial intelligence model (e.g., GPT-4). The model learns key information and patterns related to specific tasks as part of the training process. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0756] The server uses a generative artificial intelligence model to generate answers to natural language questions from users. As a means of generating answers, users input questions through smart glasses, and these questions are sent to the server. The server then poses the question to a trained AI model and generates an answer. For example, a new employee might ask, "What is the next step?", and the AI ​​might answer, "Attach part A to part B."

[0757] User Interface

[0758] The server presents the generated answers to the user via a user interface. In particular, in factory environments, smart glasses are used to provide visual work instructions and explanations. This allows new employees and transferred employees to resolve questions in real time while performing their actual work.

[0759] Hardware and software to be used

[0760] Hardware: Smart glasses (e.g., Vuzix Blade)

[0761] Software: OpenAI GPT-4 API, data preprocessing tools

[0762] As a concrete example, a new employee wearing smart glasses stands on a manufacturing line and asks, "What is the next step?" This question is sent to a server, and a generative artificial intelligence model generates the answer, "Attach part A to part B." This answer is displayed on the smart glasses' screen, allowing the new employee to follow the instructions and proceed with the task.

[0763] Example of a prompt

[0764] "Manufacturing Line Work Instruction Prompt: Generate detailed instructions to help new employees understand the steps on the manufacturing line. For example, write instructions for attaching part A to part B."

[0765] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[0766] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0767] Step 1:

[0768] Data collection

[0769] The server uses means to retrieve data related to past work. Specifically, the server collects data from the company's mail server, chat server, file server, or cloud storage. It downloads past work data and records via API and saves them to local storage. Inputs include emails, chat history, and work manuals, and output is raw data stored in local storage.

[0770] Step 2:

[0771] Data preprocessing

[0772] The server uses means to preprocess the acquired data. Specifically, it removes noise and unnecessary information (e.g., email signatures, advertisements) from the collected data and converts it to text format. It also filters and masks personal and confidential information. Raw data is given as input, and a clean dataset is generated as output. Specific data preprocessing steps include text cleaning using natural language processing tools.

[0773] Step 3:

[0774] Training of artificial intelligence models

[0775] The server performs a means of training a generative artificial intelligence model using preprocessed data. Specifically, it fine-tunes the model (e.g., GPT-4) based on a large amount of text data. The server monitors the model's performance and dynamically adjusts hyperparameters as needed. A clean dataset is used as input, and the output is a trained generative artificial intelligence model.

[0776] Step 4:

[0777] Accepting questions

[0778] The device (smart glasses) accepts questions from the user in natural language. The user inputs the question through the smart glasses' interface, and this is sent to the server. The input is the question from the user, and the output is the question data sent to the server.

[0779] Step 5:

[0780] Answer generation

[0781] The server uses a means of generating answers to user questions based on a generative artificial intelligence model. Specifically, the server feeds question data to the generative AI model and generates an appropriate answer. The question data is used as input, and the generated answer is obtained as output.

[0782] Step 6:

[0783] Providing an answer

[0784] The server performs a means of presenting the answer via a user interface. The generated answer is visually displayed to the user through smart glasses. The input is the generated answer, and the output is the answer displayed on the terminal (smart glasses).

[0785] This processing flow allows new employees and transferred employees to receive timely and relevant information, improving factory work efficiency.

[0786] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0787] This invention is a system that utilizes generative artificial intelligence and an emotion engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency. A detailed embodiment of this system will be described below.

[0788] Data collection

[0789] The server automatically retrieves historical work data for each department. Specifically, it collects data from email servers, chat servers, file servers, or cloud storage. For example, the server can use APIs to download emails, chat histories, and documents that are tagged or stored in specific folders.

[0790] Data preprocessing

[0791] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, filters out noise and unnecessary information (e.g., email signatures, ad blockers), and masks personal and confidential information. This step generates a clean dataset.

[0792] AI model training and fine-tuning

[0793] Using preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0794] Emotional engine integration

[0795] Furthermore, the server combines a generative artificial intelligence model with an emotion engine. The emotion engine can recognize and evaluate emotions from user input and interactions. It detects user emotions using text analysis, speech analysis, or facial recognition. Based on this emotion information, the tone and content of the generated responses can be adjusted.

[0796] Providing a user interface

[0797] After the generative AI model and emotion engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can input questions through it and receive emotion-based responses. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[0798] Question submission and answer generation

[0799] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and an emotion engine, which generates an appropriate answer based on its trained knowledge and the user's emotional state. For example, it might generate an answer that takes the user's emotions into consideration, such as, "At yesterday's meeting, we discussed a new sales strategy."

[0800] Providing a response

[0801] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[0802] Specific example

[0803] For example, suppose a new employee is assigned to the finance department. The new employee, as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and the emotion engine senses anxiety or tension from the user's input. The generative artificial intelligence model takes this emotional information into consideration and generates a response in a gentle tone, such as, "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement. If you have any problems, please feel free to ask." The server returns this response to the terminal, and the user can see the answer.

[0804] In this way, the system of the present invention utilizes past business data to respond quickly and accurately to questions from new employees and employees transferring to other departments, and by integrating an emotion engine, it provides responses that take into account the user's emotions, thereby enabling a smooth handover of duties.

[0805] The following describes the processing flow.

[0806] Step 1:

[0807] The server automatically retrieves historical business data from email servers, chat servers, file servers, or cloud storage. For example, the server uses an API to download email and chat history from the finance department for the past three years.

[0808] Step 2:

[0809] The server converts the acquired data into text format. For example, PDF files and scanned documents are converted into text data using OCR (Optical Character Recognition). Excel files and Word documents are also converted into text format.

[0810] Step 3:

[0811] The server preprocesses the retrieved text data to remove noise and unwanted information (e.g., email signatures and ad blockers). For example, it might use regular expressions to filter out footer text such as "Best regards" or advertisements.

[0812] Step 4:

[0813] The server masks personal and confidential information during the preprocessing stage. For example, names and email addresses are replaced with placeholders such as "[Name]" and "[Email Address]".

[0814] Step 5:

[0815] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using pre-processed data. During this process, the server uses a learning framework (e.g., PyTorch) to ensure the model learns properly.

[0816] Step 6:

[0817] The server integrates an emotion engine into the generative artificial intelligence model. For example, the emotion engine incorporates algorithms that detect and evaluate the user's emotions using text analysis, speech analysis, or facial recognition.

[0818] Step 7:

[0819] The user logs into the user interface and enters a question. This interface is provided as a web application or desktop application and includes a form for the user to enter the question in natural language.

[0820] Step 8:

[0821] The terminal sends the user-entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[0822] Step 9:

[0823] The server inputs the received question into a generative artificial intelligence model and an emotion engine. At this time, the server tokenizes the question and passes it to the model as input data.

[0824] Step 10:

[0825] The emotion engine analyzes emotional information from the user's questions. For example, the emotion engine extracts emotional tags such as "tension," "anxiety," and "joy" from the text of the question.

[0826] Step 11:

[0827] Generative artificial intelligence models generate answers to questions while taking emotional information into account. For example, if the user is nervous, the model will soften the tone of the answer and use gentle expressions such as, "I'll explain it simply, so don't worry."

[0828] Step 12:

[0829] The server restores the generated response to text format and formats it. For example, it might generate a response like, "The procedure for preparing the most recent quarterly financial report is to collect income and expenditure reports from each department, integrate them, and then create the final financial statement. Please feel free to ask questions if you have any problems."

[0830] Step 13:

[0831] The server sends the formatted response to the user interface. Specifically, it returns the response data as an HTTP response.

[0832] Step 14:

[0833] The terminal displays the response received from the server on the user interface. Users can view this response and use it to help with their work. For example, they can gain a concrete understanding of "the procedure for preparing the most recent quarterly financial report."

[0834] In this way, by using a generative artificial intelligence model that integrates an emotion engine, appropriate responses can be obtained that are tailored to the user's emotional state. This not only facilitates smooth handover of tasks but also reduces the psychological burden on the user.

[0835] (Example 2)

[0836] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0837] In companies and organizations, the handover of duties for new employees and employees transferring to other departments can sometimes be unsuccessful. In such cases, important information may be leaked, and knowledge related to the work may not be properly shared, leading to decreased work efficiency and an increased likelihood of errors. Furthermore, new employees and transferring employees often experience anxiety and stress during the handover process, which further reduces work efficiency. Traditional handover methods are insufficient to adequately address these problems.

[0838] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information and mask personal and confidential information, means for training a generative artificial intelligence model using the preprocessed data, means for integrating an emotion recognition engine into the generative artificial intelligence model and evaluating emotions from user input, means for generating answers to natural language questions from the user based on the integrated generative artificial intelligence model and adjusting the tone and content of the answers based on the user's emotions, and means for presenting the generated answers to the user via a user interface. As a result, it is possible to obtain responses that take into account the emotions the user feels during the handover of operations, preventing information handover omissions and incomplete information sharing, and enabling increased operational efficiency and reduced troubles.

[0839] An "information processing device" is a combination of hardware and software for collecting, preprocessing, analyzing, generating, and outputting data.

[0840] "Data related to past business operations" refers to documents and communication data that contain information about business activities carried out in the past.

[0841] "Preprocessing" refers to the process of removing noise and unwanted information, converting formats, and masking personal and confidential information in order to transform collected data into a usable format.

[0842] A "generative artificial intelligence model" refers to a machine learning algorithm that can find patterns in large amounts of data and generate new data.

[0843] An "emotion recognition engine" is an algorithm or software that detects emotions from user input and actions and provides feedback based on those emotions.

[0844] "User" refers to an individual or member of an organization that uses this system.

[0845] "Means for generating answers to questions" refers to a combination of software and hardware that analyzes natural language questions from users and generates appropriate answers.

[0846] A "user interface" is an application that has a display screen and input mechanism for a user to interact with the system.

[0847] "Noise and unnecessary information" refers to irrelevant data and information that does not affect the analysis or generation process.

[0848] "Personal information and confidential information" refers to information that includes the identification of an individual or highly confidential content, and is subject to confidentiality obligations.

[0849] "Adjusting the tone and content of responses" refers to generating responses that are not cruel, taking into account the user's emotional state.

[0850] "Real-time" refers to the time range in which data is processed and outputted with only a slight delay after it is input.

[0851] An "API" refers to an interface used to access the functions of different software programs.

[0852] A "web application" refers to software that can be accessed through an internet browser.

[0853] A "desktop application" refers to software that runs directly on a personal computer.

[0854] This invention is a system that utilizes generative artificial intelligence and an emotion recognition engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency.

[0855] The server first automatically retrieves data related to past work for each department. Data collection utilizes services such as email servers (e.g., Microsoft Exchange), chat servers (Slack), file servers (NAS), and cloud storage (Google Drive, Dropbox). The server uses APIs provided by these services (e.g., Google Drive API and Slack API) to download emails, chat histories, and documents associated with specific folders or tags. This allows the server to grasp a comprehensive overview of the information users need.

[0856] The collected data undergoes preprocessing. The server converts the data to text format using Python's BeautifulSoup or NLP libraries (e.g., NLTK, SpaCy), removes noise and unnecessary information (such as email signatures and ad blockers), and masks personal and sensitive information. This step generates a clean and usable dataset.

[0857] Next, the server trains a generative artificial intelligence model (e.g., GPT-4) using the preprocessed data. Machine learning libraries such as PyTorch and TensorFlow are used for training, the training progress is monitored, and hyperparameters (such as learning rate and batch size) are dynamically adjusted as needed. This allows the model to learn important information and patterns related to specific tasks.

[0858] Furthermore, the emotion recognition engine is integrated into the generative artificial intelligence model. The server uses the emotion recognition engine to evaluate emotions from user input and interactions. For this purpose, TextBlob is used for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. Based on the emotional information, the tone and content of the generated responses will be adjusted. For example, if a user enters a question while feeling anxious, a response in a gentle tone that takes that emotion into account will be provided.

[0859] After the generative AI model and emotion recognition engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, using React or Vue.js for the frontend and Node.js or Django for the backend. Users can input questions through the interface and receive answers in real time.

[0860] For example, if a new employee, who is a user, enters a question such as "Please tell me the procedure for preparing the most recent quarterly financial report," that question will be sent to the server via the terminal.

[0861] The server poses this question to a generative artificial intelligence model and an emotion recognition engine, which analyze the data and generate a response tailored to the user's emotional state. For example, if the emotion recognition engine detects anxiety from the user's input, it will generate a gentle response such as, "The procedure for preparing the most recent quarterly financial report involves first collecting income and expenditure reports from each department, and then integrating them to create the final financial statement. Please feel free to ask if you have any problems."

[0862] The generated answers are sent from the server to the terminal, and the user can display the answers on the screen. In this way, new employees and employees transferring to other departments can get quick and accurate answers to their work-related questions, leading to increased work efficiency.

[0863] This system provides responses that take into account the emotions users feel during the handover of tasks, preventing information from being missed or incomplete, and enabling increased work efficiency and a reduction in problems.

[0864] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0865] Step 1: Data Collection

[0866] The server collects data related to past business operations from cloud storage, mail servers, chat servers, and file servers. Specifically, the server uses APIs (e.g., Google Drive API, Slack API) to download emails, chat histories, and documents tagged with specific folders from each service. It accepts API keys and folder paths from each service as input and saves the downloaded data as a file as output.

[0867] Step 2: Data Preprocessing

[0868] The server preprocesses the collected data. It converts the data to plain text and removes noise and unnecessary information. Specifically, it uses Python libraries (e.g., BeautifulSoup, NLTK, SpaCy) to convert HTML emails to plain text and remove advertisements and personal information. It receives downloaded raw data as input and generates clean text data as output.

[0869] Step 3: Training the AI ​​model

[0870] The server trains a generative artificial intelligence model (e.g., GPT-4) using pre-processed text data. Specifically, it uses PyTorch or TensorFlow to feed a large amount of data into the model and train it. It takes clean text data as input and produces a trained model as output.

[0871] Step 4: Integrating the emotion recognition engine

[0872] The server integrates an emotion recognition engine with a trained generative artificial intelligence model. It analyzes user emotions using TextBlob for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. It receives user interaction data (text, audio, images) as input and outputs analyzed emotion information.

[0873] Step 5: Provide User Interface

[0874] The server provides the user interface. It is designed as a web application (React, Node.js) or a desktop application (Electron). Through this interface, users can input questions and receive answers in real time. It receives user questions as input and displays generated answers as output.

[0875] Step 6: Accepting Questions

[0876] The user enters a question into the user interface, and this is sent to the server via the terminal. Specifically, the entered question is sent to the API endpoint and received as data for processing. It receives the user's question as input and obtains data for processing as output.

[0877] Step 7: Generate Response

[0878] The server uses a generative artificial intelligence model and an emotion recognition engine to generate answers to user questions. For example, if it detects anxiety from the user's input, the model will generate an answer in a gentle tone. It receives user questions and emotion data as input and obtains the generated answer as output.

[0879] Step 8: Provide your answer

[0880] The server formats the generated response and returns it to the terminal. The terminal interprets this response and displays it on the user interface. Specifically, it displays the JSON-formatted response data sent via the API endpoint in HTML format. It receives the generated response as input and provides the response presented to the user as output.

[0881] These specific processing steps create an environment where new employees and transferred employees in each department can smoothly take over their duties, thereby improving work efficiency.

[0882] (Application Example 2)

[0883] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0884] Traditional handover systems had problems that made it difficult for new employees and employees transferring to other departments to quickly and accurately acquire information about their work. Furthermore, inefficiencies and errors were common due to insufficient information and missed handovers, and the systems often lacked consideration for the feelings of users. In addition, in places like logistics centers, there was a need for support to help staff understand complex work processes more easily.

[0885] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means including an emotion engine that analyzes the user's emotions and adjusts the tone of the response based on the results, and means for presenting the generated response to the user via a smartphone application. This enables new employees and transferred employees to answer questions about their work quickly and accurately, and allows for responses that are sensitive to emotions. It also provides support to help logistics center staff quickly understand work procedures and processes.

[0886] An "information processing device" is a device that acquires past business data, performs data preprocessing, trains generative artificial intelligence models, and generates answers to user questions in natural language.

[0887] A "generative artificial intelligence model" is an artificial intelligence model that is trained using collected data, learns information and patterns related to specific tasks, and generates answers to user questions.

[0888] An "emotion engine" is a system that recognizes and evaluates emotions from user input and interactions, and is used to adjust the tone and content of responses generated by generative artificial intelligence models.

[0889] "Preprocessing" refers to a series of processes that convert collected data into text format, remove noise and unnecessary information, and mask personal and confidential information.

[0890] The "user interface" refers to the interface through which users access this system, input questions, and review generated answers.

[0891] A "smartphone application" is an application that runs on a smartphone and allows users to input work-related questions and receive answers.

[0892] A "logistics center" refers to a facility or area where logistics operations such as the storage, management, and shipping of goods are carried out.

[0893] "Tone" refers to the manner in which the generated response is spoken and expressed, and it is adjusted to provide a user-friendly response in a kind and gentle tone.

[0894] The system for implementing this invention is centered around an information processing device and consists of the following steps.

[0895] First, the server automatically retrieves data related to past operations from various sources. These sources include email servers, chat servers, file servers, and cloud storage. Specifically, the server uses APIs to download emails, chat history, documents, and other files that are tagged or stored in specific folders.

[0896] Next, the server preprocesses the acquired data. This preprocessing includes converting the data to text format, filtering out noise and unnecessary information (such as email signatures and ad blockers), and masking personal and sensitive information. This generates a clean dataset.

[0897] Next, the server uses the preprocessed data to train and fine-tune a generative artificial intelligence model (e.g., GPT-4). This allows the model to learn important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0898] Subsequently, the server integrates an emotion engine into the generative artificial intelligence model. The emotion engine has the ability to recognize and evaluate emotions from user input and interactions. This utilizes text analysis, speech analysis, or facial recognition technology. Based on the emotional information, the tone and content of the generated responses are adjusted.

[0899] The user interface is provided in the form of a smartphone application. Through this application, users can input work-related questions in text format and receive immediate answers. Once a question is entered, it is sent via the device to a server, which generates an answer based on a generative artificial intelligence model and an emotion engine. This answer is generated in an appropriate tone, taking into account the user's emotional state.

[0900] For example, if a user asks, "Could you please explain the procedure for taking inventory?", the emotion engine senses tension and anxiety from the user's input. The generative artificial intelligence model takes this emotional information into account and generates a response in a gentle tone, such as, "The procedure for taking inventory is as follows: First, check the quantity of each product, and then enter it into the database. Please let us know if you have any problems." This response is then provided to the user through the smartphone application interface.

[0901] Examples of prompt statements include the following:

[0902] Question to the AI: "Please explain the procedure for taking inventory."

[0903] Emotional status: "Nervous"

[0904] Answer: "The procedure for inventory counting is as follows: First, confirm the quantity of each product, and then enter it into the database. Please let us know if you have any problems."

[0905] In this way, new employees and transferred employees can quickly and accurately resolve work-related questions, and logistics center staff are provided with support to immediately understand work procedures and processes.

[0906] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0907] Step 1:

[0908] The server retrieves data related to past operations. Specifically, it uses APIs to download emails, chat histories, and documents with specific folders or tags from mail servers, chat servers, file servers, or cloud storage. The input for this step is an API request, and the output is the downloaded raw data.

[0909] Step 2:

[0910] The server preprocesses the acquired data. First, it converts the data to text format, then filters out noise and unnecessary information (e.g., email signatures and ad blockers). It also masks personal and confidential information. The input for this step is the downloaded original data, and the output is clean text data.

[0911] Step 3:

[0912] The server uses preprocessed data to train and fine-tune a generative artificial intelligence model (e.g., GPT-4). Clean text data is used as training data, and the model learns important information and patterns related to specific tasks. The input for this step is clean text data, and the output is the trained AI model.

[0913] Step 4:

[0914] The server integrates an emotion engine into the generative artificial intelligence model. The emotion engine recognizes and evaluates emotions from user input and interactions, primarily through text analysis. The input for this step is a trained AI model and a dataset for emotion analysis, while the output is a turnkey system that generates emotion-sensitive responses.

[0915] Step 5:

[0916] The user uses a smartphone application to input questions related to their work. For example, they might input, "Please explain the procedure for taking inventory." The input in this step is text input by the user, and the output is that text question.

[0917] Step 6:

[0918] The terminal sends the user's question to the server. The server receives the question, performs sentiment analysis, and generates an answer using a generative artificial intelligence model. The input for this step is the question text sent by the user, and the output is the generated answer.

[0919] Step 7:

[0920] The server formats the generated response and sends it back to the user via a smartphone application. The input for this step is the generated response, and the output is the final formatted response text.

[0921] Step 8:

[0922] The user reviews the responses generated through a smartphone application. For example, they might receive a response such as, "The inventory count procedure is as follows: First, check the quantity of each product, then enter it into the database. Please let us know if you have any problems." The input in this step is the formatted response returned from the server, and the output is the user's understanding and actions.

[0923] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0924] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0925] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[0926] [Fourth Embodiment]

[0927] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0928] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0929] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0930] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0931] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0932] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0933] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0934] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0935] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0936] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0937] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0938] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0939] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0940] This invention is a system that utilizes generative artificial intelligence to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, thereby improving work efficiency. A detailed embodiment of this system will be described below.

[0941] Data collection

[0942] The server automatically retrieves historical work data for each department. Specifically, it collects relevant files and messages from the company's email server, chat server, file server, or cloud storage. For example, the server can download emails, chat history, and accounting-related documents from the finance department for the past three years via an API.

[0943] Data preprocessing

[0944] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, removes noise and unnecessary information (e.g., email signatures and advertisements), and masks personal and confidential information. This step generates a clean dataset.

[0945] AI model training and fine-tuning

[0946] Using pre-processed data, the server fine-tunes a generative artificial intelligence model (e.g., GPT-4). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[0947] Providing a user interface

[0948] After the generative artificial intelligence model is trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can enter questions through it. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[0949] Question submission and answer generation

[0950] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model, which generates an answer based on its trained knowledge. For example, the model might generate an answer such as, "At yesterday's meeting, we discussed a new sales strategy."

[0951] Providing a response

[0952] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[0953] Specific example

[0954] For example, suppose a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and a generative artificial intelligence model, trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view the answer.

[0955] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[0956] The following describes the processing flow.

[0957] Step 1:

[0958] The server retrieves past work data. Specifically, it collects data from mail servers, chat servers, file servers, or cloud storage. For example, the server uses APIs to download emails, chat history, and documents that are tagged or stored in specific folders.

[0959] Step 2:

[0960] The server converts the acquired data into text format. PDFs and scanned documents are converted into text data using OCR (Optical Character Recognition) technology.

[0961] Step 3:

[0962] The server preprocesses the converted text data. Specifically, it filters out noise and unnecessary information (e.g., email signatures, ad blockers) and cleans the data.

[0963] Step 4:

[0964] The server masks personal and confidential information during the data preprocessing process. Personal information such as names and email addresses are replaced with placeholders.

[0965] Step 5:

[0966] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using a pre-processed, clean dataset. It monitors the learning progress and adjusts hyperparameters as needed.

[0967] Step 6:

[0968] The server provides a user interface for users to enter questions. This interface is designed as either a web application or a desktop application.

[0969] Step 7:

[0970] The user logs into the user interface and enters their question in natural language. For example, they might type, "Please tell me the steps for preparing the latest quarterly financial report."

[0971] Step 8:

[0972] The terminal sends the entered question to the server. The text entered in the user interface is sent to the server as an HTTP POST request.

[0973] Step 9:

[0974] The server tokenizes the received question and queries a generative artificial intelligence model. The model generates an appropriate answer based on the training data.

[0975] Step 10:

[0976] The server restores the generated response to text format and formats it. For example, it might generate a response such as, "The procedure for preparing the latest quarterly financial report is to collect income and expenditure reports from each department, integrate them, and create the final financial statement."

[0977] Step 11:

[0978] The server sends the formatted response to the user interface.

[0979] Step 12:

[0980] The terminal displays the response received from the server on the user interface. The user can view this response and use it for their work.

[0981] (Example 1)

[0982] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[0983] Traditional job handover processes frequently resulted in insufficient information and omissions, placing a significant burden on new and transferred employees in learning their new roles. Furthermore, manual information gathering and analysis were time-consuming and labor-intensive, leading to decreased work efficiency. Therefore, a system was needed to efficiently collect and analyze past work data and provide new and transferred employees with timely and accurate information.

[0984] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0985] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, means for converting the data into text format, means for masking personal and confidential information, means for dynamically adjusting the hyperparameters of the generative artificial intelligence model, means for sending the user's questions to the server as HTTP requests, and means for formatting the generated answers into a format that is easy for the user to understand. This enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and handover omissions, and improve work efficiency.

[0986] An "information processing device" is a system that includes hardware and software for collecting, processing, analyzing, and providing data results.

[0987] "Data related to past work" refers to records of work performed by a specific department within a company in the past, and includes emails, chat history, documents, etc.

[0988] "Preprocessing" is the process of converting collected data into an analyzable format by removing noise and unnecessary information, and masking personal and confidential information.

[0989] A "generative artificial intelligence model" is an AI model trained to perform natural language processing tasks using large amounts of text data, and models like GPT-4 fall into this category.

[0990] "Natural language questions from users" are inquiries or questions written by users in human language, specifically those entered as sentences or phrases.

[0991] "Means for generating answers" refers to a system that uses a generative artificial intelligence model to perform processing to generate appropriate answers to user questions.

[0992] A "user interface" is the part that provides the visual and manipulative means for a user to interact with a system, and is implemented as a web application or desktop application.

[0993] "Means of converting to text format" refers to the process of converting collected data into text data, and may involve using technologies such as OCR (Optical Character Recognition).

[0994] "Methods for masking personal and confidential information" refer to systems that process collected data to conceal information that could identify a specific individual or confidential company information.

[0995] "Methods for dynamically adjusting hyperparameters" refers to the process of dynamically changing parameters such as learning rate and batch size in order to optimize the performance of a generative artificial intelligence model.

[0996] "Means of sending to a server as an HTTP request" refers to the means by which a user enters a question from their terminal and sends it to a server via the Internet Protocol.

[0997] "Means of formatting into a user-friendly format" refers to a system that processes generated responses into an appropriate format so that users can easily understand them.

[0998] This invention is an information processing system that utilizes generative artificial intelligence to prevent information deficiencies and omissions during the handover of duties between new employees and transferred employees, and to improve work efficiency. The embodiments for carrying out this invention will be described in detail below.

[0999] First, the server automatically collects data related to past business operations. This data is retrieved from the company's email server, chat server, file server, or cloud storage. For example, the server uses an API to download emails, chat history, and accounting-related documents from the finance department for the past three years.

[1000] The collected data is preprocessed by the server. Preprocessing includes converting the data to text format, removing noise and unwanted information, and masking personal and sensitive information. For example, the server uses Python libraries (such as pdfminer and beautifulsoup) to convert documents to text format and removes unwanted information using regular expressions. It also uses NER (Named Entity Recognition) to identify and appropriately mask sensitive information.

[1001] Next, using the preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). The server sets up the training environment (e.g., using TensorFlow or PyTorch), sets the hyperparameters appropriately, and trains the model. During training, the server monitors the progress and dynamically adjusts the hyperparameters as needed. For example, it might set the learning rate to 0.001 and the batch size to 32, and train for 500 epochs.

[1002] Once the training is complete, the generative artificial intelligence model becomes available through a user interface. The server provides the user interface as a web application or desktop application. Users can log in to the system through this interface and enter questions. For example, a new employee could ask, "What were the key topics from yesterday's meeting?"

[1003] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and generates an appropriate answer. The generated answer is then formatted by the server into a user-friendly format and sent back to the terminal. The terminal displays this answer on its user interface, allowing the user to instantly obtain an answer to their question.

[1004] As a concrete example, consider a scenario where a new employee is assigned to the finance department. The new employee, acting as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" This question is sent from the terminal to the server, and the server, using a generative artificial intelligence model trained on pre-processed data, generates the answer: "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement." The server returns this answer to the terminal, and the user can view it.

[1005] As a result, the system of the present invention enables new employees and employees transferring to other departments to quickly and accurately resolve questions related to their work, avoid information gaps and omissions in handover, and improve work efficiency.

[1006] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1007] Step 1:

[1008] Data collection

[1009] The server automatically retrieves data related to past business operations. First, the server connects to the company's mail server, chat server, file server, or cloud storage via API calls. Then, the server downloads emails, chat history, and documents from each department for the past three years. For example, the server retrieves documents from the finance department from AWS S3 and downloads email history using the Gmail API.

[1010] Input: Data from the company's email server, chat server, and file server.

[1011] Output: Email, chat history, and documents for each department over the past 3 years.

[1012] Step 2:

[1013] Data preprocessing

[1014] The server preprocesses the collected data. First, it converts the collected data into text format. It uses Python libraries (e.g., pdfminer or beautifulsoup) to convert documents to text format. Next, it uses regular expressions to remove unwanted information and noise (e.g., email signatures and advertisements). Finally, it uses NER (Named Entity Recognition) to mask personal and sensitive information.

[1015] Input: Email and chat history for each department over the past three years, and documents.

[1016] Output: Data converted to text format, with unnecessary information removed and personal and confidential information masked.

[1017] Step 3:

[1018] AI model training and fine-tuning

[1019] The server trains and fine-tunes generative artificial intelligence models using preprocessed data. First, it prepares the training dataset using Python and sets up the training environment using TensorFlow or PyTorch. The server sets hyperparameters such as the learning rate and batch size and starts training the model. During training, the server monitors the learning progress and dynamically adjusts the hyperparameters as needed.

[1020] Input: Data converted to text format, with personal and confidential information masked.

[1021] Output: Trained and fine-tuned generative artificial intelligence model

[1022] Step 4:

[1023] Providing a user interface

[1024] The server provides a user interface for using generative artificial intelligence models. First, the user interface for the web application is designed and implemented using React.js, allowing users to access the system. If it is provided as a desktop application, the interface is built using Electron or similar. The server also manages user authentication information in a database (e.g., MySQL, PostgreSQL) and integrates it into the user interface.

[1025] Input: Generative artificial intelligence model

[1026] Output: User interface provided as a web application or desktop application.

[1027] Step 5:

[1028] Question submission and answer generation

[1029] When a user enters a question, it is sent to the server via the terminal. The terminal sends the question to the server as an HTTP request, and the server poses the question to a generative artificial intelligence model. The model generates an appropriate answer based on its trained knowledge. For example, in response to the question, "What were the main topics of discussion at yesterday's meeting?", the model might generate the answer, "At yesterday's meeting, a new sales strategy was discussed."

[1030] Input: Questions entered via the user interface

[1031] Output: Answers generated by a generative artificial intelligence model

[1032] Step 6:

[1033] Providing a response

[1034] The server formats the generated response and returns it to the terminal via the user interface. The terminal receives this response and displays it on the user interface. Specifically, the response is displayed on the interface of a web application or desktop application, allowing the user to instantly obtain an answer to their question.

[1035] Input: Answer generated by a generative artificial intelligence model

[1036] Output: Answer displayed on the user interface

[1037] (Application Example 1)

[1038] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1039] New employees and transferred employees are required to quickly and efficiently understand their duties in the factory environment and perform their work smoothly. However, traditional handover processes often result in insufficient information or omissions, preventing new employees from immediately adapting to on-site situations. This raises concerns about decreased productivity and increased errors. Therefore, a system is needed that can effectively facilitate job handover in a factory environment.

[1040] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1041] In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means for generating answers to natural language questions from the user based on the generative artificial intelligence model, means for presenting the generated answers to the user via a user interface, and a user interface related to smart glasses that provide visual work instructions and explanations, particularly in a factory environment. This enables new employees and transferred employees to quickly become proficient in their work within the factory, improving productivity and reducing errors.

[1042] An "information processing device" is a device that performs tasks such as data collection, processing, analysis, and output.

[1043] "Data related to past work" includes all information related to a specific task, such as work processes, work history, meeting records, and message history.

[1044] "Preprocessing" is the process of removing noise and unnecessary information from collected data to make it suitable for useful analysis.

[1045] A "generative artificial intelligence model" is a machine learning model specialized in natural language processing and data generation. It learns from large amounts of data and is capable of answering questions and generating text.

[1046] A "user interface" is a collection of screens, input devices, and interaction methods that a user uses when operating a system.

[1047] "Smart glasses" are glasses-type devices worn by the wearer that have the function of displaying information visually.

[1048] "Visual work instructions" refer to instructions that visually indicate work procedures and precautions using text, images, videos, etc.

[1049] "Training methods" refer to the techniques and processes used to train a generative artificial intelligence model using collected and pre-processed data.

[1050] "Means of generating answers" refers to the process by which a generative artificial intelligence model creates an appropriate answer to a question from a user.

[1051] This invention provides a system that supports new employees and transferred employees in a factory environment to quickly and efficiently understand their tasks and perform their work smoothly. Specific embodiments of this system are described below.

[1052] System Overview

[1053] The server collects data from the factory department to retrieve data related to past operations. Specifically, it collects relevant files and messages from the company's mail server, chat server, file server, or cloud storage as means of data acquisition. For example, the server can download past work data and records via an API.

[1054] Since the acquired data cannot be used as is, the server preprocesses the data. As a preprocessing step, the data is converted to text format, noise and unnecessary information (e.g., email signatures and advertisements) are removed, and personal and confidential information is masked. This step generates a clean dataset.

[1055] Training of generative artificial intelligence models

[1056] Using a clean dataset, the server trains a generative artificial intelligence model (e.g., GPT-4). The model learns key information and patterns related to specific tasks as part of the training process. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[1057] The server uses a generative artificial intelligence model to generate answers to natural language questions from users. As a means of generating answers, users input questions through smart glasses, and these questions are sent to the server. The server then poses the question to a trained AI model and generates an answer. For example, a new employee might ask, "What is the next step?", and the AI ​​might answer, "Attach part A to part B."

[1058] User Interface

[1059] The server presents the generated answers to the user via a user interface. In particular, in factory environments, smart glasses are used to provide visual work instructions and explanations. This allows new employees and transferred employees to resolve questions in real time while performing their actual work.

[1060] Hardware and software to be used

[1061] Hardware: Smart glasses (e.g., Vuzix Blade)

[1062] Software: OpenAI GPT-4 API, data preprocessing tools

[1063] As a concrete example, a new employee wearing smart glasses stands on a manufacturing line and asks, "What is the next step?" This question is sent to a server, and a generative artificial intelligence model generates the answer, "Attach part A to part B." This answer is displayed on the smart glasses' screen, allowing the new employee to follow the instructions and proceed with the task.

[1064] Example of a prompt

[1065] "Manufacturing Line Work Instruction Prompt: Generate detailed instructions to help new employees understand the steps on the manufacturing line. For example, write instructions for attaching part A to part B."

[1066] In this way, the system of the present invention utilizes past business data to quickly and accurately respond to questions from new employees and employees transferring to other departments, thereby enabling a smooth handover of duties.

[1067] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1068] Step 1:

[1069] Data collection

[1070] The server uses means to retrieve data related to past work. Specifically, the server collects data from the company's mail server, chat server, file server, or cloud storage. It downloads past work data and records via API and saves them to local storage. Inputs include emails, chat history, and work manuals, and output is raw data stored in local storage.

[1071] Step 2:

[1072] Data preprocessing

[1073] The server uses means to preprocess the acquired data. Specifically, it removes noise and unnecessary information (e.g., email signatures, advertisements) from the collected data and converts it to text format. It also filters and masks personal and confidential information. Raw data is given as input, and a clean dataset is generated as output. Specific data preprocessing steps include text cleaning using natural language processing tools.

[1074] Step 3:

[1075] Training of artificial intelligence models

[1076] The server performs a means of training a generative artificial intelligence model using preprocessed data. Specifically, it fine-tunes the model (e.g., GPT-4) based on a large amount of text data. The server monitors the model's performance and dynamically adjusts hyperparameters as needed. A clean dataset is used as input, and the output is a trained generative artificial intelligence model.

[1077] Step 4:

[1078] Accepting questions

[1079] The device (smart glasses) accepts questions from the user in natural language. The user inputs the question through the smart glasses' interface, and this is sent to the server. The input is the question from the user, and the output is the question data sent to the server.

[1080] Step 5:

[1081] Answer generation

[1082] The server uses a means of generating answers to user questions based on a generative artificial intelligence model. Specifically, the server feeds question data to the generative AI model and generates an appropriate answer. The question data is used as input, and the generated answer is obtained as output.

[1083] Step 6:

[1084] Providing an answer

[1085] The server performs a means of presenting the answer via a user interface. The generated answer is visually displayed to the user through smart glasses. The input is the generated answer, and the output is the answer displayed on the terminal (smart glasses).

[1086] This processing flow allows new employees and transferred employees to receive timely and relevant information, improving factory work efficiency.

[1087] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1088] This invention is a system that utilizes generative artificial intelligence and an emotion engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency. A detailed embodiment of this system will be described below.

[1089] Data collection

[1090] The server automatically retrieves historical work data for each department. Specifically, it collects data from email servers, chat servers, file servers, or cloud storage. For example, the server can use APIs to download emails, chat histories, and documents that are tagged or stored in specific folders.

[1091] Data preprocessing

[1092] Since the collected data cannot be used as is, the server preprocesses the data. Specifically, it converts the data to text format, filters out noise and unnecessary information (e.g., email signatures, ad blockers), and masks personal and confidential information. This step generates a clean dataset.

[1093] AI model training and fine-tuning

[1094] Using preprocessed data, the server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4). During this process, the model learns important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[1095] Emotional engine integration

[1096] Furthermore, the server combines a generative artificial intelligence model with an emotion engine. The emotion engine can recognize and evaluate emotions from user input and interactions. It detects user emotions using text analysis, speech analysis, or facial recognition. Based on this emotion information, the tone and content of the generated responses can be adjusted.

[1097] Providing a user interface

[1098] After the generative AI model and emotion engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, and users can input questions through it and receive emotion-based responses. For example, a new employee could ask, "What were the key topics from yesterday's meeting?" after logging in.

[1099] Question submission and answer generation

[1100] When a user enters a question, it is sent to the server via the terminal. The server then presents this question to a generative artificial intelligence model and an emotion engine, which generates an appropriate answer based on its trained knowledge and the user's emotional state. For example, it might generate an answer that takes the user's emotions into consideration, such as, "At yesterday's meeting, we discussed a new sales strategy."

[1101] Providing a response

[1102] The server formats the generated response and returns it to the terminal via the user interface. The terminal then displays the response to the user. Specifically, the response is displayed on the web application screen, allowing new employees to immediately resolve their work-related questions. This helps avoid problems caused by handover omissions or insufficient information, thereby improving work efficiency.

[1103] Specific example

[1104] For example, suppose a new employee is assigned to the finance department. The new employee, as the user, asks, "Could you please explain the procedure for preparing the most recent quarterly financial report?" The question is sent from the terminal to the server, and the emotion engine senses anxiety or tension from the user's input. The generative artificial intelligence model takes this emotional information into consideration and generates a response in a gentle tone, such as, "The procedure for preparing the most recent quarterly financial report is to first collect income and expenditure reports from each department, and then integrate them to create the final financial statement. If you have any problems, please feel free to ask." The server returns this response to the terminal, and the user can see the answer.

[1105] In this way, the system of the present invention utilizes past business data to respond quickly and accurately to questions from new employees and employees transferring to other departments, and by integrating an emotion engine, it provides responses that take into account the user's emotions, thereby enabling a smooth handover of duties.

[1106] The following describes the processing flow.

[1107] Step 1:

[1108] The server automatically retrieves historical business data from email servers, chat servers, file servers, or cloud storage. For example, the server uses an API to download email and chat history from the finance department for the past three years.

[1109] Step 2:

[1110] The server converts the acquired data into text format. For example, PDF files and scanned documents are converted into text data using OCR (Optical Character Recognition). Excel files and Word documents are also converted into text format.

[1111] Step 3:

[1112] The server preprocesses the retrieved text data to remove noise and unwanted information (e.g., email signatures and ad blockers). For example, it might use regular expressions to filter out footer text such as "Best regards" or advertisements.

[1113] Step 4:

[1114] The server masks personal and confidential information during the preprocessing stage. For example, names and email addresses are replaced with placeholders such as "[Name]" and "[Email Address]".

[1115] Step 5:

[1116] The server trains and fine-tunes a generative artificial intelligence model (e.g., GPT-4) using pre-processed data. During this process, the server uses a learning framework (e.g., PyTorch) to ensure the model learns properly.

[1117] Step 6:

[1118] The server integrates an emotion engine into the generative artificial intelligence model. For example, the emotion engine incorporates algorithms that detect and evaluate the user's emotions using text analysis, speech analysis, or facial recognition.

[1119] Step 7:

[1120] The user logs into the user interface and enters a question. This interface is provided as a web application or desktop application and includes a form for the user to enter the question in natural language.

[1121] Step 8:

[1122] The terminal sends the user-entered question to the server. Specifically, it sends the question's text data to the server as an HTTP POST request.

[1123] Step 9:

[1124] The server inputs the received question into a generative artificial intelligence model and an emotion engine. At this time, the server tokenizes the question and passes it to the model as input data.

[1125] Step 10:

[1126] The emotion engine analyzes emotional information from the user's questions. For example, the emotion engine extracts emotional tags such as "tension," "anxiety," and "joy" from the text of the question.

[1127] Step 11:

[1128] Generative artificial intelligence models generate answers to questions while taking emotional information into account. For example, if the user is nervous, the model will soften the tone of the answer and use gentle expressions such as, "I'll explain it simply, so don't worry."

[1129] Step 12:

[1130] The server restores the generated response to text format and formats it. For example, it might generate a response like, "The procedure for preparing the most recent quarterly financial report is to collect income and expenditure reports from each department, integrate them, and then create the final financial statement. Please feel free to ask questions if you have any problems."

[1131] Step 13:

[1132] The server sends the formatted response to the user interface. Specifically, it returns the response data as an HTTP response.

[1133] Step 14:

[1134] The terminal displays the response received from the server on the user interface. Users can view this response and use it to help with their work. For example, they can gain a concrete understanding of "the procedure for preparing the most recent quarterly financial report."

[1135] In this way, by using a generative artificial intelligence model that integrates an emotion engine, appropriate responses can be obtained that are tailored to the user's emotional state. This not only facilitates smooth handover of tasks but also reduces the psychological burden on the user.

[1136] (Example 2)

[1137] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1138] In companies and organizations, the handover of duties for new employees and employees transferring to other departments can sometimes be unsuccessful. In such cases, important information may be leaked, and knowledge related to the work may not be properly shared, leading to decreased work efficiency and an increased likelihood of errors. Furthermore, new employees and transferring employees often experience anxiety and stress during the handover process, which further reduces work efficiency. Traditional handover methods are insufficient to adequately address these problems.

[1139] The identification processing by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information and mask personal and confidential information, means for training a generative artificial intelligence model using the preprocessed data, means for integrating an emotion recognition engine into the generative artificial intelligence model and evaluating emotions from user input, means for generating answers to natural language questions from the user based on the integrated generative artificial intelligence model and adjusting the tone and content of the answers based on the user's emotions, and means for presenting the generated answers to the user via a user interface. As a result, it is possible to obtain responses that take into account the emotions the user feels during the handover of operations, preventing information handover omissions and incomplete information sharing, and enabling increased operational efficiency and reduced troubles.

[1140] An "information processing device" is a combination of hardware and software for collecting, preprocessing, analyzing, generating, and outputting data.

[1141] "Data related to past business operations" refers to documents and communication data that contain information about business activities carried out in the past.

[1142] "Preprocessing" refers to the process of removing noise and unwanted information, converting formats, and masking personal and confidential information in order to transform collected data into a usable format.

[1143] A "generative artificial intelligence model" refers to a machine learning algorithm that can find patterns in large amounts of data and generate new data.

[1144] An "emotion recognition engine" is an algorithm or software that detects emotions from user input and actions and provides feedback based on those emotions.

[1145] "User" refers to an individual or member of an organization that uses this system.

[1146] "Means for generating answers to questions" refers to a combination of software and hardware that analyzes natural language questions from users and generates appropriate answers.

[1147] A "user interface" is an application that has a display screen and input mechanism for a user to interact with the system.

[1148] "Noise and unnecessary information" refers to irrelevant data and information that does not affect the analysis or generation process.

[1149] "Personal information and confidential information" refers to information that includes the identification of an individual or highly confidential content, and is subject to confidentiality obligations.

[1150] "Adjusting the tone and content of responses" refers to generating responses that are not cruel, taking into account the user's emotional state.

[1151] "Real-time" refers to the time range in which data is processed and outputted with only a slight delay after it is input.

[1152] An "API" refers to an interface used to access the functions of different software programs.

[1153] A "web application" refers to software that can be accessed through an internet browser.

[1154] A "desktop application" refers to software that runs directly on a personal computer.

[1155] This invention is a system that utilizes generative artificial intelligence and an emotion recognition engine to prevent information deficiencies and omissions in the handover of duties between new employees and transferred employees in each department, and to improve work efficiency.

[1156] The server first automatically retrieves data related to past work for each department. Data collection utilizes services such as email servers (e.g., Microsoft Exchange), chat servers (Slack), file servers (NAS), and cloud storage (Google Drive, Dropbox). The server uses APIs provided by these services (e.g., Google Drive API and Slack API) to download emails, chat histories, and documents associated with specific folders or tags. This allows the server to grasp a comprehensive overview of the information users need.

[1157] The collected data undergoes preprocessing. The server converts the data to text format using Python's BeautifulSoup or NLP libraries (e.g., NLTK, SpaCy), removes noise and unnecessary information (such as email signatures and ad blockers), and masks personal and sensitive information. This step generates a clean and usable dataset.

[1158] Next, the server trains a generative artificial intelligence model (e.g., GPT-4) using the preprocessed data. Machine learning libraries such as PyTorch and TensorFlow are used for training, the training progress is monitored, and hyperparameters (such as learning rate and batch size) are dynamically adjusted as needed. This allows the model to learn important information and patterns related to specific tasks.

[1159] Furthermore, the emotion recognition engine is integrated into the generative artificial intelligence model. The server uses the emotion recognition engine to evaluate emotions from user input and interactions. For this purpose, TextBlob is used for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. Based on the emotional information, the tone and content of the generated responses will be adjusted. For example, if a user enters a question while feeling anxious, a response in a gentle tone that takes that emotion into account will be provided.

[1160] After the generative AI model and emotion recognition engine are trained, the server provides a user interface. This interface is designed as a web or desktop application, using React or Vue.js for the frontend and Node.js or Django for the backend. Users can input questions through the interface and receive answers in real time.

[1161] For example, if a new employee, who is a user, enters a question such as "Please tell me the procedure for preparing the most recent quarterly financial report," that question will be sent to the server via the terminal.

[1162] The server poses this question to a generative artificial intelligence model and an emotion recognition engine, which analyze the data and generate a response tailored to the user's emotional state. For example, if the emotion recognition engine detects anxiety from the user's input, it will generate a gentle response such as, "The procedure for preparing the most recent quarterly financial report involves first collecting income and expenditure reports from each department, and then integrating them to create the final financial statement. Please feel free to ask if you have any problems."

[1163] The generated answers are sent from the server to the terminal, and the user can display the answers on the screen. In this way, new employees and employees transferring to other departments can get quick and accurate answers to their work-related questions, leading to increased work efficiency.

[1164] This system provides responses that take into account the emotions users feel during the handover of tasks, preventing information from being missed or incomplete, and enabling increased work efficiency and a reduction in problems.

[1165] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1166] Step 1: Data Collection

[1167] The server collects data related to past business operations from cloud storage, mail servers, chat servers, and file servers. Specifically, the server uses APIs (e.g., Google Drive API, Slack API) to download emails, chat histories, and documents tagged with specific folders from each service. It accepts API keys and folder paths from each service as input and saves the downloaded data as a file as output.

[1168] Step 2: Data Preprocessing

[1169] The server preprocesses the collected data. It converts the data to plain text and removes noise and unnecessary information. Specifically, it uses Python libraries (e.g., BeautifulSoup, NLTK, SpaCy) to convert HTML emails to plain text and remove advertisements and personal information. It receives downloaded raw data as input and generates clean text data as output.

[1170] Step 3: Training the AI ​​model

[1171] The server trains a generative artificial intelligence model (e.g., GPT-4) using pre-processed text data. Specifically, it uses PyTorch or TensorFlow to feed a large amount of data into the model and train it. It takes clean text data as input and produces a trained model as output.

[1172] Step 4: Integrating the emotion recognition engine

[1173] The server integrates an emotion recognition engine with a trained generative artificial intelligence model. It analyzes user emotions using TextBlob for text analysis, Google Speech-to-Text API for speech analysis, and OpenCV for facial recognition. It receives user interaction data (text, audio, images) as input and outputs analyzed emotion information.

[1174] Step 5: Provide User Interface

[1175] The server provides the user interface. It is designed as a web application (React, Node.js) or a desktop application (Electron). Through this interface, users can input questions and receive answers in real time. It receives user questions as input and displays generated answers as output.

[1176] Step 6: Accepting Questions

[1177] The user enters a question into the user interface, and this is sent to the server via the terminal. Specifically, the entered question is sent to the API endpoint and received as data for processing. It receives the user's question as input and obtains data for processing as output.

[1178] Step 7: Generate Response

[1179] The server uses a generative artificial intelligence model and an emotion recognition engine to generate answers to user questions. For example, if it detects anxiety from the user's input, the model will generate an answer in a gentle tone. It receives user questions and emotion data as input and obtains the generated answer as output.

[1180] Step 8: Provide your answer

[1181] The server formats the generated response and returns it to the terminal. The terminal interprets this response and displays it on the user interface. Specifically, it displays the JSON-formatted response data sent via the API endpoint in HTML format. It receives the generated response as input and provides the response presented to the user as output.

[1182] These specific processing steps create an environment where new employees and transferred employees in each department can smoothly take over their duties, thereby improving work efficiency.

[1183] (Application Example 2)

[1184] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1185] Traditional handover systems had problems that made it difficult for new employees and employees transferring to other departments to quickly and accurately acquire information about their work. Furthermore, inefficiencies and errors were common due to insufficient information and missed handovers, and the systems often lacked consideration for the feelings of users. In addition, in places like logistics centers, there was a need for support to help staff understand complex work processes more easily.

[1186] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for acquiring data related to past operations, means for preprocessing the acquired data to remove noise and unnecessary information, means for training a generative artificial intelligence model using the preprocessed data, means including an emotion engine that analyzes the user's emotions and adjusts the tone of the response based on the results, and means for presenting the generated response to the user via a smartphone application. This enables new employees and transferred employees to answer questions about their work quickly and accurately, and allows for responses that are sensitive to emotions. It also provides support to help logistics center staff quickly understand work procedures and processes.

[1187] An "information processing device" is a device that acquires past business data, performs data preprocessing, trains generative artificial intelligence models, and generates answers to user questions in natural language.

[1188] A "generative artificial intelligence model" is an artificial intelligence model that is trained using collected data, learns information and patterns related to specific tasks, and generates answers to user questions.

[1189] An "emotion engine" is a system that recognizes and evaluates emotions from user input and interactions, and is used to adjust the tone and content of responses generated by generative artificial intelligence models.

[1190] "Preprocessing" refers to a series of processes that convert collected data into text format, remove noise and unnecessary information, and mask personal and confidential information.

[1191] The "user interface" refers to the interface through which users access this system, input questions, and review generated answers.

[1192] A "smartphone application" is an application that runs on a smartphone and allows users to input work-related questions and receive answers.

[1193] A "logistics center" refers to a facility or area where logistics operations such as the storage, management, and shipping of goods are carried out.

[1194] "Tone" refers to the manner in which the generated response is spoken and expressed, and it is adjusted to provide a user-friendly response in a kind and gentle tone.

[1195] The system for implementing this invention is centered around an information processing device and consists of the following steps.

[1196] First, the server automatically retrieves data related to past operations from various sources. These sources include email servers, chat servers, file servers, and cloud storage. Specifically, the server uses APIs to download emails, chat history, documents, and other files that are tagged or stored in specific folders.

[1197] Next, the server preprocesses the acquired data. This preprocessing includes converting the data to text format, filtering out noise and unnecessary information (such as email signatures and ad blockers), and masking personal and sensitive information. This generates a clean dataset.

[1198] Next, the server uses the preprocessed data to train and fine-tune a generative artificial intelligence model (e.g., GPT-4). This allows the model to learn important information and patterns related to specific tasks. The server monitors the model's learning progress and dynamically adjusts hyperparameters as needed.

[1199] Subsequently, the server integrates an emotion engine into the generative artificial intelligence model. The emotion engine has the ability to recognize and evaluate emotions from user input and interactions. This utilizes text analysis, speech analysis, or facial recognition technology. Based on the emotional information, the tone and content of the generated responses are adjusted.

[1200] The user interface is provided in the form of a smartphone application. Through this application, users can input work-related questions in text format and receive immediate answers. Once a question is entered, it is sent via the device to a server, which generates an answer based on a generative artificial intelligence model and an emotion engine. This answer is generated in an appropriate tone, taking into account the user's emotional state.

[1201] For example, if a user asks, "Could you please explain the procedure for taking inventory?", the emotion engine senses tension and anxiety from the user's input. The generative artificial intelligence model takes this emotional information into account and generates a response in a gentle tone, such as, "The procedure for taking inventory is as follows: First, check the quantity of each product, and then enter it into the database. Please let us know if you have any problems." This response is then provided to the user through the smartphone application interface.

[1202] Examples of prompt statements include the following:

[1203] Question to the AI: "Please explain the procedure for taking inventory."

[1204] Emotional status: "Nervous"

[1205] Answer: "The procedure for inventory counting is as follows: First, confirm the quantity of each product, and then enter it into the database. Please let us know if you have any problems."

[1206] In this way, new employees and transferred employees can quickly and accurately resolve work-related questions, and logistics center staff are provided with support to immediately understand work procedures and processes.

[1207] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1208] Step 1:

[1209] The server retrieves data related to past operations. Specifically, it uses APIs to download emails, chat histories, and documents with specific folders or tags from mail servers, chat servers, file servers, or cloud storage. The input for this step is an API request, and the output is the downloaded raw data.

[1210] Step 2:

[1211] The server preprocesses the acquired data. First, it converts the data to text format, then filters out noise and unnecessary information (e.g., email signatures and ad blockers). It also masks personal and confidential information. The input for this step is the downloaded original data, and the output is clean text data.

[1212] Step 3:

[1213] The server uses preprocessed data to train and fine-tune a generative artificial intelligence model (e.g., GPT-4). Clean text data is used as training data, and the model learns important information and patterns related to specific tasks. The input for this step is clean text data, and the output is the trained AI model.

[1214] Step 4:

[1215] The server integrates an emotion engine into the generative artificial intelligence model. The emotion engine recognizes and evaluates emotions from user input and interactions, primarily through text analysis. The input for this step is a trained AI model and a dataset for emotion analysis, while the output is a turnkey system that generates emotion-sensitive responses.

[1216] Step 5:

[1217] The user uses a smartphone application to input questions related to their work. For example, they might input, "Please explain the procedure for taking inventory." The input in this step is text input by the user, and the output is that text question.

[1218] Step 6:

[1219] The terminal sends the user's question to the server. The server receives the question, performs sentiment analysis, and generates an answer using a generative artificial intelligence model. The input for this step is the question text sent by the user, and the output is the generated answer.

[1220] Step 7:

[1221] The server formats the generated response and sends it back to the user via a smartphone application. The input for this step is the generated response, and the output is the final formatted response text.

[1222] Step 8:

[1223] The user reviews the responses generated through a smartphone application. For example, they might receive a response such as, "The inventory count procedure is as follows: First, check the quantity of each product, then enter it into the database. Please let us know if you have any problems." The input in this step is the formatted response returned from the server, and the output is the user's understanding and actions.

[1224] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1225] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1226] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1227] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1228] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1229] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1230] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1231] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1232] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1233] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1234] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1235] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1236] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1237] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1238] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1239] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1240] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1241] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1242] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1243] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1244] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1245] The following is further disclosed regarding the embodiments described above.

[1246] (Claim 1)

[1247] The information processing device provides a means for acquiring data related to past operations,

[1248] A means of preprocessing the acquired data to remove noise and unnecessary information,

[1249] A means for training a generative artificial intelligence model using preprocessed data,

[1250] A means for generating answers to natural language questions from users based on a generative artificial intelligence model,

[1251] A system that includes means for presenting a generated response to a user via a user interface.

[1252] (Claim 2)

[1253] The system according to claim 1, characterized in that the user interface is provided as a web application or a desktop application.

[1254] (Claim 3)

[1255] The system according to claim 1, characterized in that the data preprocessing means includes a filtering function for masking personal information and confidential information.

[1256] "Example 1"

[1257] (Claim 1)

[1258] The information processing device provides a means for acquiring data related to past operations,

[1259] A means of preprocessing the acquired data to remove noise and unnecessary information,

[1260] A means for training a generative artificial intelligence model using preprocessed data,

[1261] A means for generating answers to natural language questions from users based on a generative artificial intelligence model,

[1262] A means of presenting a generated answer to the user via a user interface,

[1263] A means of converting data into text format,

[1264] Methods for masking personal and confidential information,

[1265] A means for dynamically adjusting the hyperparameters of a generative artificial intelligence model,

[1266] A means of sending a user's question to the server as an HTTP request,

[1267] A system that includes means for formatting the generated answers into a format that is easy for the user to understand.

[1268] (Claim 2)

[1269] The system according to claim 1, characterized in that the user interface is provided as a web application or a desktop application.

[1270] (Claim 3)

[1271] The system according to claim 1, characterized in that the data preprocessing means includes a filtering function for masking personal information and confidential information.

[1272] "Application Example 1"

[1273] (Claim 1)

[1274] The information processing device provides a means for acquiring data related to past operations,

[1275] A means of preprocessing the acquired data to remove noise and unnecessary information,

[1276] A means for training a generative artificial intelligence model using preprocessed data,

[1277] A means for generating answers to natural language questions from users based on a generative artificial intelligence model,

[1278] A means of presenting a generated answer to the user via a user interface,

[1279] This includes user interfaces related to smart glasses that provide visual work instructions and explanations, particularly in factory environments.

[1280] system.

[1281] (Claim 2)

[1282] The system according to claim 1, characterized in that the user interface is provided as a web application or a desktop application.

[1283] (Claim 3)

[1284] The system according to claim 1, characterized in that the data preprocessing means includes a filtering function for masking personal information and confidential information.

[1285] "Example 2 of combining an emotion engine"

[1286] (Claim 1)

[1287] The information processing device provides a means for acquiring data related to past operations,

[1288] Methods for preprocessing acquired data to remove noise and unnecessary information, and for masking personal and confidential information,

[1289] A means for training a generative artificial intelligence model using preprocessed data,

[1290] Integrating an emotion recognition engine into a generative artificial intelligence model, providing a means to evaluate emotions from user input,

[1291] A means for generating answers to natural language questions from users based on an integrated generative artificial intelligence model, and for adjusting the tone and content of the answers based on the user's emotions,

[1292] A system that includes means for presenting a generated response to a user via a user interface.

[1293] (Claim 2)

[1294] The system according to claim 1, characterized in that the user interface is provided as a web application or a desktop application.

[1295] (Claim 3)

[1296] The system according to claim 1, characterized in that the emotion recognition engine includes a function to evaluate the user's emotions based on text analysis, speech analysis, or facial recognition.

[1297] "Application example 2 when combining with an emotional engine"

[1298] (Claim 1)

[1299] The information processing device provides a means for acquiring data related to past operations,

[1300] A means of preprocessing the acquired data to remove noise and unnecessary information,

[1301] A means for training a generative artificial intelligence model using preprocessed data,

[1302] A means for generating answers to natural language questions from users based on a generative artificial intelligence model,

[1303] A means including an emotion engine that analyzes the user's emotions and adjusts the tone of the response based on the results,

[1304] A means of presenting a user with a generated answer via a smartphone application,

[1305] A system that includes this.

[1306] (Claim 2)

[1307] The system according to claim 1, characterized in that the user interface is provided as a web application or a desktop application.

[1308] (Claim 3)

[1309] The system according to claim 1, characterized in that the data preprocessing means includes a filtering function for masking personal information and confidential information. [Explanation of Symbols]

[1310] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. The information processing device provides a means for acquiring data related to past operations, A means of preprocessing the acquired data to remove noise and unnecessary information, A means for training a generative artificial intelligence model using preprocessed data, A means for generating answers to natural language questions from users based on a generative artificial intelligence model, A system that includes means for presenting a generated response to a user via a user interface.

2. The system according to claim 1, characterized in that the user interface is provided as a web application or a desktop application.

3. The system according to claim 1, characterized in that the data preprocessing means includes a filtering function for masking personal information and confidential information.

Images