system

Abstract

We provide the system. [Solution] A means by which the user specifies the location where business-related data is stored, The means by which the server collects data from the storage location, The server provides a means for cleansing and standardizing the format of the collected data, A server extracts important information from cleansed and formatted data and provides a means for building a knowledge base. A server provides a means for inputting a knowledge base into a generative artificial intelligence and training it, A means for a server to generate a handover document using a trained generative artificial intelligence model, A means for users to review and modify the generated handover document, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, the method including: receiving a user utterance; adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot; encoding the prompt; and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Conventionally, in the handover of work, the person in charge has to manually sort out emails, meeting logs, and documents and create a handover document, which requires a great deal of time and labor. In addition, problems such as missing or leaking handover information and lack of information consistency are likely to occur. As a result, the person receiving the handover may not be able to start work smoothly, which may cause a decrease in productivity and a delay in work. In order to solve these problems, there is a need for a system that reduces the labor and time required for handover and automatically generates a more accurate and consistent handover document.

Means for Solving the Problems

[0005] This invention provides a means for a user to specify a location for storing business-related data, for a server to collect data from that location, and for the server to cleanse and standardize the format of that data. Furthermore, it provides a means for the server to extract important information from the cleansed and standardized data and build a knowledge base. Next, it provides a means for the server to input the knowledge base it has built into a generative artificial intelligence and train it, and finally provides a means for the server to automatically generate a handover document using the trained generative artificial intelligence model. It also provides a means for the user to review the generated handover document and make corrections as needed. This creates a system that significantly reduces the effort and time required for handover and enables the creation of more accurate and consistent handover documents.

[0006] A "user" is someone who utilizes the system, specifies the location for saving business-related data, and is responsible for reviewing and modifying the final generated handover document.

[0007] A "server" refers to a device or system that performs a series of processes, including collecting data from a storage location specified by the user, cleansing and standardizing the format of that data, building a knowledge base, training generative artificial intelligence, and generating handover documents.

[0008] "Data" refers to information such as emails, meeting logs, and documents related to work.

[0009] "Cleansing" refers to the process of removing unnecessary information and noise from collected data and extracting only useful information.

[0010] "Format standardization" refers to the process of converting data collected in different formats into a consistent format.

[0011] A "knowledge base" is a database that aggregates important information extracted from cleansed and formatted data, and possesses a certain structure.

[0012] "Generative artificial intelligence" refers to artificial intelligence technology that learns from a provided knowledge base and automatically generates handover documents.

[0013] "Training" refers to the process of teaching a generative artificial intelligence to perform a specific task using data from a knowledge base.

[0014] A "handover document" refers to a document automatically generated by generative artificial intelligence that summarizes the information necessary for handing over duties.

[0015] "Specified storage location" refers to storage areas such as email services, cloud storage, and internal file servers where users save work-related data. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9]Shows an emotion map to which multiple emotions are mapped. [Figure 10] Shows an emotion map to which multiple emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when the emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when the emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0018] First, the language used in the following description will be explained.

[0019] In the following embodiments, the numbered processor (hereinafter simply referred to as "processor") may be one arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be one type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), etc.

[0020] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, cleansing and standardizing the data format, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence system, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[0038] System operation description

[0039] 1. Data Collection Phase

[0040] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0041] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[0042] 2. Data preprocessing phase

[0043] The server cleanses the collected data. For example, the server removes spam emails and deletes unnecessary information from meeting logs.

[0044] The server converts the cleansed data into a unified format. For example, it formats meeting logs in various formats into a consistent template.

[0045] 3. Knowledge Base Construction Phase

[0046] The server extracts important information from pre-processed data. For example, it extracts key topics and project progress from email content.

[0047] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[0048] 4. AI Learning Phase

[0049] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[0050] 5. Handover document generation phase

[0051] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. For example, the AI ​​model might create a "Project X Personnel Handover Document" which would include the project's objectives, progress, details of the team members, and the next tasks to be performed.

[0052] The user reviews the generated handover document and makes corrections as needed. For example, if the user thinks "this section needs additional explanation," they manually edit that section.

[0053] Specific example

[0054] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0055] Email storage location: Email service A

[0056] Location where meeting logs are stored: Cloud storage service B

[0057] Document storage location: Internal file server C

[0058] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[0059] Once the generative AI model has completed its training, it automatically generates a handover document. This document includes a comprehensive overview of the project, ongoing tasks, and contact information for each person involved. The project manager reviews this document and makes revisions as needed. As a result, it becomes possible to create accurate and consistent handover documents, significantly reducing time and effort.

[0060] The following describes the processing flow.

[0061] Step 1:

[0062] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0063] Step 2:

[0064] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[0065] Step 3:

[0066] The server cleanses the collected data. For example, it filters spam from email data and performs text processing to remove unnecessary information from meeting logs.

[0067] Step 4:

[0068] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[0069] Step 5:

[0070] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[0071] Step 6:

[0072] The server builds a knowledge base based on the extracted information. This knowledge base integrates project overviews, progress, and contact information for each person in charge.

[0073] Step 7:

[0074] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[0075] Step 8:

[0076] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[0077] Step 9:

[0078] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[0079] (Example 1)

[0080] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0081] In today's business environment, managing and transferring business data is a critical challenge. However, because various data are stored in scattered locations, data collection and organization are cumbersome and inefficient. Furthermore, creating handover documents is time-consuming and laborious, and ensuring the accuracy and consistency of the information is difficult. To solve these problems, efficient collection and standardization of business data formats, extraction of key information and construction of a knowledge base, and automated generation of handover documents using AI are necessary.

[0082] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0083] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location using an API; means for the server to cleanse and format the collected data using a Python data processing library; means for the server to extract important information from the cleansed and formatted data using a natural language processing library, store it in a relational database, and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it using an API; means for the server to generate a handover document based on prompt statements using the trained generative artificial intelligence model; and means for the user to review and modify the generated handover document through a web interface. This enables efficient collection and organization of business data, and allows for the creation of consistent and accurate handover documents in a short amount of time.

[0084] A "user" is an entity that uses the system to specify the storage location for business-related data and to review and modify the generated handover documents.

[0085] A "server" is a computing device that collects data based on user instructions, cleanses and standardizes the format, extracts important information to build a knowledge base, and generates handover documents using generative artificial intelligence.

[0086] "API" stands for Application Programming Interface, and it is an interface for exchanging data and functions between different systems.

[0087] Python is a high-level programming language and a widely used tool for data processing, machine learning, web development, and more.

[0088] A "data processing library" is a programming library used for tasks such as reading, processing, cleaning, and formatting data, and includes Python's pandas and numpy.

[0089] "Cleaning" is the process of removing redundant or unnecessary information from collected data to improve its quality.

[0090] "Format unification" is the process of converting data stored in different formats into a consistent format.

[0091] A "natural language processing library" is a programming library for understanding and processing human language, and includes Python's spaCy and nltk.

[0092] "Extraction" is the process of extracting important information from collected and cleansed data.

[0093] A "relational database" is a database that manages data by establishing relationships between multiple tables, and includes MySQL (registered trademark) and PostgreSQL.

[0094] A "knowledge base" is an information database that compiles extracted important information, including project overview, progress, and contact information for team members.

[0095] "Generative artificial intelligence" refers to artificial intelligence that generates new content based on given data, and includes, for example, GPT-3 (registered trademark).

[0096] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on what to output.

[0097] A "web interface" is a user interface that allows users to access, operate, check, and modify a system through a web browser.

[0098] A "handover document" is a document that describes the progress of the work, details of the person in charge, and the next tasks to be performed, and contains the information necessary for the next person in charge to smoothly take over the work.

[0099] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, performing cleansing and formatting, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[0100] Specifically, users log into the system and specify where their work data is stored. For example, a user might set "emails are stored in the email service, meeting logs in the cloud storage service, and documents in the company's internal file server." The server automatically collects data using the APIs of each specified storage location. Specifically, it uses the Python requests library to retrieve data from APIs such as GOOGLE WORKSPACE® API, MICROSOFT® TEAMS® API, and file system APIs, and saves it to local storage (for example, an S3 bucket).

[0101] The collected data then proceeds to the data preprocessing phase. The server uses the Python pandas library to cleanse the data, removing spam emails and unnecessary information from meeting logs. Furthermore, the cleansed data is converted into a unified format. For example, meeting logs in different formats are formatted into a consistent CSV format.

[0102] Next, the server extracts important information from the pre-processed data. Using Python's natural language processing libraries (e.g., spaCy or nltk), it extracts key information such as agenda items and project progress from the email content. The extracted information is stored in a relational database such as MySQL or PostgreSQL and built as a knowledge base. The knowledge base organizes project overviews, ongoing tasks, and contact information for those in charge.

[0103] The constructed knowledge base is input into a generative artificial intelligence (e.g., OpenAI®'s GPT-3). The server uses the OpenAI API to input the knowledge base information and train the AI ​​model. Through training, the AI ​​model learns patterns and templates for generating handover documents.

[0104] The server automatically generates the handover document using a pre-trained generative artificial intelligence model. For example, use the following prompt:

[0105] Please create a handover document for Project X. Include the following information:

[0106] Project objectives

[0107] Progress

[0108] Details of the assigned members

[0109] "Next task"

[0110] The generated handover document covers the overall project overview, ongoing tasks, and contact information for each person in charge. Users log into the system and review the handover document through the web interface, making corrections as needed. This enables efficient collection and organization of business data, allowing for the creation of consistent and accurate handover documents in a short amount of time.

[0111] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0112] Step 1:

[0113] The user logs into the system and specifies the location where their work data is stored. As input, the user specifies the location for emails, meeting logs, documents, etc. As output, this storage location information is sent to the server. Specifically, the user accesses a web interface and enters the storage location information via a form.

[0114] Step 2:

[0115] The server receives data storage location information from the user and collects data from each specified storage location. The input is storage location information (e.g., Google® Workspace API, Microsoft Teams API, File System API). The output is the collected data stored in local storage. Specifically, the server uses the Python requests library to send requests to each API and saves the retrieved data to a local S3 bucket.

[0116] Step 3:

[0117] The server cleanses and standardizes the format of the collected data. The collected data is given as input. The cleansed and standardized data is generated as output. Specifically, the server uses the Python pandas library to read the data, performs spam filtering, and converts data in different formats to CSV format.

[0118] Step 4:

[0119] The server extracts important information from pre-processed data and builds a knowledge base. The input is cleansed and formatted data. The output is extracted important information and a knowledge base is built. Specifically, the server uses the Python spaCy library to perform natural language processing, extracting agenda items and project progress from email content and saving it to a MySQL database.

[0120] Step 5:

[0121] The server inputs the knowledge base into a generative artificial intelligence (AI) and performs training. The input is the information from the knowledge base. The output is a trained generative AI model. Specifically, the server uses the OpenAI API to upload the knowledge base information in JSON format and executes the AI ​​model training job.

[0122] Step 6:

[0123] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The input is a prompt (e.g., "Please create a handover document for Project X. Include the following information: project objectives, progress, details of team members, and next steps"). The output is a generated handover document. Specifically, the server sends the prompt to the generative AI model's endpoint and formats the response as an HTML or PDF document.

[0124] Step 7:

[0125] The user reviews the generated handover document and makes corrections as needed. The generated handover document is provided as input. The corrected handover document is completed as output. Specifically, the user reviews the document via a web interface and edits it using an editor to make necessary corrections.

[0126] (Application Example 1)

[0127] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0128] Currently, in many business operations, data collection, cleansing, formatting standardization, extraction of important information, knowledge base construction, and the generation of handover documents based on that data are performed manually. This is inefficient and prone to errors. Furthermore, in factory work environments, if proper handover procedures are not followed, there is a high risk of business stagnation and problems. In addition, current handover document generation systems are often complex to operate and require technical knowledge from users. To solve these problems, there is a need to provide a system that automates the process from data collection to handover document generation and makes it easy for users to use through smart devices.

[0129] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0130] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location; means for the server to cleanse and standardize the format of the collected data; means for the server to extract important information from the cleansed and standardized data and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it; means for the server to generate a handover document using the trained generative artificial intelligence model; means for the user to review and modify the generated handover document; and means for the user to specify the data storage location by voice command or code scan using a smart device. As a result, the process from data collection to handover document generation is automated, enabling the user to generate handover documents that are easy to operate and efficient.

[0131] A "user" is an individual or group that operates the system and specifies where data is stored.

[0132] "Business-related data" refers to information and records necessary for carrying out business operations, and includes emails, meeting logs, documents, etc.

[0133] "Storage location" refers to the physical or virtual location where business-related data is stored.

[0134] A "server" is a computer device that collects, cleanses, and converts data formats, builds a knowledge base, and generates handover documents.

[0135] "Means of data collection" refers to methods or techniques for retrieving data from a storage location specified by the user.

[0136] "Methods for cleansing data" refer to methods or techniques for removing unnecessary information from collected data.

[0137] "Means of standardizing data formats" refers to methods or techniques for converting data in different formats into a consistent format.

[0138] "Important information" refers to information that is particularly valuable in carrying out business operations, and examples include agenda items and project progress.

[0139] A "knowledge base" is a database where important collected information is organized and stored.

[0140] "Generative artificial intelligence" is an artificial intelligence technology used to automatically generate handover documents using a knowledge base as training data.

[0141] A "handover document" is a document used to explain the content and progress of a task to a new person in charge when the person in charge of that task changes.

[0142] A "smart device" is a device that has voice command or code scanning capabilities, and includes smart glasses and smartphones.

[0143] A system implementing this invention uses multiple hardware and software components to automate multi-stage data processing and generate accurate and efficient handover documents.

[0144] Hardware and software configuration

[0145] 1. User terminal:

[0146] Smart devices: These devices, particularly smart glasses and smartphones, incorporate voice commands and code scanning capabilities. This allows users to easily specify where data is stored. Examples include the smart glasses "Vuzix Blade" and "Google Glass®".

[0147] 2. Server:

[0148] Cloud storage APIs: Use APIs such as Google Drive API and Dropbox API to collect data from storage locations specified by the user.

[0149] Data Cleansing Module: Uses Python scripts and the Pandas library to cleanse collected data and remove unnecessary information.

[0150] Data Format Unification Module: Use Python scripts and other data conversion tools to convert data in various formats into a unified format.

[0151] Knowledge base building module: Extracts important information and stores it as a knowledge base in an SQL database or Elasticsearch®.

[0152] Generative artificial intelligence: AI models (e.g., GPT-3, BERT) that are trained based on knowledge base information using tools like TENSORFLOW® or PyTorch.

[0153] Processing flow

[0154] 1. Data collection phase:

[0155] Users use smart glasses to issue voice commands or scan QR codes (registered trademark) to specify the data storage location.

[0156] The server uses a cloud storage API to collect data from a specified storage location.

[0157] 2. Data preprocessing phase:

[0158] The server's data cleansing module removes unwanted information from the collected data. For example, it filters out spam emails.

[0159] The data format unification module converts data in different formats into a consistent format. For example, it can format various types of meeting logs into a single template.

[0160] 3. Knowledge Base Construction Phase:

[0161] The server extracts important information from pre-processed data and builds a knowledge base. This knowledge base includes project overviews, progress, and contact information for team members.

[0162] 4. AI Learning Phase:

[0163] The server-based knowledge base is input into a generative artificial intelligence system for training. TensorFlow or PyTorch are used.

[0164] 5. Handover document generation phase:

[0165] The server automatically generates the handover document using a pre-trained generative artificial intelligence model.

[0166] The user reviews the handover document generated on the smart glasses and makes corrections as needed.

[0167] Specific example

[0168] For example, consider a factory worker using this system during a shift change. The worker wears smart glasses and gives a voice command such as "Start acquiring meeting logs." The server then collects the necessary data from cloud storage, cleanses it, and standardizes the format. After that, it builds a knowledge base and generates a handover document using generative artificial intelligence. The worker reviews the handover document on the smart glasses' display and makes corrections as needed using voice input.

[0169] Example of a prompt:

[0170] "Starting to retrieve meeting logs"

[0171] Thus, a system implementing this invention significantly reduces the burden on users and enables efficient and accurate handover of tasks.

[0172] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0173] Step 1:

[0174] The user specifies the data storage location using a smart device via voice command or code scan. The input is the voice command or code recognition result. The output is information about the specified storage location.

[0175] Step 2:

[0176] The server uses a cloud storage API to collect data from a specified storage location. The input is information about the storage location specified by the user. The output is the collected raw data.

[0177] Step 3:

[0178] The server uses a data cleansing module to remove unwanted information from the collected raw data. The input is the collected raw data. The output is the cleansed data. Specifically, the server applies a spam filtering algorithm to remove spam data and filters out irrelevant information.

[0179] Step 4:

[0180] The server uses a data format unification module to convert the cleaned data into a consistent format. The input is the cleaned data. The output is the formatted data. Specifically, it formats meeting logs in different formats into a single template.

[0181] Step 5:

[0182] The server extracts important information from pre-processed data and builds a knowledge base. The input is data in a standardized format. The output is information from the knowledge base. Specifically, the server uses natural language processing algorithms to extract keywords and important topics from each data point and stores them in the database.

[0183] Step 6:

[0184] The server inputs the constructed knowledge base into a generative artificial intelligence and performs training. The input is the information from the knowledge base. The output is the trained AI model. Specifically, the server uses TensorFlow or PyTorch to learn patterns and templates for generating handover documents.

[0185] Step 7:

[0186] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. The input is information from the knowledge base and the trained AI model. The output is the generated handover document. Specifically, the AI ​​model integrates information such as project objectives, progress, and details of assigned members to create the handover document.

[0187] Step 8:

[0188] The user reviews the generated handover document on their smart device and makes corrections as needed. The input is the generated handover document. The output is the reviewed and corrected handover document. Specifically, the user reviews the content and corrects any errors or omissions using voice commands or the input functions of their smart device.

[0189] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0190] This invention relates to a system in which a user specifies a location for storing business-related data, a server collects the data from that location, cleanses and standardizes the format, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. This system incorporates an emotion engine that recognizes the user's emotions and provides appropriate feedback.

[0191] System operation description

[0192] 1. Data Collection Phase

[0193] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0194] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[0195] 2. Data preprocessing phase

[0196] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[0197] The server converts the cleansed data into a unified format. For example, it formats various types of meeting logs into a consistent template format.

[0198] 3. Knowledge Base Construction Phase

[0199] The server extracts important information from pre-processed data. For example, it extracts key topics, task progress, and important decisions from email content.

[0200] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[0201] 4. AI Learning Phase

[0202] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[0203] 5. Handover document generation phase

[0204] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes the project objectives, progress, details of the team members involved, and future tasks.

[0205] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[0206] 6. Emotional Engine Phase

[0207] When a user reviews a handover document, the emotion engine analyzes the user's voice and facial expressions to identify their emotional state. For example, if the user is confused, the emotion engine recognizes this and provides feedback.

[0208] The server readjusts or supplements the contents of the handover document based on the emotional state identified by the emotion engine. For example, if the user is feeling anxious about a certain item, detailed supplementary explanations will be added to that section.

[0209] Specific example

[0210] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0211] Email storage location: Email service A

[0212] Location where meeting logs are stored: Cloud storage service B

[0213] Document storage location: Internal file server C

[0214] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[0215] Once the generative artificial intelligence model has completed its training, it automatically generates a handover document. This document includes an overview of the project, ongoing tasks, and contact information for each person in charge. The project manager reviews this document, and the emotion engine recognizes the project manager's facial expressions and voice to analyze their emotional state. For example, if anxiety or doubt is detected, the emotion engine senses this and automatically adds supplementary explanations or details to the relevant sections of the handover document.

[0216] Based on the finalized handover document, users can smoothly complete the handover of their duties. This improves the efficiency of the handover process and reduces stress.

[0217] The following describes the processing flow.

[0218] Step 1:

[0219] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0220] Step 2:

[0221] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[0222] Step 3:

[0223] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[0224] Step 4:

[0225] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[0226] Step 5:

[0227] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[0228] Step 6:

[0229] The server builds a knowledge base based on the extracted information. The knowledge base integrates project overviews, ongoing tasks, and contact information for assigned personnel.

[0230] Step 7:

[0231] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[0232] Step 8:

[0233] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[0234] Step 9:

[0235] The user views and reviews the handover document. At this time, the emotion engine activates, analyzing the user's voice and facial expressions to determine their emotional state. For example, if the user is confused, the emotion engine recognizes this from their facial expressions and tone of voice.

[0236] Step 10:

[0237] The server receives feedback from the emotion engine and readjusts or supplements the handover document as needed. For example, it might add detailed explanations to sections where the user expressed anxiety or doubt.

[0238] Step 11:

[0239] The user reviews the final handover document and makes manual corrections as needed. Once the user is satisfied with all the content, they declare the handover document complete and provide it to the next person in charge.

[0240] Through the steps described above, the system of the present invention can streamline the user handover process and improve the quality of the handover.

[0241] (Example 2)

[0242] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0243] During the handover of work responsibilities, organizing data and extracting information is time-consuming and laborious, making efficient handover difficult. Furthermore, the lack of feedback functions that consider user emotions prevents stress reduction. As a result, deficiencies in handover content and insufficient communication often occur.

[0244] In Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, means for the sentiment analysis engine to analyze the user's emotions and provide feedback, and means for the server to readjust or supplement the contents of the handover document based on the results of the sentiment analysis engine. As a result, a series of processes from automatic data collection, cleansing, and information extraction to the generation of the handover document and feedback based on the user's emotions are automated, enabling an efficient and user-friendly handover.

[0245] A "user" is a person or entity whose role is to access the system, specify the location where business-related data is stored, and review and modify the generated handover documents.

[0246] A "server" is a computer system that collects data from a storage location specified by the user, cleanses and standardizes the format of the collected data, extracts important information, and builds a knowledge base.

[0247] "Storage location" refers to a physical or cloud storage system where business-related data is stored.

[0248] "Cleansing" is the process of removing unnecessary information and noise from collected data.

[0249] "Format unification" is the process of converting data stored in different formats into a consistent, standardized format.

[0250] "Important information" refers to information that is particularly valuable in business operations and is necessary for building knowledge bases and generating handover documents.

[0251] A "knowledge base" is a database that systematically organizes and stores important extracted information.

[0252] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information or documents from given data.

[0253] "Training" is the process by which a generative artificial intelligence learns from data in a given knowledge base and builds a model for generating handover documents.

[0254] A "handover document" is a document used for handing over work responsibilities, including the project's objectives, progress, details of the person in charge, and future tasks.

[0255] An "emotion analysis engine" is a system that analyzes a user's voice and facial expressions to identify their emotional state.

[0256] "Feedback" refers to instructions and supplementary information provided based on the user's emotional state identified by the emotion analysis engine.

[0257] This invention is a system in which the user specifies a location for storing business-related data, and a server collects, cleanses, and standardizes the data from that location, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. An emotion analysis engine is also incorporated to recognize the user's emotions and provide appropriate feedback.

[0258] System Configuration

[0259] Data collection

[0260] The user logs into the system and specifies where work-related data is stored. For example, a user can set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company file server C." The server retrieves the data from the specified storage locations using an API.

[0261] Data preprocessing

[0262] The server cleanses the acquired data. This phase involves removing spam emails and unnecessary information from meeting logs. In particular, a text analysis engine is used for data cleansing. Next, the server converts the cleansed data into a unified format, shaping various data formats into a consistent template.

[0263] Building a knowledge base

[0264] The server extracts key information from pre-processed data and builds a knowledge base. This extracted information includes, for example, the main topics of emails, task progress, and important decisions. The resulting knowledge base includes project summaries, ongoing tasks, and contact information for those responsible.

[0265] AI learning

[0266] The server inputs data from the knowledge base into a generative artificial intelligence (AI) and performs training. The generative AI used for training utilizes various machine learning libraries (e.g., TensorFlow, PyTorch) to learn patterns and templates for generating handover documents.

[0267] Handover document generation

[0268] Using a completed generative artificial intelligence model, the server automatically generates a handover document. The generated document details the project's objectives, progress, team members, and future tasks. The user reviews this generated document and makes any necessary modifications.

[0269] Emotion analysis

[0270] When a user reviews a handover document, an emotion analysis engine analyzes the user's voice and facial expressions to identify their emotional state. Emotion analysis is performed in real time; for example, if the user is confused, the emotion analysis engine recognizes this, and the server automatically adds detailed supplementary explanations to the relevant section.

[0271] Specific example

[0272] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0273] Email storage location: Email service A

[0274] Location where meeting logs are stored: Cloud storage service B

[0275] Document storage location: Internal file server C

[0276] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. It then extracts important information from the cleansed data to build a knowledge base. This knowledge base is input into a generative artificial intelligence (AI) model for training. The improved AI model automatically generates a handover document. The project manager reviews this document, and an emotion analysis engine recognizes the project manager's emotions through facial expressions and voice, providing supplementary explanations where necessary.

[0277] Example of prompt text

[0278] "Please create a detailed handover document for the project to be handed over next. Include the project objectives, progress status, details of the responsible persons, and future tasks. And please provide appropriate feedback on the user's emotional state."

[0279] The flow of the specific process in Example 2 will be described using FIG. 13.

[0280] Step 1:

[0281] The user logs in to the system and specifies the storage location for business-related data.

[0282] Input: The user's login information and the specification of the data storage location (e.g., mail service, cloud storage, in-house file server).

[0283] Specific operation: The user enters the storage location of mails as "Mail Service A", the storage location of meeting logs as "Cloud Storage B", and the storage location of documents as "In-house File Server C" in the input fields of the interface.

[0284] Output: The specified information of the storage location is sent to the server and recorded.

[0285] Step 2:

[0286] The server automatically collects data from the storage location specified by the user.

[0287] Input: Information on the storage location specified by the user.

[0288] Specific operation: The server calls the API of the storage location to obtain mail data from Mail Service A, meeting log data from Cloud Storage B, and document data from In-house File Server C.

[0289] Output: The collected raw data is stored on the server.

[0290] Step 3:

[0291] The server cleanses the collected data.

[0292] Input: The collected raw data.

[0293] Specific operation: The server uses a text analysis engine to remove spam emails and unnecessary information from meeting logs. For example, it uses an NLP model to perform spam filtering and remove unnecessary words such as "um" and "uh" from meeting logs.

[0294] Output: Cleaned, clean data.

[0295] Step 4:

[0296] The server converts the cleansed data into a unified format.

[0297] Input: Cleansed, clean data.

[0298] Specific operation: The server executes a script to convert various data formats into a consistent template format. For example, it formats meeting logs in different formats into a single unified template.

[0299] Output: Data in a standardized format.

[0300] Step 5:

[0301] The server extracts important information from the pre-processed data.

[0302] Input: Data in a standardized format.

[0303] Specific operation: The server applies an information extraction algorithm to extract the main topics of emails, the progress of tasks, and important decision-making matters. For example, extract "next action points" and "decisions" from meeting logs.

[0304] Output: The extracted important information.

[0305] Step 6:

[0306] The server constructs a knowledge base based on the extracted information.

[0307] Input: The extracted important information.

[0308] Specific operation: The server sorts out the information extracted into the database and stores the project overview, ongoing tasks, and contact information of the person in charge for each project.

[0309] Output: The constructed knowledge base.

[0310] Step 7:

[0311] The server inputs the knowledge base into generative artificial intelligence for training.

[0312] Input: The knowledge base.

[0313] Specific operation: The server supplies the information of the knowledge base to the generative artificial intelligence model for training. Use machine learning libraries such as TensorFlow or PyTorch to train the model.

[0314] Output: The trained generative artificial intelligence model.

[0315] Step 8:

[0316] The server automatically generates a handover document using the trained generative artificial intelligence model.

[0317] Input: Trained generative artificial intelligence models and knowledge base data.

[0318] Specific operation: The server generates a handover document using a generative artificial intelligence model. The handover document will include the project's objectives, progress, details of the team members in charge, and future tasks.

[0319] Output: Automated handover document.

[0320] Step 9:

[0321] The user reviews the generated handover document and makes corrections as needed.

[0322] Input: Automated handover document.

[0323] Specific operation: The user reviews the contents of the handover document on the interface and edits any unclear parts or sections that require additional explanation.

[0324] Output: Revised final version of the handover document.

[0325] Step 10:

[0326] When a user reviews a handover document, an emotion analysis engine analyzes the user's emotions and provides feedback.

[0327] Input: Voice and facial expression data of the user reviewing the handover document.

[0328] Specific operation: The emotion analysis engine analyzes the user's emotions in real time, identifying feelings of confusion, anxiety, etc. Based on this, it provides feedback.

[0329] Output: User sentiment analysis results and provided feedback.

[0330] Step 11:

[0331] The server readjusts or supplements the contents of the handover document based on the results of the emotion analysis engine.

[0332] Input: Results from the emotion analysis engine.

[0333] Specific operation: The server adjusts the relevant sections of the handover document based on the user's feelings and adds necessary supplementary explanations. For example, it adds detailed explanations to sections where the user felt uneasy.

[0334] Output: Adjusted and supplemented handover documents.

[0335] (Application Example 2)

[0336] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0337] In today's work environment, information management and handover processes are becoming increasingly complex. In particular, information regarding the maintenance and operation of factory robots is diverse, making it difficult for new engineers to smoothly take over the work. Typical handover documents are prone to missing or inconsistent information, reducing the efficiency of training. Furthermore, there is a lack of means to alleviate the stress and anxiety engineers experience while reviewing handover documents. It is necessary to address these problems.

[0338] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, and sentiment analysis means to analyze the user's emotions during the review of the handover document and provide appropriate feedback. This makes it possible to perform an efficient and detailed handover while reducing the anxiety and confusion felt by the technician.

[0339] A "user" is an individual or legal entity that uses the system to specify, verify, or modify business data.

[0340] "Business data" refers to information such as emails, meeting logs, and documents necessary for the progress of work.

[0341] "Storage location" refers to the storage medium, such as a server, cloud storage, or local storage, where business data is stored.

[0342] A "server" is a central processing unit that collects, cleanses, and formats data, builds a knowledge base, trains generative artificial intelligence, and generates handover documents.

[0343] "Data collection" refers to retrieving specified business data from a storage location.

[0344] "Cleaning" is the process of removing unnecessary information from collected data to improve its quality.

[0345] "Format unification" is the process of standardizing data stored in different formats into a consistent format.

[0346] A "knowledge base" is a database that stores important information extracted from cleansed and formatted data.

[0347] "Generative artificial intelligence" refers to an artificial intelligence model that learns from a knowledge base and has the function of generating specific outputs (for example, handover documents).

[0348] A "handover document" is a document used for handing over work, which includes details of the work, progress, and contact information of the person in charge.

[0349] "Emotional analysis" is a process that provides appropriate feedback by analyzing the user's emotional state.

[0350] Basic System Configuration

[0351] A system for implementing this invention includes the following main elements:

[0352] 1. Method for specifying data storage location: The user logs into the system and specifies the storage location for business data (e.g., email service, cloud storage, local file server).

[0353] 2. Data Collection Method: The server collects data from designated storage locations. It efficiently retrieves data using the API of each storage location.

[0354] 3. Data cleansing and formatting unification means: The server cleanses the collected data (e.g., removes unnecessary information) and converts the data into a consistent format.

[0355] 4. Knowledge Base Construction Method: The server extracts important information from cleansed and formatted data to build a knowledge base. The knowledge base includes project overviews, ongoing tasks, and contact information for assigned personnel.

[0356] 5. Generative AI Training Method: The server inputs the knowledge base into the generative AI and performs training.

[0357] 6. Handover document generation method: The server automatically generates the handover document using a trained generative artificial intelligence model.

[0358] 7. User Verification and Correction Method: The user will review the generated handover document and make corrections as necessary.

[0359] 8. Emotion Analysis Method: Analyzes the user's facial expressions and voice while reviewing the handover document, and provides feedback based on the user's emotional state.

[0360] Hardware and software configuration

[0361] Hardware: Servers, user terminals (PCs or mobile devices), sensors (cameras and microphones).

[0362] Software: API communication software (REST API, etc.), text processing software (Python libraries), data cleansing software (Pandas), generative artificial intelligence (OpenAI's GPT, etc.), sentiment analysis software (EmotionAnalyzer).

[0363] Specific example

[0364] For example, consider using this invention in a system to support the maintenance and operation of robots used in a factory. When a new technician takes over a task, this system is used to automatically collect data such as emails, maintenance logs, and work procedures left by the previous technician. The collected data is cleansed and standardized in format. A knowledge base is built, and this information is used to train generative artificial intelligence. The technician reviews the automatically generated handover document, and if sentiment analysis indicates confusion or anxiety, the system provides appropriate feedback.

[0365] Example of a prompt

[0366] Design a system for new technicians to take over factory robot maintenance tasks. This system will extract key information from the predecessor's emails, maintenance logs, and work procedures, and automatically generate a handover document. It will also include a function to analyze the technician's emotions and add detailed supplementary explanations if they are feeling anxious or confused.

[0367] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0368] Step 1:

[0369] The user logs into the system and specifies the location where work-related data is stored. In this step, the user can specify an email service, cloud storage, local file server, etc. The input is the user's specified storage location information, and the output is a list of the specified storage locations.

[0370] Step 2:

[0371] The server collects data from a storage location specified by the user. It accesses the storage location and retrieves the data using APIs or the file system. The input is a list of storage locations, and the output is a collection of the collected original data.

[0372] Step 3:

[0373] The server cleanses the collected data. It performs text processing to remove spam emails and unwanted information from meeting logs. The input is the original collected data, and the output is the cleansed data.

[0374] Step 4:

[0375] The server cleanses the data and standardizes its format. It converts data in different formats into a unified format. The input is cleansed data, and the output is data with a unified format.

[0376] Step 5:

[0377] The server extracts important information from standardized data and builds a knowledge base. For example, it extracts key topics, task progress, and important decisions from email content. The input is standardized data, and the output is a knowledge base.

[0378] Step 6:

[0379] The server inputs the knowledge base into a generative artificial intelligence (AI) model and trains it. It learns patterns and templates for generating handover documents from the information in the knowledge base. The input is the knowledge base, and the output is the trained generative AI model.

[0380] Step 7:

[0381] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes details of the task, progress, and contact information of the person in charge. The input is the pre-trained generative artificial intelligence model, and the output is the handover document.

[0382] Step 8:

[0383] The user reviews the generated handover document and makes corrections as needed. They access the handover document using a terminal and modify the necessary parts. The input is the handover document, and the output is the corrected handover document.

[0384] Step 9:

[0385] While the server is verifying the handover document, it runs an emotion analysis engine to analyze the user's emotions. It analyzes voice and facial expression data to identify the user's emotional state. The input is the user's voice and facial expression data, and the output is the user's emotional state.

[0386] Step 10:

[0387] The server readjusts or supplements the handover document based on the user's emotional state identified through sentiment analysis. Detailed explanations are added to sections where the user felt confused or anxious. The input is the user's emotional state and the handover document; the output is the supplemented handover document.

[0388] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0389] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0390] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0391] [Second Embodiment]

[0392] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0393] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0394] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0395] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0396] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0397] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0398] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0399] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0400] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0401] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0402] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0403] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0404] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, cleansing and standardizing the data format, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence system, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[0405] System operation description

[0406] 1. Data Collection Phase

[0407] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0408] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[0409] 2. Data preprocessing phase

[0410] The server cleanses the collected data. For example, the server removes spam emails and deletes unnecessary information from meeting logs.

[0411] The server converts the cleansed data into a unified format. For example, it formats meeting logs in various formats into a consistent template.

[0412] 3. Knowledge Base Construction Phase

[0413] The server extracts important information from pre-processed data. For example, it extracts key topics and project progress from email content.

[0414] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[0415] 4. AI Learning Phase

[0416] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[0417] 5. Handover document generation phase

[0418] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. For example, the AI ​​model might create a "Project X Personnel Handover Document" which would include the project's objectives, progress, details of the team members, and the next tasks to be performed.

[0419] The user reviews the generated handover document and makes corrections as needed. For example, if the user thinks "this section needs additional explanation," they manually edit that section.

[0420] Specific example

[0421] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0422] Email storage location: Email service A

[0423] Location where meeting logs are stored: Cloud storage service B

[0424] Document storage location: Internal file server C

[0425] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[0426] Once the generative AI model has completed its training, it automatically generates a handover document. This document includes a comprehensive overview of the project, ongoing tasks, and contact information for each person involved. The project manager reviews this document and makes revisions as needed. As a result, it becomes possible to create accurate and consistent handover documents, significantly reducing time and effort.

[0427] The following describes the processing flow.

[0428] Step 1:

[0429] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0430] Step 2:

[0431] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[0432] Step 3:

[0433] The server cleanses the collected data. For example, it filters spam from email data and performs text processing to remove unnecessary information from meeting logs.

[0434] Step 4:

[0435] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[0436] Step 5:

[0437] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[0438] Step 6:

[0439] The server builds a knowledge base based on the extracted information. This knowledge base integrates project overviews, progress, and contact information for each person in charge.

[0440] Step 7:

[0441] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[0442] Step 8:

[0443] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[0444] Step 9:

[0445] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[0446] (Example 1)

[0447] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0448] In today's business environment, managing and transferring business data is a critical challenge. However, because various data are stored in scattered locations, data collection and organization are cumbersome and inefficient. Furthermore, creating handover documents is time-consuming and laborious, and ensuring the accuracy and consistency of the information is difficult. To solve these problems, efficient collection and standardization of business data formats, extraction of key information and construction of a knowledge base, and automated generation of handover documents using AI are necessary.

[0449] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0450] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location using an API; means for the server to cleanse and format the collected data using a Python data processing library; means for the server to extract important information from the cleansed and formatted data using a natural language processing library, store it in a relational database, and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it using an API; means for the server to generate a handover document based on prompt statements using the trained generative artificial intelligence model; and means for the user to review and modify the generated handover document through a web interface. This enables efficient collection and organization of business data, and allows for the creation of consistent and accurate handover documents in a short amount of time.

[0451] A "user" is an entity that uses the system to specify the storage location for business-related data and to review and modify the generated handover documents.

[0452] A "server" is a computing device that collects data based on user instructions, cleanses and standardizes the format, extracts important information to build a knowledge base, and generates handover documents using generative artificial intelligence.

[0453] "API" stands for Application Programming Interface, and it is an interface for exchanging data and functions between different systems.

[0454] Python is a high-level programming language and a widely used tool for data processing, machine learning, web development, and more.

[0455] A "data processing library" is a programming library used for tasks such as reading, processing, cleaning, and formatting data, and includes Python's pandas and numpy.

[0456] "Cleaning" is the process of removing redundant or unnecessary information from collected data to improve its quality.

[0457] "Format unification" is the process of converting data stored in different formats into a consistent format.

[0458] A "natural language processing library" is a programming library for understanding and processing human language, and includes Python's spaCy and nltk.

[0459] "Extraction" is the process of extracting important information from collected and cleansed data.

[0460] A "relational database" is a database that manages data by establishing relationships between multiple tables, and includes MySQL and PostgreSQL.

[0461] A "knowledge base" is an information database that compiles extracted important information, including project overview, progress, and contact information for team members.

[0462] "Generative artificial intelligence" refers to artificial intelligence that generates new content based on given data, and includes, for example, GPT-3.

[0463] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on what to output.

[0464] A "web interface" is a user interface that allows users to access, operate, check, and modify a system through a web browser.

[0465] A "handover document" is a document that describes the progress of the work, details of the person in charge, and the next tasks to be performed, and contains the information necessary for the next person in charge to smoothly take over the work.

[0466] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, performing cleansing and formatting, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[0467] Specifically, users log into the system and specify where their work data is stored. For example, a user might set "emails are stored in the email service, meeting logs in the cloud storage service, and documents in the company's internal file server." The server then automatically collects the data using the APIs of each specified storage location. Specifically, it uses the Python requests library to retrieve data from APIs such as Google Workspace API, Microsoft Teams API, and File System API, and saves it to local storage (for example, an S3 bucket).

[0468] The collected data then proceeds to the data preprocessing phase. The server uses the Python pandas library to cleanse the data, removing spam emails and unnecessary information from meeting logs. Furthermore, the cleansed data is converted into a unified format. For example, meeting logs in different formats are formatted into a consistent CSV format.

[0469] Next, the server extracts important information from the pre-processed data. Using Python's natural language processing libraries (e.g., spaCy or nltk), it extracts key information such as agenda items and project progress from the email content. The extracted information is stored in a relational database such as MySQL or PostgreSQL and built as a knowledge base. The knowledge base organizes project overviews, ongoing tasks, and contact information for those in charge.

[0470] The constructed knowledge base is input into a generative artificial intelligence (e.g., OpenAI's GPT-3). The server uses the OpenAI API to input the knowledge base information and train the AI ​​model. Through training, the AI ​​model learns patterns and templates for generating handover documents.

[0471] The server automatically generates the handover document using a pre-trained generative artificial intelligence model. For example, use the following prompt:

[0472] Please create a handover document for Project X. Include the following information:

[0473] Project objectives

[0474] Progress

[0475] Details of the assigned members

[0476] "Next task"

[0477] The generated handover document covers the overall project overview, ongoing tasks, and contact information for each person in charge. Users log into the system and review the handover document through the web interface, making corrections as needed. This enables efficient collection and organization of business data, allowing for the creation of consistent and accurate handover documents in a short amount of time.

[0478] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0479] Step 1:

[0480] The user logs into the system and specifies the location where their work data is stored. As input, the user specifies the location for emails, meeting logs, documents, etc. As output, this storage location information is sent to the server. Specifically, the user accesses a web interface and enters the storage location information via a form.

[0481] Step 2:

[0482] The server receives data storage location information from the user and collects data from each specified storage location. The input is storage location information (e.g., Google Workspace API, Microsoft Teams API, File System API). The output is the collected data stored in local storage. Specifically, the server uses the Python requests library to send requests to each API and saves the retrieved data to a local S3 bucket.

[0483] Step 3:

[0484] The server cleanses and standardizes the format of the collected data. The collected data is given as input. The cleansed and standardized data is generated as output. Specifically, the server uses the Python pandas library to read the data, performs spam filtering, and converts data in different formats to CSV format.

[0485] Step 4:

[0486] The server extracts important information from pre-processed data and builds a knowledge base. The input is cleansed and formatted data. The output is extracted important information and a knowledge base is built. Specifically, the server uses the Python spaCy library to perform natural language processing, extracting agenda items and project progress from email content and saving it to a MySQL database.

[0487] Step 5:

[0488] The server inputs the knowledge base into a generative artificial intelligence (AI) and performs training. The input is the information from the knowledge base. The output is a trained generative AI model. Specifically, the server uses the OpenAI API to upload the knowledge base information in JSON format and executes the AI ​​model training job.

[0489] Step 6:

[0490] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The input is a prompt (e.g., "Please create a handover document for Project X. Include the following information: project objectives, progress, details of team members, and next steps"). The output is a generated handover document. Specifically, the server sends the prompt to the generative AI model's endpoint and formats the response as an HTML or PDF document.

[0491] Step 7:

[0492] The user reviews the generated handover document and makes corrections as needed. The generated handover document is provided as input. The corrected handover document is completed as output. Specifically, the user reviews the document via a web interface and edits it using an editor to make necessary corrections.

[0493] (Application Example 1)

[0494] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0495] Currently, in many business operations, data collection, cleansing, formatting standardization, extraction of important information, knowledge base construction, and the generation of handover documents based on that data are performed manually. This is inefficient and prone to errors. Furthermore, in factory work environments, if proper handover procedures are not followed, there is a high risk of business stagnation and problems. In addition, current handover document generation systems are often complex to operate and require technical knowledge from users. To solve these problems, there is a need to provide a system that automates the process from data collection to handover document generation and makes it easy for users to use through smart devices.

[0496] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0497] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location; means for the server to cleanse and standardize the format of the collected data; means for the server to extract important information from the cleansed and standardized data and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it; means for the server to generate a handover document using the trained generative artificial intelligence model; means for the user to review and modify the generated handover document; and means for the user to specify the data storage location by voice command or code scan using a smart device. As a result, the process from data collection to handover document generation is automated, enabling the user to generate handover documents that are easy to operate and efficient.

[0498] A "user" is an individual or group that operates the system and specifies where data is stored.

[0499] "Business-related data" refers to information and records necessary for carrying out business operations, and includes emails, meeting logs, documents, etc.

[0500] "Storage location" refers to the physical or virtual location where business-related data is stored.

[0501] A "server" is a computer device that collects, cleanses, and converts data formats, builds a knowledge base, and generates handover documents.

[0502] "Means of data collection" refers to methods or techniques for retrieving data from a storage location specified by the user.

[0503] "Methods for cleansing data" refer to methods or techniques for removing unnecessary information from collected data.

[0504] "Means of standardizing data formats" refers to methods or techniques for converting data in different formats into a consistent format.

[0505] "Important information" refers to information that is particularly valuable in carrying out business operations, and examples include agenda items and project progress.

[0506] A "knowledge base" is a database where important collected information is organized and stored.

[0507] "Generative artificial intelligence" is an artificial intelligence technology used to automatically generate handover documents using a knowledge base as training data.

[0508] A "handover document" is a document used to explain the content and progress of a task to a new person in charge when the person in charge of that task changes.

[0509] A "smart device" is a device that has voice command or code scanning capabilities, and includes smart glasses and smartphones.

[0510] A system implementing this invention uses multiple hardware and software components to automate multi-stage data processing and generate accurate and efficient handover documents.

[0511] Hardware and software configuration

[0512] 1. User terminal:

[0513] Smart devices: These devices, particularly smart glasses and smartphones, incorporate voice commands and code scanning capabilities. This allows users to easily specify where data is stored. Examples include the Vuzix Blade and Google Glass smart glasses.

[0514] 2. Server:

[0515] Cloud storage APIs: Use APIs such as Google Drive API and Dropbox API to collect data from storage locations specified by the user.

[0516] Data Cleansing Module: Uses Python scripts and the Pandas library to cleanse collected data and remove unnecessary information.

[0517] Data Format Unification Module: Use Python scripts and other data conversion tools to convert data in various formats into a unified format.

[0518] Knowledge base building module: Extracts important information and stores it as a knowledge base in an SQL database or Elasticsearch.

[0519] Generative artificial intelligence: AI models that are trained using knowledge base information, such as GPT-3 and BERT, using TensorFlow or PyTorch.

[0520] Processing flow

[0521] 1. Data collection phase:

[0522] Users use smart glasses to issue voice commands or scan QR codes to specify the data storage location.

[0523] The server uses a cloud storage API to collect data from a specified storage location.

[0524] 2. Data preprocessing phase:

[0525] The server's data cleansing module removes unwanted information from the collected data. For example, it filters out spam emails.

[0526] The data format unification module converts data in different formats into a consistent format. For example, it can format various types of meeting logs into a single template.

[0527] 3. Knowledge Base Construction Phase:

[0528] The server extracts important information from pre-processed data and builds a knowledge base. This knowledge base includes project overviews, progress, and contact information for team members.

[0529] 4. AI Learning Phase:

[0530] The server-based knowledge base is input into a generative artificial intelligence system for training. TensorFlow or PyTorch are used.

[0531] 5. Handover document generation phase:

[0532] The server automatically generates the handover document using a pre-trained generative artificial intelligence model.

[0533] The user reviews the handover document generated on the smart glasses and makes corrections as needed.

[0534] Specific example

[0535] For example, consider a factory worker using this system during a shift change. The worker wears smart glasses and gives a voice command such as "Start acquiring meeting logs." The server then collects the necessary data from cloud storage, cleanses it, and standardizes the format. After that, it builds a knowledge base and generates a handover document using generative artificial intelligence. The worker reviews the handover document on the smart glasses' display and makes corrections as needed using voice input.

[0536] Example of a prompt:

[0537] "Starting to retrieve meeting logs"

[0538] Thus, a system implementing this invention significantly reduces the burden on users and enables efficient and accurate handover of tasks.

[0539] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0540] Step 1:

[0541] The user specifies the data storage location using a smart device via voice command or code scan. The input is the voice command or code recognition result. The output is information about the specified storage location.

[0542] Step 2:

[0543] The server uses a cloud storage API to collect data from a specified storage location. The input is information about the storage location specified by the user. The output is the collected raw data.

[0544] Step 3:

[0545] The server uses a data cleansing module to remove unwanted information from the collected raw data. The input is the collected raw data. The output is the cleansed data. Specifically, the server applies a spam filtering algorithm to remove spam data and filters out irrelevant information.

[0546] Step 4:

[0547] The server uses a data format unification module to convert the cleaned data into a consistent format. The input is the cleaned data. The output is the formatted data. Specifically, it formats meeting logs in different formats into a single template.

[0548] Step 5:

[0549] The server extracts important information from pre-processed data and builds a knowledge base. The input is data in a standardized format. The output is information from the knowledge base. Specifically, the server uses natural language processing algorithms to extract keywords and important topics from each data point and stores them in the database.

[0550] Step 6:

[0551] The server inputs the constructed knowledge base into a generative artificial intelligence and performs training. The input is the information from the knowledge base. The output is the trained AI model. Specifically, the server uses TensorFlow or PyTorch to learn patterns and templates for generating handover documents.

[0552] Step 7:

[0553] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. The input is information from the knowledge base and the trained AI model. The output is the generated handover document. Specifically, the AI ​​model integrates information such as project objectives, progress, and details of assigned members to create the handover document.

[0554] Step 8:

[0555] The user reviews the generated handover document on their smart device and makes corrections as needed. The input is the generated handover document. The output is the reviewed and corrected handover document. Specifically, the user reviews the content and corrects any errors or omissions using voice commands or the input functions of their smart device.

[0556] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0557] This invention relates to a system in which a user specifies a location for storing business-related data, a server collects the data from that location, cleanses and standardizes the format, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. This system incorporates an emotion engine that recognizes the user's emotions and provides appropriate feedback.

[0558] System operation description

[0559] 1. Data Collection Phase

[0560] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0561] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[0562] 2. Data preprocessing phase

[0563] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[0564] The server converts the cleansed data into a unified format. For example, it formats various types of meeting logs into a consistent template format.

[0565] 3. Knowledge Base Construction Phase

[0566] The server extracts important information from pre-processed data. For example, it extracts key topics, task progress, and important decisions from email content.

[0567] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[0568] 4. AI Learning Phase

[0569] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[0570] 5. Handover document generation phase

[0571] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes the project objectives, progress, details of the team members involved, and future tasks.

[0572] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[0573] 6. Emotional Engine Phase

[0574] When a user reviews a handover document, the emotion engine analyzes the user's voice and facial expressions to identify their emotional state. For example, if the user is confused, the emotion engine recognizes this and provides feedback.

[0575] The server readjusts or supplements the contents of the handover document based on the emotional state identified by the emotion engine. For example, if the user is feeling anxious about a certain item, detailed supplementary explanations will be added to that section.

[0576] Specific example

[0577] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0578] Email storage location: Email service A

[0579] Location where meeting logs are stored: Cloud storage service B

[0580] Document storage location: Internal file server C

[0581] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[0582] Once the generative artificial intelligence model has completed its training, it automatically generates a handover document. This document includes an overview of the project, ongoing tasks, and contact information for each person in charge. The project manager reviews this document, and the emotion engine recognizes the project manager's facial expressions and voice to analyze their emotional state. For example, if anxiety or doubt is detected, the emotion engine senses this and automatically adds supplementary explanations or details to the relevant sections of the handover document.

[0583] Based on the finalized handover document, users can smoothly complete the handover of their duties. This improves the efficiency of the handover process and reduces stress.

[0584] The following describes the processing flow.

[0585] Step 1:

[0586] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0587] Step 2:

[0588] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[0589] Step 3:

[0590] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[0591] Step 4:

[0592] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[0593] Step 5:

[0594] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[0595] Step 6:

[0596] The server builds a knowledge base based on the extracted information. The knowledge base integrates project overviews, ongoing tasks, and contact information for assigned personnel.

[0597] Step 7:

[0598] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[0599] Step 8:

[0600] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[0601] Step 9:

[0602] The user views and reviews the handover document. At this time, the emotion engine activates, analyzing the user's voice and facial expressions to determine their emotional state. For example, if the user is confused, the emotion engine recognizes this from their facial expressions and tone of voice.

[0603] Step 10:

[0604] The server receives feedback from the emotion engine and readjusts or supplements the handover document as needed. For example, it might add detailed explanations to sections where the user expressed anxiety or doubt.

[0605] Step 11:

[0606] The user reviews the final handover document and makes manual corrections as needed. Once the user is satisfied with all the content, they declare the handover document complete and provide it to the next person in charge.

[0607] Through the steps described above, the system of the present invention can streamline the user handover process and improve the quality of the handover.

[0608] (Example 2)

[0609] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0610] During the handover of work responsibilities, organizing data and extracting information is time-consuming and laborious, making efficient handover difficult. Furthermore, the lack of feedback functions that consider user emotions prevents stress reduction. As a result, deficiencies in handover content and insufficient communication often occur.

[0611] In Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, means for the sentiment analysis engine to analyze the user's emotions and provide feedback, and means for the server to readjust or supplement the contents of the handover document based on the results of the sentiment analysis engine. As a result, a series of processes from automatic data collection, cleansing, and information extraction to the generation of the handover document and feedback based on the user's emotions are automated, enabling an efficient and user-friendly handover.

[0612] A "user" is a person or entity whose role is to access the system, specify the location where business-related data is stored, and review and modify the generated handover documents.

[0613] A "server" is a computer system that collects data from a storage location specified by the user, cleanses and standardizes the format of the collected data, extracts important information, and builds a knowledge base.

[0614] "Storage location" refers to a physical or cloud storage system where business-related data is stored.

[0615] "Cleansing" is the process of removing unnecessary information and noise from collected data.

[0616] "Format unification" is the process of converting data stored in different formats into a consistent, standardized format.

[0617] "Important information" refers to information that is particularly valuable in business operations and is necessary for building knowledge bases and generating handover documents.

[0618] A "knowledge base" is a database that systematically organizes and stores important extracted information.

[0619] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information or documents from given data.

[0620] "Training" is the process by which a generative artificial intelligence learns from data in a given knowledge base and builds a model for generating handover documents.

[0621] A "handover document" is a document used for handing over work responsibilities, including the project's objectives, progress, details of the person in charge, and future tasks.

[0622] An "emotion analysis engine" is a system that analyzes a user's voice and facial expressions to identify their emotional state.

[0623] "Feedback" refers to instructions and supplementary information provided based on the user's emotional state identified by the emotion analysis engine.

[0624] This invention is a system in which the user specifies a location for storing business-related data, and a server collects, cleanses, and standardizes the data from that location, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. An emotion analysis engine is also incorporated to recognize the user's emotions and provide appropriate feedback.

[0625] System Configuration

[0626] Data collection

[0627] The user logs into the system and specifies where work-related data is stored. For example, a user can set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company file server C." The server retrieves the data from the specified storage locations using an API.

[0628] Data preprocessing

[0629] The server cleanses the acquired data. This phase involves removing spam emails and unnecessary information from meeting logs. In particular, a text analysis engine is used for data cleansing. Next, the server converts the cleansed data into a unified format, shaping various data formats into a consistent template.

[0630] Building a knowledge base

[0631] The server extracts key information from pre-processed data and builds a knowledge base. This extracted information includes, for example, the main topics of emails, task progress, and important decisions. The resulting knowledge base includes project summaries, ongoing tasks, and contact information for those responsible.

[0632] AI learning

[0633] The server inputs data from the knowledge base into a generative artificial intelligence (AI) and performs training. The generative AI used for training utilizes various machine learning libraries (e.g., TensorFlow, PyTorch) to learn patterns and templates for generating handover documents.

[0634] Handover document generation

[0635] Using a completed generative artificial intelligence model, the server automatically generates a handover document. The generated document details the project's objectives, progress, team members, and future tasks. The user reviews this generated document and makes any necessary modifications.

[0636] Emotion analysis

[0637] When a user reviews a handover document, an emotion analysis engine analyzes the user's voice and facial expressions to identify their emotional state. Emotion analysis is performed in real time; for example, if the user is confused, the emotion analysis engine recognizes this, and the server automatically adds detailed supplementary explanations to the relevant section.

[0638] Specific example

[0639] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0640] Email storage location: Email service A

[0641] Location where meeting logs are stored: Cloud storage service B

[0642] Document storage location: Internal file server C

[0643] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. It then extracts important information from the cleansed data to build a knowledge base. This knowledge base is input into a generative artificial intelligence (AI) model for training. The improved AI model automatically generates a handover document. The project manager reviews this document, and an emotion analysis engine recognizes the project manager's emotions through facial expressions and voice, providing supplementary explanations where necessary.

[0644] Example of a prompt

[0645] "Please create a detailed handover document for the next project you will be taking over. Include the project's objectives, progress, details of the person in charge, and future tasks. Also, ensure that appropriate feedback is provided regarding the user's emotional state."

[0646] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0647] Step 1:

[0648] The user logs into the system and specifies the location where work-related data will be stored.

[0649] Input: Specify the user's login information and data storage location (e.g., email service, cloud storage, internal file server).

[0650] Specific operation: The user enters "Email Service A" as the email storage location, "Cloud Storage B" as the meeting log storage location, and "Internal File Server C" as the document storage location into the interface's input fields.

[0651] Output: The specified save location information is sent to the server and recorded.

[0652] Step 2:

[0653] The server automatically collects data from a storage location specified by the user.

[0654] Input: Information about the save location specified by the user.

[0655] Specific operation: The server calls the storage location API to retrieve email data from email service A, meeting log data from cloud storage B, and document data from the internal file server C.

[0656] Output: The collected raw data is stored on the server.

[0657] Step 3:

[0658] The server cleanses the collected data.

[0659] Input: The collected raw data.

[0660] Specific operation: The server uses a text analysis engine to remove spam emails and unnecessary information from meeting logs. For example, it uses an NLP model to perform spam filtering and remove unnecessary words such as "um" and "uh" from meeting logs.

[0661] Output: Cleaned, clean data.

[0662] Step 4:

[0663] The server converts the cleansed data into a unified format.

[0664] Input: Cleansed, clean data.

[0665] Specific operation: The server executes a script to convert various data formats into a consistent template format. For example, it formats meeting logs in different formats into a single unified template.

[0666] Output: Data in a standardized format.

[0667] Step 5:

[0668] The server extracts important information from the pre-processed data.

[0669] Input: Data in a standardized format.

[0670] Specific operation: The server applies an information extraction algorithm to extract the main topics, task progress, and important decisions from emails. For example, it might extract "next action points" or "decisions" from meeting logs.

[0671] Output: Extracted key information.

[0672] Step 6:

[0673] The server builds a knowledge base based on the extracted information.

[0674] Input: Extracted key information.

[0675] Specific operation: The server organizes the information extracted into the database and stores project summaries, ongoing tasks, and contact information for assigned personnel.

[0676] Output: The constructed knowledge base.

[0677] Step 7:

[0678] The server inputs the knowledge base into the generative artificial intelligence and trains it.

[0679] Input: Knowledge base.

[0680] Specific operation: The server supplies knowledge base information to a generative artificial intelligence model and performs training. Machine learning libraries such as TensorFlow and PyTorch are used to train the model.

[0681] Output: A trained generative artificial intelligence model.

[0682] Step 8:

[0683] The server automatically generates the handover document using a pre-trained generative artificial intelligence model.

[0684] Input: Trained generative artificial intelligence models and knowledge base data.

[0685] Specific operation: The server generates a handover document using a generative artificial intelligence model. The handover document will include the project's objectives, progress, details of the team members in charge, and future tasks.

[0686] Output: Automated handover document.

[0687] Step 9:

[0688] The user reviews the generated handover document and makes corrections as needed.

[0689] Input: Automated handover document.

[0690] Specific operation: The user reviews the contents of the handover document on the interface and edits any unclear parts or sections that require additional explanation.

[0691] Output: Revised final version of the handover document.

[0692] Step 10:

[0693] When a user reviews a handover document, an emotion analysis engine analyzes the user's emotions and provides feedback.

[0694] Input: Voice and facial expression data of the user reviewing the handover document.

[0695] Specific operation: The emotion analysis engine analyzes the user's emotions in real time, identifying feelings of confusion, anxiety, etc. Based on this, it provides feedback.

[0696] Output: User sentiment analysis results and provided feedback.

[0697] Step 11:

[0698] The server readjusts or supplements the contents of the handover document based on the results of the emotion analysis engine.

[0699] Input: Results from the emotion analysis engine.

[0700] Specific operation: The server adjusts the relevant sections of the handover document based on the user's feelings and adds necessary supplementary explanations. For example, it adds detailed explanations to sections where the user felt uneasy.

[0701] Output: Adjusted and supplemented handover documents.

[0702] (Application Example 2)

[0703] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0704] In today's work environment, information management and handover processes are becoming increasingly complex. In particular, information regarding the maintenance and operation of factory robots is diverse, making it difficult for new engineers to smoothly take over the work. Typical handover documents are prone to missing or inconsistent information, reducing the efficiency of training. Furthermore, there is a lack of means to alleviate the stress and anxiety engineers experience while reviewing handover documents. It is necessary to address these problems.

[0705] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, and sentiment analysis means to analyze the user's emotions during the review of the handover document and provide appropriate feedback. This makes it possible to perform an efficient and detailed handover while reducing the anxiety and confusion felt by the technician.

[0706] A "user" is an individual or legal entity that uses the system to specify, verify, or modify business data.

[0707] "Business data" refers to information such as emails, meeting logs, and documents necessary for the progress of work.

[0708] "Storage location" refers to the storage medium, such as a server, cloud storage, or local storage, where business data is stored.

[0709] A "server" is a central processing unit that collects, cleanses, and formats data, builds a knowledge base, trains generative artificial intelligence, and generates handover documents.

[0710] "Data collection" refers to retrieving specified business data from a storage location.

[0711] "Cleaning" is the process of removing unnecessary information from collected data to improve its quality.

[0712] "Format unification" is the process of standardizing data stored in different formats into a consistent format.

[0713] A "knowledge base" is a database that stores important information extracted from cleansed and formatted data.

[0714] "Generative artificial intelligence" refers to an artificial intelligence model that learns from a knowledge base and has the function of generating specific outputs (for example, handover documents).

[0715] A "handover document" is a document used for handing over work, which includes details of the work, progress, and contact information of the person in charge.

[0716] "Emotional analysis" is a process that provides appropriate feedback by analyzing the user's emotional state.

[0717] Basic System Configuration

[0718] A system for implementing this invention includes the following main elements:

[0719] 1. Method for specifying data storage location: The user logs into the system and specifies the storage location for business data (e.g., email service, cloud storage, local file server).

[0720] 2. Data Collection Method: The server collects data from designated storage locations. It efficiently retrieves data using the API of each storage location.

[0721] 3. Data cleansing and formatting unification means: The server cleanses the collected data (e.g., removes unnecessary information) and converts the data into a consistent format.

[0722] 4. Knowledge Base Construction Method: The server extracts important information from cleansed and formatted data to build a knowledge base. The knowledge base includes project overviews, ongoing tasks, and contact information for assigned personnel.

[0723] 5. Generative AI Training Method: The server inputs the knowledge base into the generative AI and performs training.

[0724] 6. Handover document generation method: The server automatically generates the handover document using a trained generative artificial intelligence model.

[0725] 7. User Verification and Correction Method: The user will review the generated handover document and make corrections as necessary.

[0726] 8. Emotion Analysis Method: Analyzes the user's facial expressions and voice while reviewing the handover document, and provides feedback based on the user's emotional state.

[0727] Hardware and software configuration

[0728] Hardware: Servers, user terminals (PCs or mobile devices), sensors (cameras and microphones).

[0729] Software: API communication software (REST API, etc.), text processing software (Python libraries), data cleansing software (Pandas), generative artificial intelligence (OpenAI's GPT, etc.), sentiment analysis software (EmotionAnalyzer).

[0730] Specific example

[0731] For example, consider using this invention in a system to support the maintenance and operation of robots used in a factory. When a new technician takes over a task, this system is used to automatically collect data such as emails, maintenance logs, and work procedures left by the previous technician. The collected data is cleansed and standardized in format. A knowledge base is built, and this information is used to train generative artificial intelligence. The technician reviews the automatically generated handover document, and if sentiment analysis indicates confusion or anxiety, the system provides appropriate feedback.

[0732] Example of a prompt

[0733] Design a system for new technicians to take over factory robot maintenance tasks. This system will extract key information from the predecessor's emails, maintenance logs, and work procedures, and automatically generate a handover document. It will also include a function to analyze the technician's emotions and add detailed supplementary explanations if they are feeling anxious or confused.

[0734] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0735] Step 1:

[0736] The user logs into the system and specifies the location where work-related data is stored. In this step, the user can specify an email service, cloud storage, local file server, etc. The input is the user's specified storage location information, and the output is a list of the specified storage locations.

[0737] Step 2:

[0738] The server collects data from a storage location specified by the user. It accesses the storage location and retrieves the data using APIs or the file system. The input is a list of storage locations, and the output is a collection of the collected original data.

[0739] Step 3:

[0740] The server cleanses the collected data. It performs text processing to remove spam emails and unwanted information from meeting logs. The input is the original collected data, and the output is the cleansed data.

[0741] Step 4:

[0742] The server cleanses the data and standardizes its format. It converts data in different formats into a unified format. The input is cleansed data, and the output is data with a unified format.

[0743] Step 5:

[0744] The server extracts important information from standardized data and builds a knowledge base. For example, it extracts key topics, task progress, and important decisions from email content. The input is standardized data, and the output is a knowledge base.

[0745] Step 6:

[0746] The server inputs the knowledge base into a generative artificial intelligence (AI) model and trains it. It learns patterns and templates for generating handover documents from the information in the knowledge base. The input is the knowledge base, and the output is the trained generative AI model.

[0747] Step 7:

[0748] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes details of the task, progress, and contact information of the person in charge. The input is the pre-trained generative artificial intelligence model, and the output is the handover document.

[0749] Step 8:

[0750] The user reviews the generated handover document and makes corrections as needed. They access the handover document using a terminal and modify the necessary parts. The input is the handover document, and the output is the corrected handover document.

[0751] Step 9:

[0752] While the server is verifying the handover document, it runs an emotion analysis engine to analyze the user's emotions. It analyzes voice and facial expression data to identify the user's emotional state. The input is the user's voice and facial expression data, and the output is the user's emotional state.

[0753] Step 10:

[0754] The server readjusts or supplements the handover document based on the user's emotional state identified through sentiment analysis. Detailed explanations are added to sections where the user felt confused or anxious. The input is the user's emotional state and the handover document; the output is the supplemented handover document.

[0755] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0756] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0757] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0758] [Third Embodiment]

[0759] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0760] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0761] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0762] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0763] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0764] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0765] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0766] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0767] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0768] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0769] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0770] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0771] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, cleansing and standardizing the data format, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence system, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[0772] System operation description

[0773] 1. Data Collection Phase

[0774] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0775] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[0776] 2. Data preprocessing phase

[0777] The server cleanses the collected data. For example, the server removes spam emails and deletes unnecessary information from meeting logs.

[0778] The server converts the cleansed data into a unified format. For example, it formats meeting logs in various formats into a consistent template.

[0779] 3. Knowledge Base Construction Phase

[0780] The server extracts important information from pre-processed data. For example, it extracts key topics and project progress from email content.

[0781] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[0782] 4. AI Learning Phase

[0783] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[0784] 5. Handover document generation phase

[0785] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. For example, the AI ​​model might create a "Project X Personnel Handover Document" which would include the project's objectives, progress, details of the team members, and the next tasks to be performed.

[0786] The user reviews the generated handover document and makes corrections as needed. For example, if the user thinks "this section needs additional explanation," they manually edit that section.

[0787] Specific example

[0788] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0789] Email storage location: Email service A

[0790] Location where meeting logs are stored: Cloud storage service B

[0791] Document storage location: Internal file server C

[0792] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[0793] Once the generative AI model has completed its training, it automatically generates a handover document. This document includes a comprehensive overview of the project, ongoing tasks, and contact information for each person involved. The project manager reviews this document and makes revisions as needed. As a result, it becomes possible to create accurate and consistent handover documents, significantly reducing time and effort.

[0794] The following describes the processing flow.

[0795] Step 1:

[0796] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0797] Step 2:

[0798] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[0799] Step 3:

[0800] The server cleanses the collected data. For example, it filters spam from email data and performs text processing to remove unnecessary information from meeting logs.

[0801] Step 4:

[0802] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[0803] Step 5:

[0804] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[0805] Step 6:

[0806] The server builds a knowledge base based on the extracted information. This knowledge base integrates project overviews, progress, and contact information for each person in charge.

[0807] Step 7:

[0808] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[0809] Step 8:

[0810] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[0811] Step 9:

[0812] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[0813] (Example 1)

[0814] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0815] In today's business environment, managing and transferring business data is a critical challenge. However, because various data are stored in scattered locations, data collection and organization are cumbersome and inefficient. Furthermore, creating handover documents is time-consuming and laborious, and ensuring the accuracy and consistency of the information is difficult. To solve these problems, efficient collection and standardization of business data formats, extraction of key information and construction of a knowledge base, and automated generation of handover documents using AI are necessary.

[0816] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0817] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location using an API; means for the server to cleanse and format the collected data using a Python data processing library; means for the server to extract important information from the cleansed and formatted data using a natural language processing library, store it in a relational database, and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it using an API; means for the server to generate a handover document based on prompt statements using the trained generative artificial intelligence model; and means for the user to review and modify the generated handover document through a web interface. This enables efficient collection and organization of business data, and allows for the creation of consistent and accurate handover documents in a short amount of time.

[0818] A "user" is an entity that uses the system to specify the storage location for business-related data and to review and modify the generated handover documents.

[0819] A "server" is a computing device that collects data based on user instructions, cleanses and standardizes the format, extracts important information to build a knowledge base, and generates handover documents using generative artificial intelligence.

[0820] "API" stands for Application Programming Interface, and it is an interface for exchanging data and functions between different systems.

[0821] Python is a high-level programming language and a widely used tool for data processing, machine learning, web development, and more.

[0822] A "data processing library" is a programming library used for tasks such as reading, processing, cleaning, and formatting data, and includes Python's pandas and numpy.

[0823] "Cleaning" is the process of removing redundant or unnecessary information from collected data to improve its quality.

[0824] "Format unification" is the process of converting data stored in different formats into a consistent format.

[0825] A "natural language processing library" is a programming library for understanding and processing human language, and includes Python's spaCy and nltk.

[0826] "Extraction" is the process of extracting important information from collected and cleansed data.

[0827] A "relational database" is a database that manages data by establishing relationships between multiple tables, and includes MySQL and PostgreSQL.

[0828] A "knowledge base" is an information database that compiles extracted important information, including project overview, progress, and contact information for team members.

[0829] "Generative artificial intelligence" refers to artificial intelligence that generates new content based on given data, and includes, for example, GPT-3.

[0830] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on what to output.

[0831] A "web interface" is a user interface that allows users to access, operate, check, and modify a system through a web browser.

[0832] A "handover document" is a document that describes the progress of the work, details of the person in charge, and the next tasks to be performed, and contains the information necessary for the next person in charge to smoothly take over the work.

[0833] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, performing cleansing and formatting, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[0834] Specifically, users log into the system and specify where their work data is stored. For example, a user might set "emails are stored in the email service, meeting logs in the cloud storage service, and documents in the company's internal file server." The server then automatically collects the data using the APIs of each specified storage location. Specifically, it uses the Python requests library to retrieve data from APIs such as Google Workspace API, Microsoft Teams API, and File System API, and saves it to local storage (for example, an S3 bucket).

[0835] The collected data then proceeds to the data preprocessing phase. The server uses the Python pandas library to cleanse the data, removing spam emails and unnecessary information from meeting logs. Furthermore, the cleansed data is converted into a unified format. For example, meeting logs in different formats are formatted into a consistent CSV format.

[0836] Next, the server extracts important information from the pre-processed data. Using Python's natural language processing libraries (e.g., spaCy or nltk), it extracts key information such as agenda items and project progress from the email content. The extracted information is stored in a relational database such as MySQL or PostgreSQL and built as a knowledge base. The knowledge base organizes project overviews, ongoing tasks, and contact information for those in charge.

[0837] The constructed knowledge base is input into a generative artificial intelligence (e.g., OpenAI's GPT-3). The server uses the OpenAI API to input the knowledge base information and train the AI ​​model. Through training, the AI ​​model learns patterns and templates for generating handover documents.

[0838] The server automatically generates the handover document using a pre-trained generative artificial intelligence model. For example, use the following prompt:

[0839] Please create a handover document for Project X. Include the following information:

[0840] Project objectives

[0841] Progress

[0842] Details of the assigned members

[0843] "Next task"

[0844] The generated handover document covers the overall project overview, ongoing tasks, and contact information for each person in charge. Users log into the system and review the handover document through the web interface, making corrections as needed. This enables efficient collection and organization of business data, allowing for the creation of consistent and accurate handover documents in a short amount of time.

[0845] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0846] Step 1:

[0847] The user logs into the system and specifies the location where their work data is stored. As input, the user specifies the location for emails, meeting logs, documents, etc. As output, this storage location information is sent to the server. Specifically, the user accesses a web interface and enters the storage location information via a form.

[0848] Step 2:

[0849] The server receives data storage location information from the user and collects data from each specified storage location. The input is storage location information (e.g., Google Workspace API, Microsoft Teams API, File System API). The output is the collected data stored in local storage. Specifically, the server uses the Python requests library to send requests to each API and saves the retrieved data to a local S3 bucket.

[0850] Step 3:

[0851] The server cleanses and standardizes the format of the collected data. The collected data is given as input. The cleansed and standardized data is generated as output. Specifically, the server uses the Python pandas library to read the data, performs spam filtering, and converts data in different formats to CSV format.

[0852] Step 4:

[0853] The server extracts important information from pre-processed data and builds a knowledge base. The input is cleansed and formatted data. The output is extracted important information and a knowledge base is built. Specifically, the server uses the Python spaCy library to perform natural language processing, extracting agenda items and project progress from email content and saving it to a MySQL database.

[0854] Step 5:

[0855] The server inputs the knowledge base into a generative artificial intelligence (AI) and performs training. The input is the information from the knowledge base. The output is a trained generative AI model. Specifically, the server uses the OpenAI API to upload the knowledge base information in JSON format and executes the AI ​​model training job.

[0856] Step 6:

[0857] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The input is a prompt (e.g., "Please create a handover document for Project X. Include the following information: project objectives, progress, details of team members, and next steps"). The output is a generated handover document. Specifically, the server sends the prompt to the generative AI model's endpoint and formats the response as an HTML or PDF document.

[0858] Step 7:

[0859] The user reviews the generated handover document and makes corrections as needed. The generated handover document is provided as input. The corrected handover document is completed as output. Specifically, the user reviews the document via a web interface and edits it using an editor to make necessary corrections.

[0860] (Application Example 1)

[0861] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0862] Currently, in many business operations, data collection, cleansing, formatting standardization, extraction of important information, knowledge base construction, and the generation of handover documents based on that data are performed manually. This is inefficient and prone to errors. Furthermore, in factory work environments, if proper handover procedures are not followed, there is a high risk of business stagnation and problems. In addition, current handover document generation systems are often complex to operate and require technical knowledge from users. To solve these problems, there is a need to provide a system that automates the process from data collection to handover document generation and makes it easy for users to use through smart devices.

[0863] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0864] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location; means for the server to cleanse and standardize the format of the collected data; means for the server to extract important information from the cleansed and standardized data and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it; means for the server to generate a handover document using the trained generative artificial intelligence model; means for the user to review and modify the generated handover document; and means for the user to specify the data storage location by voice command or code scan using a smart device. As a result, the process from data collection to handover document generation is automated, enabling the user to generate handover documents that are easy to operate and efficient.

[0865] A "user" is an individual or group that operates the system and specifies where data is stored.

[0866] "Business-related data" refers to information and records necessary for carrying out business operations, and includes emails, meeting logs, documents, etc.

[0867] "Storage location" refers to the physical or virtual location where business-related data is stored.

[0868] A "server" is a computer device that collects, cleanses, and converts data formats, builds a knowledge base, and generates handover documents.

[0869] "Means of data collection" refers to methods or techniques for retrieving data from a storage location specified by the user.

[0870] "Methods for cleansing data" refer to methods or techniques for removing unnecessary information from collected data.

[0871] "Means of standardizing data formats" refers to methods or techniques for converting data in different formats into a consistent format.

[0872] "Important information" refers to information that is particularly valuable in carrying out business operations, and examples include agenda items and project progress.

[0873] A "knowledge base" is a database where important collected information is organized and stored.

[0874] "Generative artificial intelligence" is an artificial intelligence technology used to automatically generate handover documents using a knowledge base as training data.

[0875] A "handover document" is a document used to explain the content and progress of a task to a new person in charge when the person in charge of that task changes.

[0876] A "smart device" is a device that has voice command or code scanning capabilities, and includes smart glasses and smartphones.

[0877] A system implementing this invention uses multiple hardware and software components to automate multi-stage data processing and generate accurate and efficient handover documents.

[0878] Hardware and software configuration

[0879] 1. User terminal:

[0880] Smart devices: These devices, particularly smart glasses and smartphones, incorporate voice commands and code scanning capabilities. This allows users to easily specify where data is stored. Examples include the Vuzix Blade and Google Glass smart glasses.

[0881] 2. Server:

[0882] Cloud storage APIs: Use APIs such as Google Drive API and Dropbox API to collect data from storage locations specified by the user.

[0883] Data Cleansing Module: Uses Python scripts and the Pandas library to cleanse collected data and remove unnecessary information.

[0884] Data Format Unification Module: Use Python scripts and other data conversion tools to convert data in various formats into a unified format.

[0885] Knowledge base building module: Extracts important information and stores it as a knowledge base in an SQL database or Elasticsearch.

[0886] Generative artificial intelligence: AI models that are trained using knowledge base information, such as GPT-3 and BERT, using TensorFlow or PyTorch.

[0887] Processing flow

[0888] 1. Data collection phase:

[0889] Users use smart glasses to issue voice commands or scan QR codes to specify the data storage location.

[0890] The server uses a cloud storage API to collect data from a specified storage location.

[0891] 2. Data preprocessing phase:

[0892] The server's data cleansing module removes unwanted information from the collected data. For example, it filters out spam emails.

[0893] The data format unification module converts data in different formats into a consistent format. For example, it can format various types of meeting logs into a single template.

[0894] 3. Knowledge Base Construction Phase:

[0895] The server extracts important information from pre-processed data and builds a knowledge base. This knowledge base includes project overviews, progress, and contact information for team members.

[0896] 4. AI Learning Phase:

[0897] The server-based knowledge base is input into a generative artificial intelligence system for training. TensorFlow or PyTorch are used.

[0898] 5. Handover document generation phase:

[0899] The server automatically generates the handover document using a pre-trained generative artificial intelligence model.

[0900] The user reviews the handover document generated on the smart glasses and makes corrections as needed.

[0901] Specific example

[0902] For example, consider a factory worker using this system during a shift change. The worker wears smart glasses and gives a voice command such as "Start acquiring meeting logs." The server then collects the necessary data from cloud storage, cleanses it, and standardizes the format. After that, it builds a knowledge base and generates a handover document using generative artificial intelligence. The worker reviews the handover document on the smart glasses' display and makes corrections as needed using voice input.

[0903] Example of a prompt:

[0904] "Starting to retrieve meeting logs"

[0905] Thus, a system implementing this invention significantly reduces the burden on users and enables efficient and accurate handover of tasks.

[0906] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0907] Step 1:

[0908] The user specifies the data storage location using a smart device via voice command or code scan. The input is the voice command or code recognition result. The output is information about the specified storage location.

[0909] Step 2:

[0910] The server uses a cloud storage API to collect data from a specified storage location. The input is information about the storage location specified by the user. The output is the collected raw data.

[0911] Step 3:

[0912] The server uses a data cleansing module to remove unwanted information from the collected raw data. The input is the collected raw data. The output is the cleansed data. Specifically, the server applies a spam filtering algorithm to remove spam data and filters out irrelevant information.

[0913] Step 4:

[0914] The server uses a data format unification module to convert the cleaned data into a consistent format. The input is the cleaned data. The output is the formatted data. Specifically, it formats meeting logs in different formats into a single template.

[0915] Step 5:

[0916] The server extracts important information from pre-processed data and builds a knowledge base. The input is data in a standardized format. The output is information from the knowledge base. Specifically, the server uses natural language processing algorithms to extract keywords and important topics from each data point and stores them in the database.

[0917] Step 6:

[0918] The server inputs the constructed knowledge base into a generative artificial intelligence and performs training. The input is the information from the knowledge base. The output is the trained AI model. Specifically, the server uses TensorFlow or PyTorch to learn patterns and templates for generating handover documents.

[0919] Step 7:

[0920] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. The input is information from the knowledge base and the trained AI model. The output is the generated handover document. Specifically, the AI ​​model integrates information such as project objectives, progress, and details of assigned members to create the handover document.

[0921] Step 8:

[0922] The user reviews the generated handover document on their smart device and makes corrections as needed. The input is the generated handover document. The output is the reviewed and corrected handover document. Specifically, the user reviews the content and corrects any errors or omissions using voice commands or the input functions of their smart device.

[0923] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0924] This invention relates to a system in which a user specifies a location for storing business-related data, a server collects the data from that location, cleanses and standardizes the format, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. This system incorporates an emotion engine that recognizes the user's emotions and provides appropriate feedback.

[0925] System operation description

[0926] 1. Data Collection Phase

[0927] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0928] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[0929] 2. Data preprocessing phase

[0930] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[0931] The server converts the cleansed data into a unified format. For example, it formats various types of meeting logs into a consistent template format.

[0932] 3. Knowledge Base Construction Phase

[0933] The server extracts important information from pre-processed data. For example, it extracts key topics, task progress, and important decisions from email content.

[0934] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[0935] 4. AI Learning Phase

[0936] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[0937] 5. Handover document generation phase

[0938] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes the project objectives, progress, details of the team members involved, and future tasks.

[0939] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[0940] 6. Emotional Engine Phase

[0941] When a user reviews a handover document, the emotion engine analyzes the user's voice and facial expressions to identify their emotional state. For example, if the user is confused, the emotion engine recognizes this and provides feedback.

[0942] The server readjusts or supplements the contents of the handover document based on the emotional state identified by the emotion engine. For example, if the user is feeling anxious about a certain item, detailed supplementary explanations will be added to that section.

[0943] Specific example

[0944] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[0945] Email storage location: Email service A

[0946] Location where meeting logs are stored: Cloud storage service B

[0947] Document storage location: Internal file server C

[0948] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[0949] Once the generative artificial intelligence model has completed its training, it automatically generates a handover document. This document includes an overview of the project, ongoing tasks, and contact information for each person in charge. The project manager reviews this document, and the emotion engine recognizes the project manager's facial expressions and voice to analyze their emotional state. For example, if anxiety or doubt is detected, the emotion engine senses this and automatically adds supplementary explanations or details to the relevant sections of the handover document.

[0950] Based on the finalized handover document, users can smoothly complete the handover of their duties. This improves the efficiency of the handover process and reduces stress.

[0951] The following describes the processing flow.

[0952] Step 1:

[0953] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[0954] Step 2:

[0955] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[0956] Step 3:

[0957] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[0958] Step 4:

[0959] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[0960] Step 5:

[0961] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[0962] Step 6:

[0963] The server builds a knowledge base based on the extracted information. The knowledge base integrates project overviews, ongoing tasks, and contact information for assigned personnel.

[0964] Step 7:

[0965] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[0966] Step 8:

[0967] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[0968] Step 9:

[0969] The user views and reviews the handover document. At this time, the emotion engine activates, analyzing the user's voice and facial expressions to determine their emotional state. For example, if the user is confused, the emotion engine recognizes this from their facial expressions and tone of voice.

[0970] Step 10:

[0971] The server receives feedback from the emotion engine and readjusts or supplements the handover document as needed. For example, it might add detailed explanations to sections where the user expressed anxiety or doubt.

[0972] Step 11:

[0973] The user reviews the final handover document and makes manual corrections as needed. Once the user is satisfied with all the content, they declare the handover document complete and provide it to the next person in charge.

[0974] Through the steps described above, the system of the present invention can streamline the user handover process and improve the quality of the handover.

[0975] (Example 2)

[0976] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0977] During the handover of work responsibilities, organizing data and extracting information is time-consuming and laborious, making efficient handover difficult. Furthermore, the lack of feedback functions that consider user emotions prevents stress reduction. As a result, deficiencies in handover content and insufficient communication often occur.

[0978] In Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, means for the sentiment analysis engine to analyze the user's emotions and provide feedback, and means for the server to readjust or supplement the contents of the handover document based on the results of the sentiment analysis engine. As a result, a series of processes from automatic data collection, cleansing, and information extraction to the generation of the handover document and feedback based on the user's emotions are automated, enabling an efficient and user-friendly handover.

[0979] A "user" is a person or entity whose role is to access the system, specify the location where business-related data is stored, and review and modify the generated handover documents.

[0980] A "server" is a computer system that collects data from a storage location specified by the user, cleanses and standardizes the format of the collected data, extracts important information, and builds a knowledge base.

[0981] "Storage location" refers to a physical or cloud storage system where business-related data is stored.

[0982] "Cleansing" is the process of removing unnecessary information and noise from collected data.

[0983] "Format unification" is the process of converting data stored in different formats into a consistent, standardized format.

[0984] "Important information" refers to information that is particularly valuable in business operations and is necessary for building knowledge bases and generating handover documents.

[0985] A "knowledge base" is a database that systematically organizes and stores important extracted information.

[0986] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information or documents from given data.

[0987] "Training" is the process by which a generative artificial intelligence learns from data in a given knowledge base and builds a model for generating handover documents.

[0988] A "handover document" is a document used for handing over work responsibilities, including the project's objectives, progress, details of the person in charge, and future tasks.

[0989] An "emotion analysis engine" is a system that analyzes a user's voice and facial expressions to identify their emotional state.

[0990] "Feedback" refers to instructions and supplementary information provided based on the user's emotional state identified by the emotion analysis engine.

[0991] This invention is a system in which the user specifies a location for storing business-related data, and a server collects, cleanses, and standardizes the data from that location, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. An emotion analysis engine is also incorporated to recognize the user's emotions and provide appropriate feedback.

[0992] System Configuration

[0993] Data collection

[0994] The user logs into the system and specifies where work-related data is stored. For example, a user can set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company file server C." The server retrieves the data from the specified storage locations using an API.

[0995] Data preprocessing

[0996] The server cleanses the acquired data. This phase involves removing spam emails and unnecessary information from meeting logs. In particular, a text analysis engine is used for data cleansing. Next, the server converts the cleansed data into a unified format, shaping various data formats into a consistent template.

[0997] Building a knowledge base

[0998] The server extracts key information from pre-processed data and builds a knowledge base. This extracted information includes, for example, the main topics of emails, task progress, and important decisions. The resulting knowledge base includes project summaries, ongoing tasks, and contact information for those responsible.

[0999] AI learning

[1000] The server inputs data from the knowledge base into a generative artificial intelligence (AI) and performs training. The generative AI used for training utilizes various machine learning libraries (e.g., TensorFlow, PyTorch) to learn patterns and templates for generating handover documents.

[1001] Handover document generation

[1002] Using a completed generative artificial intelligence model, the server automatically generates a handover document. The generated document details the project's objectives, progress, team members, and future tasks. The user reviews this generated document and makes any necessary modifications.

[1003] Emotion analysis

[1004] When a user reviews a handover document, an emotion analysis engine analyzes the user's voice and facial expressions to identify their emotional state. Emotion analysis is performed in real time; for example, if the user is confused, the emotion analysis engine recognizes this, and the server automatically adds detailed supplementary explanations to the relevant section.

[1005] Specific example

[1006] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[1007] Email storage location: Email service A

[1008] Location where meeting logs are stored: Cloud storage service B

[1009] Document storage location: Internal file server C

[1010] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. It then extracts important information from the cleansed data to build a knowledge base. This knowledge base is input into a generative artificial intelligence (AI) model for training. The improved AI model automatically generates a handover document. The project manager reviews this document, and an emotion analysis engine recognizes the project manager's emotions through facial expressions and voice, providing supplementary explanations where necessary.

[1011] Example of a prompt

[1012] "Please create a detailed handover document for the next project you will be taking over. Include the project's objectives, progress, details of the person in charge, and future tasks. Also, ensure that appropriate feedback is provided regarding the user's emotional state."

[1013] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1014] Step 1:

[1015] The user logs into the system and specifies the location where work-related data will be stored.

[1016] Input: Specify the user's login information and data storage location (e.g., email service, cloud storage, internal file server).

[1017] Specific operation: The user enters "Email Service A" as the email storage location, "Cloud Storage B" as the meeting log storage location, and "Internal File Server C" as the document storage location into the interface's input fields.

[1018] Output: The specified save location information is sent to the server and recorded.

[1019] Step 2:

[1020] The server automatically collects data from a storage location specified by the user.

[1021] Input: Information about the save location specified by the user.

[1022] Specific operation: The server calls the storage location API to retrieve email data from email service A, meeting log data from cloud storage B, and document data from the internal file server C.

[1023] Output: The collected raw data is stored on the server.

[1024] Step 3:

[1025] The server cleanses the collected data.

[1026] Input: The collected raw data.

[1027] Specific operation: The server uses a text analysis engine to remove spam emails and unnecessary information from meeting logs. For example, it uses an NLP model to perform spam filtering and remove unnecessary words such as "um" and "uh" from meeting logs.

[1028] Output: Cleaned, clean data.

[1029] Step 4:

[1030] The server converts the cleansed data into a unified format.

[1031] Input: Cleansed, clean data.

[1032] Specific operation: The server executes a script to convert various data formats into a consistent template format. For example, it formats meeting logs in different formats into a single unified template.

[1033] Output: Data in a standardized format.

[1034] Step 5:

[1035] The server extracts important information from the pre-processed data.

[1036] Input: Data in a standardized format.

[1037] Specific operation: The server applies an information extraction algorithm to extract the main topics, task progress, and important decisions from emails. For example, it might extract "next action points" or "decisions" from meeting logs.

[1038] Output: Extracted key information.

[1039] Step 6:

[1040] The server builds a knowledge base based on the extracted information.

[1041] Input: Extracted key information.

[1042] Specific operation: The server organizes the information extracted into the database and stores project summaries, ongoing tasks, and contact information for assigned personnel.

[1043] Output: The constructed knowledge base.

[1044] Step 7:

[1045] The server inputs the knowledge base into the generative artificial intelligence and trains it.

[1046] Input: Knowledge base.

[1047] Specific operation: The server supplies knowledge base information to a generative artificial intelligence model and performs training. Machine learning libraries such as TensorFlow and PyTorch are used to train the model.

[1048] Output: A trained generative artificial intelligence model.

[1049] Step 8:

[1050] The server automatically generates the handover document using a pre-trained generative artificial intelligence model.

[1051] Input: Trained generative artificial intelligence models and knowledge base data.

[1052] Specific operation: The server generates a handover document using a generative artificial intelligence model. The handover document will include the project's objectives, progress, details of the team members in charge, and future tasks.

[1053] Output: Automated handover document.

[1054] Step 9:

[1055] The user reviews the generated handover document and makes corrections as needed.

[1056] Input: Automated handover document.

[1057] Specific operation: The user reviews the contents of the handover document on the interface and edits any unclear parts or sections that require additional explanation.

[1058] Output: Revised final version of the handover document.

[1059] Step 10:

[1060] When a user reviews a handover document, an emotion analysis engine analyzes the user's emotions and provides feedback.

[1061] Input: Voice and facial expression data of the user reviewing the handover document.

[1062] Specific operation: The emotion analysis engine analyzes the user's emotions in real time, identifying feelings of confusion, anxiety, etc. Based on this, it provides feedback.

[1063] Output: User sentiment analysis results and provided feedback.

[1064] Step 11:

[1065] The server readjusts or supplements the contents of the handover document based on the results of the emotion analysis engine.

[1066] Input: Results from the emotion analysis engine.

[1067] Specific operation: The server adjusts the relevant sections of the handover document based on the user's feelings and adds necessary supplementary explanations. For example, it adds detailed explanations to sections where the user felt uneasy.

[1068] Output: Adjusted and supplemented handover documents.

[1069] (Application Example 2)

[1070] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1071] In today's work environment, information management and handover processes are becoming increasingly complex. In particular, information regarding the maintenance and operation of factory robots is diverse, making it difficult for new engineers to smoothly take over the work. Typical handover documents are prone to missing or inconsistent information, reducing the efficiency of training. Furthermore, there is a lack of means to alleviate the stress and anxiety engineers experience while reviewing handover documents. It is necessary to address these problems.

[1072] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, and sentiment analysis means to analyze the user's emotions during the review of the handover document and provide appropriate feedback. This makes it possible to perform an efficient and detailed handover while reducing the anxiety and confusion felt by the technician.

[1073] A "user" is an individual or legal entity that uses the system to specify, verify, or modify business data.

[1074] "Business data" refers to information such as emails, meeting logs, and documents necessary for the progress of work.

[1075] "Storage location" refers to the storage medium, such as a server, cloud storage, or local storage, where business data is stored.

[1076] A "server" is a central processing unit that collects, cleanses, and formats data, builds a knowledge base, trains generative artificial intelligence, and generates handover documents.

[1077] "Data collection" refers to retrieving specified business data from a storage location.

[1078] "Cleaning" is the process of removing unnecessary information from collected data to improve its quality.

[1079] "Format unification" is the process of standardizing data stored in different formats into a consistent format.

[1080] A "knowledge base" is a database that stores important information extracted from cleansed and formatted data.

[1081] "Generative artificial intelligence" refers to an artificial intelligence model that learns from a knowledge base and has the function of generating specific outputs (for example, handover documents).

[1082] A "handover document" is a document used for handing over work, which includes details of the work, progress, and contact information of the person in charge.

[1083] "Emotional analysis" is a process that provides appropriate feedback by analyzing the user's emotional state.

[1084] Basic System Configuration

[1085] A system for implementing this invention includes the following main elements:

[1086] 1. Method for specifying data storage location: The user logs into the system and specifies the storage location for business data (e.g., email service, cloud storage, local file server).

[1087] 2. Data Collection Method: The server collects data from designated storage locations. It efficiently retrieves data using the API of each storage location.

[1088] 3. Data cleansing and formatting unification means: The server cleanses the collected data (e.g., removes unnecessary information) and converts the data into a consistent format.

[1089] 4. Knowledge Base Construction Method: The server extracts important information from cleansed and formatted data to build a knowledge base. The knowledge base includes project overviews, ongoing tasks, and contact information for assigned personnel.

[1090] 5. Generative AI Training Method: The server inputs the knowledge base into the generative AI and performs training.

[1091] 6. Handover document generation method: The server automatically generates the handover document using a trained generative artificial intelligence model.

[1092] 7. User Verification and Correction Method: The user will review the generated handover document and make corrections as necessary.

[1093] 8. Emotion Analysis Method: Analyzes the user's facial expressions and voice while reviewing the handover document, and provides feedback based on the user's emotional state.

[1094] Hardware and software configuration

[1095] Hardware: Servers, user terminals (PCs or mobile devices), sensors (cameras and microphones).

[1096] Software: API communication software (REST API, etc.), text processing software (Python libraries), data cleansing software (Pandas), generative artificial intelligence (OpenAI's GPT, etc.), sentiment analysis software (EmotionAnalyzer).

[1097] Specific example

[1098] For example, consider using this invention in a system to support the maintenance and operation of robots used in a factory. When a new technician takes over a task, this system is used to automatically collect data such as emails, maintenance logs, and work procedures left by the previous technician. The collected data is cleansed and standardized in format. A knowledge base is built, and this information is used to train generative artificial intelligence. The technician reviews the automatically generated handover document, and if sentiment analysis indicates confusion or anxiety, the system provides appropriate feedback.

[1099] Example of a prompt

[1100] Design a system for new technicians to take over factory robot maintenance tasks. This system will extract key information from the predecessor's emails, maintenance logs, and work procedures, and automatically generate a handover document. It will also include a function to analyze the technician's emotions and add detailed supplementary explanations if they are feeling anxious or confused.

[1101] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1102] Step 1:

[1103] The user logs into the system and specifies the location where work-related data is stored. In this step, the user can specify an email service, cloud storage, local file server, etc. The input is the user's specified storage location information, and the output is a list of the specified storage locations.

[1104] Step 2:

[1105] The server collects data from a storage location specified by the user. It accesses the storage location and retrieves the data using APIs or the file system. The input is a list of storage locations, and the output is a collection of the collected original data.

[1106] Step 3:

[1107] The server cleanses the collected data. It performs text processing to remove spam emails and unwanted information from meeting logs. The input is the original collected data, and the output is the cleansed data.

[1108] Step 4:

[1109] The server cleanses the data and standardizes its format. It converts data in different formats into a unified format. The input is cleansed data, and the output is data with a unified format.

[1110] Step 5:

[1111] The server extracts important information from standardized data and builds a knowledge base. For example, it extracts key topics, task progress, and important decisions from email content. The input is standardized data, and the output is a knowledge base.

[1112] Step 6:

[1113] The server inputs the knowledge base into a generative artificial intelligence (AI) model and trains it. It learns patterns and templates for generating handover documents from the information in the knowledge base. The input is the knowledge base, and the output is the trained generative AI model.

[1114] Step 7:

[1115] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes details of the task, progress, and contact information of the person in charge. The input is the pre-trained generative artificial intelligence model, and the output is the handover document.

[1116] Step 8:

[1117] The user reviews the generated handover document and makes corrections as needed. They access the handover document using a terminal and modify the necessary parts. The input is the handover document, and the output is the corrected handover document.

[1118] Step 9:

[1119] While the server is verifying the handover document, it runs an emotion analysis engine to analyze the user's emotions. It analyzes voice and facial expression data to identify the user's emotional state. The input is the user's voice and facial expression data, and the output is the user's emotional state.

[1120] Step 10:

[1121] The server readjusts or supplements the handover document based on the user's emotional state identified through sentiment analysis. Detailed explanations are added to sections where the user felt confused or anxious. The input is the user's emotional state and the handover document; the output is the supplemented handover document.

[1122] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1123] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1124] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1125] [Fourth Embodiment]

[1126] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1127] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1128] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1129] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1130] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1131] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1132] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1133] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1134] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1135] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1136] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1137] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1138] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1139] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, cleansing and standardizing the data format, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence system, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[1140] System operation description

[1141] 1. Data Collection Phase

[1142] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[1143] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[1144] 2. Data preprocessing phase

[1145] The server cleanses the collected data. For example, the server removes spam emails and deletes unnecessary information from meeting logs.

[1146] The server converts the cleansed data into a unified format. For example, it formats meeting logs in various formats into a consistent template.

[1147] 3. Knowledge Base Construction Phase

[1148] The server extracts important information from pre-processed data. For example, it extracts key topics and project progress from email content.

[1149] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[1150] 4. AI Learning Phase

[1151] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[1152] 5. Handover document generation phase

[1153] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. For example, the AI ​​model might create a "Project X Personnel Handover Document" which would include the project's objectives, progress, details of the team members, and the next tasks to be performed.

[1154] The user reviews the generated handover document and makes corrections as needed. For example, if the user thinks "this section needs additional explanation," they manually edit that section.

[1155] Specific example

[1156] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[1157] Email storage location: Email service A

[1158] Location where meeting logs are stored: Cloud storage service B

[1159] Document storage location: Internal file server C

[1160] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[1161] Once the generative AI model has completed its training, it automatically generates a handover document. This document includes a comprehensive overview of the project, ongoing tasks, and contact information for each person involved. The project manager reviews this document and makes revisions as needed. As a result, it becomes possible to create accurate and consistent handover documents, significantly reducing time and effort.

[1162] The following describes the processing flow.

[1163] Step 1:

[1164] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[1165] Step 2:

[1166] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[1167] Step 3:

[1168] The server cleanses the collected data. For example, it filters spam from email data and performs text processing to remove unnecessary information from meeting logs.

[1169] Step 4:

[1170] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[1171] Step 5:

[1172] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[1173] Step 6:

[1174] The server builds a knowledge base based on the extracted information. This knowledge base integrates project overviews, progress, and contact information for each person in charge.

[1175] Step 7:

[1176] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[1177] Step 8:

[1178] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[1179] Step 9:

[1180] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[1181] (Example 1)

[1182] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1183] In today's business environment, managing and transferring business data is a critical challenge. However, because various data are stored in scattered locations, data collection and organization are cumbersome and inefficient. Furthermore, creating handover documents is time-consuming and laborious, and ensuring the accuracy and consistency of the information is difficult. To solve these problems, efficient collection and standardization of business data formats, extraction of key information and construction of a knowledge base, and automated generation of handover documents using AI are necessary.

[1184] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1185] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location using an API; means for the server to cleanse and format the collected data using a Python data processing library; means for the server to extract important information from the cleansed and formatted data using a natural language processing library, store it in a relational database, and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it using an API; means for the server to generate a handover document based on prompt statements using the trained generative artificial intelligence model; and means for the user to review and modify the generated handover document through a web interface. This enables efficient collection and organization of business data, and allows for the creation of consistent and accurate handover documents in a short amount of time.

[1186] A "user" is an entity that uses the system to specify the storage location for business-related data and to review and modify the generated handover documents.

[1187] A "server" is a computing device that collects data based on user instructions, cleanses and standardizes the format, extracts important information to build a knowledge base, and generates handover documents using generative artificial intelligence.

[1188] "API" stands for Application Programming Interface, and it is an interface for exchanging data and functions between different systems.

[1189] Python is a high-level programming language and a widely used tool for data processing, machine learning, web development, and more.

[1190] A "data processing library" is a programming library used for tasks such as reading, processing, cleaning, and formatting data, and includes Python's pandas and numpy.

[1191] "Cleaning" is the process of removing redundant or unnecessary information from collected data to improve its quality.

[1192] "Format unification" is the process of converting data stored in different formats into a consistent format.

[1193] A "natural language processing library" is a programming library for understanding and processing human language, and includes Python's spaCy and nltk.

[1194] "Extraction" is the process of extracting important information from collected and cleansed data.

[1195] A "relational database" is a database that manages data by establishing relationships between multiple tables, and includes MySQL and PostgreSQL.

[1196] A "knowledge base" is an information database that compiles extracted important information, including project overview, progress, and contact information for team members.

[1197] "Generative artificial intelligence" refers to artificial intelligence that generates new content based on given data, and includes, for example, GPT-3.

[1198] A "prompt statement" is an input statement used to instruct a generative artificial intelligence on what to output.

[1199] A "web interface" is a user interface that allows users to access, operate, check, and modify a system through a web browser.

[1200] A "handover document" is a document that describes the progress of the work, details of the person in charge, and the next tasks to be performed, and contains the information necessary for the next person in charge to smoothly take over the work.

[1201] The system for implementing this invention involves a user specifying a location for storing business-related data, a server collecting the data from that location, performing cleansing and formatting, extracting important information, and building a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the finally generated handover document and makes corrections as needed.

[1202] Specifically, users log into the system and specify where their work data is stored. For example, a user might set "emails are stored in the email service, meeting logs in the cloud storage service, and documents in the company's internal file server." The server then automatically collects the data using the APIs of each specified storage location. Specifically, it uses the Python requests library to retrieve data from APIs such as Google Workspace API, Microsoft Teams API, and File System API, and saves it to local storage (for example, an S3 bucket).

[1203] The collected data then proceeds to the data preprocessing phase. The server uses the Python pandas library to cleanse the data, removing spam emails and unnecessary information from meeting logs. Furthermore, the cleansed data is converted into a unified format. For example, meeting logs in different formats are formatted into a consistent CSV format.

[1204] Next, the server extracts important information from the pre-processed data. Using Python's natural language processing libraries (e.g., spaCy or nltk), it extracts key information such as agenda items and project progress from the email content. The extracted information is stored in a relational database such as MySQL or PostgreSQL and built as a knowledge base. The knowledge base organizes project overviews, ongoing tasks, and contact information for those in charge.

[1205] The constructed knowledge base is input into a generative artificial intelligence (e.g., OpenAI's GPT-3). The server uses the OpenAI API to input the knowledge base information and train the AI ​​model. Through training, the AI ​​model learns patterns and templates for generating handover documents.

[1206] The server automatically generates the handover document using a pre-trained generative artificial intelligence model. For example, use the following prompt:

[1207] Please create a handover document for Project X. Include the following information:

[1208] Project objectives

[1209] Progress

[1210] Details of the assigned members

[1211] "Next task"

[1212] The generated handover document covers the overall project overview, ongoing tasks, and contact information for each person in charge. Users log into the system and review the handover document through the web interface, making corrections as needed. This enables efficient collection and organization of business data, allowing for the creation of consistent and accurate handover documents in a short amount of time.

[1213] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1214] Step 1:

[1215] The user logs into the system and specifies the location where their work data is stored. As input, the user specifies the location for emails, meeting logs, documents, etc. As output, this storage location information is sent to the server. Specifically, the user accesses a web interface and enters the storage location information via a form.

[1216] Step 2:

[1217] The server receives data storage location information from the user and collects data from each specified storage location. The input is storage location information (e.g., Google Workspace API, Microsoft Teams API, File System API). The output is the collected data stored in local storage. Specifically, the server uses the Python requests library to send requests to each API and saves the retrieved data to a local S3 bucket.

[1218] Step 3:

[1219] The server cleanses and standardizes the format of the collected data. The collected data is given as input. The cleansed and standardized data is generated as output. Specifically, the server uses the Python pandas library to read the data, performs spam filtering, and converts data in different formats to CSV format.

[1220] Step 4:

[1221] The server extracts important information from pre-processed data and builds a knowledge base. The input is cleansed and formatted data. The output is extracted important information and a knowledge base is built. Specifically, the server uses the Python spaCy library to perform natural language processing, extracting agenda items and project progress from email content and saving it to a MySQL database.

[1222] Step 5:

[1223] The server inputs the knowledge base into a generative artificial intelligence (AI) and performs training. The input is the information from the knowledge base. The output is a trained generative AI model. Specifically, the server uses the OpenAI API to upload the knowledge base information in JSON format and executes the AI ​​model training job.

[1224] Step 6:

[1225] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The input is a prompt (e.g., "Please create a handover document for Project X. Include the following information: project objectives, progress, details of team members, and next steps"). The output is a generated handover document. Specifically, the server sends the prompt to the generative AI model's endpoint and formats the response as an HTML or PDF document.

[1226] Step 7:

[1227] The user reviews the generated handover document and makes corrections as needed. The generated handover document is provided as input. The corrected handover document is completed as output. Specifically, the user reviews the document via a web interface and edits it using an editor to make necessary corrections.

[1228] (Application Example 1)

[1229] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1230] Currently, in many business operations, data collection, cleansing, formatting standardization, extraction of important information, knowledge base construction, and the generation of handover documents based on that data are performed manually. This is inefficient and prone to errors. Furthermore, in factory work environments, if proper handover procedures are not followed, there is a high risk of business stagnation and problems. In addition, current handover document generation systems are often complex to operate and require technical knowledge from users. To solve these problems, there is a need to provide a system that automates the process from data collection to handover document generation and makes it easy for users to use through smart devices.

[1231] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1232] In this invention, the server includes means for the user to specify a storage location for business-related data; means for the server to collect data from the storage location; means for the server to cleanse and standardize the format of the collected data; means for the server to extract important information from the cleansed and standardized data and build a knowledge base; means for the server to input the knowledge base into a generative artificial intelligence and train it; means for the server to generate a handover document using the trained generative artificial intelligence model; means for the user to review and modify the generated handover document; and means for the user to specify the data storage location by voice command or code scan using a smart device. As a result, the process from data collection to handover document generation is automated, enabling the user to generate handover documents that are easy to operate and efficient.

[1233] A "user" is an individual or group that operates the system and specifies where data is stored.

[1234] "Business-related data" refers to information and records necessary for carrying out business operations, and includes emails, meeting logs, documents, etc.

[1235] "Storage location" refers to the physical or virtual location where business-related data is stored.

[1236] A "server" is a computer device that collects, cleanses, and converts data formats, builds a knowledge base, and generates handover documents.

[1237] "Means of data collection" refers to methods or techniques for retrieving data from a storage location specified by the user.

[1238] "Methods for cleansing data" refer to methods or techniques for removing unnecessary information from collected data.

[1239] "Means of standardizing data formats" refers to methods or techniques for converting data in different formats into a consistent format.

[1240] "Important information" refers to information that is particularly valuable in carrying out business operations, and examples include agenda items and project progress.

[1241] A "knowledge base" is a database where important collected information is organized and stored.

[1242] "Generative artificial intelligence" is an artificial intelligence technology used to automatically generate handover documents using a knowledge base as training data.

[1243] A "handover document" is a document used to explain the content and progress of a task to a new person in charge when the person in charge of that task changes.

[1244] A "smart device" is a device that has voice command or code scanning capabilities, and includes smart glasses and smartphones.

[1245] A system implementing this invention uses multiple hardware and software components to automate multi-stage data processing and generate accurate and efficient handover documents.

[1246] Hardware and software configuration

[1247] 1. User terminal:

[1248] Smart devices: These devices, particularly smart glasses and smartphones, incorporate voice commands and code scanning capabilities. This allows users to easily specify where data is stored. Examples include the Vuzix Blade and Google Glass smart glasses.

[1249] 2. Server:

[1250] Cloud storage APIs: Use APIs such as Google Drive API and Dropbox API to collect data from storage locations specified by the user.

[1251] Data Cleansing Module: Uses Python scripts and the Pandas library to cleanse collected data and remove unnecessary information.

[1252] Data Format Unification Module: Use Python scripts and other data conversion tools to convert data in various formats into a unified format.

[1253] Knowledge base building module: Extracts important information and stores it as a knowledge base in an SQL database or Elasticsearch.

[1254] Generative artificial intelligence: AI models that are trained using knowledge base information, such as GPT-3 and BERT, using TensorFlow or PyTorch.

[1255] Processing flow

[1256] 1. Data collection phase:

[1257] Users use smart glasses to issue voice commands or scan QR codes to specify the data storage location.

[1258] The server uses a cloud storage API to collect data from a specified storage location.

[1259] 2. Data preprocessing phase:

[1260] The server's data cleansing module removes unwanted information from the collected data. For example, it filters out spam emails.

[1261] The data format unification module converts data in different formats into a consistent format. For example, it can format various types of meeting logs into a single template.

[1262] 3. Knowledge Base Construction Phase:

[1263] The server extracts important information from pre-processed data and builds a knowledge base. This knowledge base includes project overviews, progress, and contact information for team members.

[1264] 4. AI Learning Phase:

[1265] The server-based knowledge base is input into a generative artificial intelligence system for training. TensorFlow or PyTorch are used.

[1266] 5. Handover document generation phase:

[1267] The server automatically generates the handover document using a pre-trained generative artificial intelligence model.

[1268] The user reviews the handover document generated on the smart glasses and makes corrections as needed.

[1269] Specific example

[1270] For example, consider a factory worker using this system during a shift change. The worker wears smart glasses and gives a voice command such as "Start acquiring meeting logs." The server then collects the necessary data from cloud storage, cleanses it, and standardizes the format. After that, it builds a knowledge base and generates a handover document using generative artificial intelligence. The worker reviews the handover document on the smart glasses' display and makes corrections as needed using voice input.

[1271] Example of a prompt:

[1272] "Starting to retrieve meeting logs"

[1273] Thus, a system implementing this invention significantly reduces the burden on users and enables efficient and accurate handover of tasks.

[1274] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1275] Step 1:

[1276] The user specifies the data storage location using a smart device via voice command or code scan. The input is the voice command or code recognition result. The output is information about the specified storage location.

[1277] Step 2:

[1278] The server uses a cloud storage API to collect data from a specified storage location. The input is information about the storage location specified by the user. The output is the collected raw data.

[1279] Step 3:

[1280] The server uses a data cleansing module to remove unwanted information from the collected raw data. The input is the collected raw data. The output is the cleansed data. Specifically, the server applies a spam filtering algorithm to remove spam data and filters out irrelevant information.

[1281] Step 4:

[1282] The server uses a data format unification module to convert the cleaned data into a consistent format. The input is the cleaned data. The output is the formatted data. Specifically, it formats meeting logs in different formats into a single template.

[1283] Step 5:

[1284] The server extracts important information from pre-processed data and builds a knowledge base. The input is data in a standardized format. The output is information from the knowledge base. Specifically, the server uses natural language processing algorithms to extract keywords and important topics from each data point and stores them in the database.

[1285] Step 6:

[1286] The server inputs the constructed knowledge base into a generative artificial intelligence and performs training. The input is the information from the knowledge base. The output is the trained AI model. Specifically, the server uses TensorFlow or PyTorch to learn patterns and templates for generating handover documents.

[1287] Step 7:

[1288] The server automatically generates handover documents using a pre-trained generative artificial intelligence model. The input is information from the knowledge base and the trained AI model. The output is the generated handover document. Specifically, the AI ​​model integrates information such as project objectives, progress, and details of assigned members to create the handover document.

[1289] Step 8:

[1290] The user reviews the generated handover document on their smart device and makes corrections as needed. The input is the generated handover document. The output is the reviewed and corrected handover document. Specifically, the user reviews the content and corrects any errors or omissions using voice commands or the input functions of their smart device.

[1291] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1292] This invention relates to a system in which a user specifies a location for storing business-related data, a server collects the data from that location, cleanses and standardizes the format, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. This system incorporates an emotion engine that recognizes the user's emotions and provides appropriate feedback.

[1293] System operation description

[1294] 1. Data Collection Phase

[1295] The user logs into the system and specifies where their work-related data is stored. For example, the user might set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[1296] The server automatically collects data from storage locations specified by the user. It efficiently retrieves data using APIs for each storage location.

[1297] 2. Data preprocessing phase

[1298] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[1299] The server converts the cleansed data into a unified format. For example, it formats various types of meeting logs into a consistent template format.

[1300] 3. Knowledge Base Construction Phase

[1301] The server extracts important information from pre-processed data. For example, it extracts key topics, task progress, and important decisions from email content.

[1302] The server builds a knowledge base based on the extracted information. The knowledge base includes project overviews, ongoing tasks, contact information for assigned personnel, etc.

[1303] 4. AI Learning Phase

[1304] The server inputs the knowledge base into the generative artificial intelligence (AI) and trains it. The generative AI learns patterns and templates for generating handover documents from the information in the knowledge base.

[1305] 5. Handover document generation phase

[1306] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes the project objectives, progress, details of the team members involved, and future tasks.

[1307] The user reviews the generated handover document and makes corrections as needed. For example, the user may add explanations to parts that are unclear.

[1308] 6. Emotional Engine Phase

[1309] When a user reviews a handover document, the emotion engine analyzes the user's voice and facial expressions to identify their emotional state. For example, if the user is confused, the emotion engine recognizes this and provides feedback.

[1310] The server readjusts or supplements the contents of the handover document based on the emotional state identified by the emotion engine. For example, if the user is feeling anxious about a certain item, detailed supplementary explanations will be added to that section.

[1311] Specific example

[1312] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[1313] Email storage location: Email service A

[1314] Location where meeting logs are stored: Cloud storage service B

[1315] Document storage location: Internal file server C

[1316] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. Next, it extracts important information from the cleansed data and builds a knowledge base. The built knowledge base is then input into a generative artificial intelligence for training.

[1317] Once the generative artificial intelligence model has completed its training, it automatically generates a handover document. This document includes an overview of the project, ongoing tasks, and contact information for each person in charge. The project manager reviews this document, and the emotion engine recognizes the project manager's facial expressions and voice to analyze their emotional state. For example, if anxiety or doubt is detected, the emotion engine senses this and automatically adds supplementary explanations or details to the relevant sections of the handover document.

[1318] Based on the finalized handover document, users can smoothly complete the handover of their duties. This improves the efficiency of the handover process and reduces stress.

[1319] The following describes the processing flow.

[1320] Step 1:

[1321] The user logs into the system and specifies where work-related data is stored. For example, the user might set "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company's internal file server C."

[1322] Step 2:

[1323] The server accesses the storage location specified by the user and collects data. It uses the API of email service A to retrieve email data, the API of cloud storage service B to download meeting logs, and collects documents from the company's internal file server C.

[1324] Step 3:

[1325] The server cleanses the collected data. For example, the server performs text processing to remove spam emails and delete unnecessary information from meeting logs.

[1326] Step 4:

[1327] The server converts the cleansed data into a unified format. For example, it might format meeting logs into a consistent template format and convert email data into JSON format.

[1328] Step 5:

[1329] The server extracts important information from formatted data. For example, it extracts key topics, task progress, and important decisions from email content.

[1330] Step 6:

[1331] The server builds a knowledge base based on the extracted information. The knowledge base integrates project overviews, ongoing tasks, and contact information for assigned personnel.

[1332] Step 7:

[1333] The server inputs the built knowledge base into a generative artificial intelligence (AI) system to train the AI ​​model. The AI ​​model learns patterns for generating handover documents using the knowledge base.

[1334] Step 8:

[1335] The server uses a pre-trained AI model to automatically generate a handover document. The generated document includes the project objectives, progress, details of the team members involved, and future tasks.

[1336] Step 9:

[1337] The user views and reviews the handover document. At this time, the emotion engine activates, analyzing the user's voice and facial expressions to determine their emotional state. For example, if the user is confused, the emotion engine recognizes this from their facial expressions and tone of voice.

[1338] Step 10:

[1339] The server receives feedback from the emotion engine and readjusts or supplements the handover document as needed. For example, it might add detailed explanations to sections where the user expressed anxiety or doubt.

[1340] Step 11:

[1341] The user reviews the final handover document and makes manual corrections as needed. Once the user is satisfied with all the content, they declare the handover document complete and provide it to the next person in charge.

[1342] Through the steps described above, the system of the present invention can streamline the user handover process and improve the quality of the handover.

[1343] (Example 2)

[1344] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1345] During the handover of work responsibilities, organizing data and extracting information is time-consuming and laborious, making efficient handover difficult. Furthermore, the lack of feedback functions that consider user emotions prevents stress reduction. As a result, deficiencies in handover content and insufficient communication often occur.

[1346] In Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, means for the sentiment analysis engine to analyze the user's emotions and provide feedback, and means for the server to readjust or supplement the contents of the handover document based on the results of the sentiment analysis engine. As a result, a series of processes from automatic data collection, cleansing, and information extraction to the generation of the handover document and feedback based on the user's emotions are automated, enabling an efficient and user-friendly handover.

[1347] A "user" is a person or entity whose role is to access the system, specify the location where business-related data is stored, and review and modify the generated handover documents.

[1348] A "server" is a computer system that collects data from a storage location specified by the user, cleanses and standardizes the format of the collected data, extracts important information, and builds a knowledge base.

[1349] "Storage location" refers to a physical or cloud storage system where business-related data is stored.

[1350] "Cleansing" is the process of removing unnecessary information and noise from collected data.

[1351] "Format unification" is the process of converting data stored in different formats into a consistent, standardized format.

[1352] "Important information" refers to information that is particularly valuable in business operations and is necessary for building knowledge bases and generating handover documents.

[1353] A "knowledge base" is a database that systematically organizes and stores important extracted information.

[1354] "Generative artificial intelligence" refers to artificial intelligence that has the ability to generate new information or documents from given data.

[1355] "Training" is the process by which a generative artificial intelligence learns from data in a given knowledge base and builds a model for generating handover documents.

[1356] A "handover document" is a document used for handing over work responsibilities, including the project's objectives, progress, details of the person in charge, and future tasks.

[1357] An "emotion analysis engine" is a system that analyzes a user's voice and facial expressions to identify their emotional state.

[1358] "Feedback" refers to instructions and supplementary information provided based on the user's emotional state identified by the emotion analysis engine.

[1359] This invention is a system in which the user specifies a location for storing business-related data, and a server collects, cleanses, and standardizes the data from that location, extracts important information, and builds a knowledge base. Furthermore, the server inputs the knowledge base into a generative artificial intelligence, trains it, and generates a handover document. The user then reviews the generated handover document and makes corrections as needed. An emotion analysis engine is also incorporated to recognize the user's emotions and provide appropriate feedback.

[1360] System Configuration

[1361] Data collection

[1362] The user logs into the system and specifies where work-related data is stored. For example, a user can set it up so that "emails are stored in email service A, meeting logs in cloud storage service B, and documents in the company file server C." The server retrieves the data from the specified storage locations using an API.

[1363] Data preprocessing

[1364] The server cleanses the acquired data. This phase involves removing spam emails and unnecessary information from meeting logs. In particular, a text analysis engine is used for data cleansing. Next, the server converts the cleansed data into a unified format, shaping various data formats into a consistent template.

[1365] Building a knowledge base

[1366] The server extracts key information from pre-processed data and builds a knowledge base. This extracted information includes, for example, the main topics of emails, task progress, and important decisions. The resulting knowledge base includes project summaries, ongoing tasks, and contact information for those responsible.

[1367] AI learning

[1368] The server inputs data from the knowledge base into a generative artificial intelligence (AI) and performs training. The generative AI used for training utilizes various machine learning libraries (e.g., TensorFlow, PyTorch) to learn patterns and templates for generating handover documents.

[1369] Handover document generation

[1370] Using a completed generative artificial intelligence model, the server automatically generates a handover document. The generated document details the project's objectives, progress, team members, and future tasks. The user reviews this generated document and makes any necessary modifications.

[1371] Emotion analysis

[1372] When a user reviews a handover document, an emotion analysis engine analyzes the user's voice and facial expressions to identify their emotional state. Emotion analysis is performed in real time; for example, if the user is confused, the emotion analysis engine recognizes this, and the server automatically adds detailed supplementary explanations to the relevant section.

[1373] Specific example

[1374] For example, suppose a project manager at a certain company uses this system when handing over responsibilities. The project manager logs into the system and configures it as follows:

[1375] Email storage location: Email service A

[1376] Location where meeting logs are stored: Cloud storage service B

[1377] Document storage location: Internal file server C

[1378] The server collects data from these storage locations, removes spam emails, deletes unnecessary parts of meeting logs, and standardizes data formats. It then extracts important information from the cleansed data to build a knowledge base. This knowledge base is input into a generative artificial intelligence (AI) model for training. The improved AI model automatically generates a handover document. The project manager reviews this document, and an emotion analysis engine recognizes the project manager's emotions through facial expressions and voice, providing supplementary explanations where necessary.

[1379] Example of a prompt

[1380] "Please create a detailed handover document for the next project you will be taking over. Include the project's objectives, progress, details of the person in charge, and future tasks. Also, ensure that appropriate feedback is provided regarding the user's emotional state."

[1381] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1382] Step 1:

[1383] The user logs into the system and specifies the location where work-related data will be stored.

[1384] Input: Specify the user's login information and data storage location (e.g., email service, cloud storage, internal file server).

[1385] Specific operation: The user enters "Email Service A" as the email storage location, "Cloud Storage B" as the meeting log storage location, and "Internal File Server C" as the document storage location into the interface's input fields.

[1386] Output: The specified save location information is sent to the server and recorded.

[1387] Step 2:

[1388] The server automatically collects data from a storage location specified by the user.

[1389] Input: Information about the save location specified by the user.

[1390] Specific operation: The server calls the storage location API to retrieve email data from email service A, meeting log data from cloud storage B, and document data from the internal file server C.

[1391] Output: The collected raw data is stored on the server.

[1392] Step 3:

[1393] The server cleanses the collected data.

[1394] Input: The collected raw data.

[1395] Specific operation: The server uses a text analysis engine to remove spam emails and unnecessary information from meeting logs. For example, it uses an NLP model to perform spam filtering and remove unnecessary words such as "um" and "uh" from meeting logs.

[1396] Output: Cleaned, clean data.

[1397] Step 4:

[1398] The server converts the cleansed data into a unified format.

[1399] Input: Cleansed, clean data.

[1400] Specific operation: The server executes a script to convert various data formats into a consistent template format. For example, it formats meeting logs in different formats into a single unified template.

[1401] Output: Data in a standardized format.

[1402] Step 5:

[1403] The server extracts important information from the pre-processed data.

[1404] Input: Data in a standardized format.

[1405] Specific operation: The server applies an information extraction algorithm to extract the main topics, task progress, and important decisions from emails. For example, it might extract "next action points" or "decisions" from meeting logs.

[1406] Output: Extracted key information.

[1407] Step 6:

[1408] The server builds a knowledge base based on the extracted information.

[1409] Input: Extracted key information.

[1410] Specific operation: The server organizes the information extracted into the database and stores project summaries, ongoing tasks, and contact information for assigned personnel.

[1411] Output: The constructed knowledge base.

[1412] Step 7:

[1413] The server inputs the knowledge base into the generative artificial intelligence and trains it.

[1414] Input: Knowledge base.

[1415] Specific operation: The server supplies knowledge base information to a generative artificial intelligence model and performs training. Machine learning libraries such as TensorFlow and PyTorch are used to train the model.

[1416] Output: A trained generative artificial intelligence model.

[1417] Step 8:

[1418] The server automatically generates the handover document using a pre-trained generative artificial intelligence model.

[1419] Input: Trained generative artificial intelligence models and knowledge base data.

[1420] Specific operation: The server generates a handover document using a generative artificial intelligence model. The handover document will include the project's objectives, progress, details of the team members in charge, and future tasks.

[1421] Output: Automated handover document.

[1422] Step 9:

[1423] The user reviews the generated handover document and makes corrections as needed.

[1424] Input: Automated handover document.

[1425] Specific operation: The user reviews the contents of the handover document on the interface and edits any unclear parts or sections that require additional explanation.

[1426] Output: Revised final version of the handover document.

[1427] Step 10:

[1428] When a user reviews a handover document, an emotion analysis engine analyzes the user's emotions and provides feedback.

[1429] Input: Voice and facial expression data of the user reviewing the handover document.

[1430] Specific operation: The emotion analysis engine analyzes the user's emotions in real time, identifying feelings of confusion, anxiety, etc. Based on this, it provides feedback.

[1431] Output: User sentiment analysis results and provided feedback.

[1432] Step 11:

[1433] The server readjusts or supplements the contents of the handover document based on the results of the emotion analysis engine.

[1434] Input: Results from the emotion analysis engine.

[1435] Specific operation: The server adjusts the relevant sections of the handover document based on the user's feelings and adds necessary supplementary explanations. For example, it adds detailed explanations to sections where the user felt uneasy.

[1436] Output: Adjusted and supplemented handover documents.

[1437] (Application Example 2)

[1438] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1439] In today's work environment, information management and handover processes are becoming increasingly complex. In particular, information regarding the maintenance and operation of factory robots is diverse, making it difficult for new engineers to smoothly take over the work. Typical handover documents are prone to missing or inconsistent information, reducing the efficiency of training. Furthermore, there is a lack of means to alleviate the stress and anxiety engineers experience while reviewing handover documents. It is necessary to address these problems.

[1440] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for the user to specify a storage location for business-related data, means for the server to collect data from the storage location, means for the server to cleanse and standardize the format of the collected data, means for the server to extract important information from the cleansed and standardized data and build a knowledge base, means for the server to input the knowledge base into a generative artificial intelligence and train it, means for the server to generate a handover document using the trained generative artificial intelligence model, means for the user to review and correct the generated handover document, and sentiment analysis means to analyze the user's emotions during the review of the handover document and provide appropriate feedback. This makes it possible to perform an efficient and detailed handover while reducing the anxiety and confusion felt by the technician.

[1441] A "user" is an individual or legal entity that uses the system to specify, verify, or modify business data.

[1442] "Business data" refers to information such as emails, meeting logs, and documents necessary for the progress of work.

[1443] "Storage location" refers to the storage medium, such as a server, cloud storage, or local storage, where business data is stored.

[1444] A "server" is a central processing unit that collects, cleanses, and formats data, builds a knowledge base, trains generative artificial intelligence, and generates handover documents.

[1445] "Data collection" refers to retrieving specified business data from a storage location.

[1446] "Cleaning" is the process of removing unnecessary information from collected data to improve its quality.

[1447] "Format unification" is the process of standardizing data stored in different formats into a consistent format.

[1448] A "knowledge base" is a database that stores important information extracted from cleansed and formatted data.

[1449] "Generative artificial intelligence" refers to an artificial intelligence model that learns from a knowledge base and has the function of generating specific outputs (for example, handover documents).

[1450] A "handover document" is a document used for handing over work, which includes details of the work, progress, and contact information of the person in charge.

[1451] "Emotional analysis" is a process that provides appropriate feedback by analyzing the user's emotional state.

[1452] Basic System Configuration

[1453] A system for implementing this invention includes the following main elements:

[1454] 1. Method for specifying data storage location: The user logs into the system and specifies the storage location for business data (e.g., email service, cloud storage, local file server).

[1455] 2. Data Collection Method: The server collects data from designated storage locations. It efficiently retrieves data using the API of each storage location.

[1456] 3. Data cleansing and formatting unification means: The server cleanses the collected data (e.g., removes unnecessary information) and converts the data into a consistent format.

[1457] 4. Knowledge Base Construction Method: The server extracts important information from cleansed and formatted data to build a knowledge base. The knowledge base includes project overviews, ongoing tasks, and contact information for assigned personnel.

[1458] 5. Generative AI Training Method: The server inputs the knowledge base into the generative AI and performs training.

[1459] 6. Handover document generation method: The server automatically generates the handover document using a trained generative artificial intelligence model.

[1460] 7. User Verification and Correction Method: The user will review the generated handover document and make corrections as necessary.

[1461] 8. Emotion Analysis Method: Analyzes the user's facial expressions and voice while reviewing the handover document, and provides feedback based on the user's emotional state.

[1462] Hardware and software configuration

[1463] Hardware: Servers, user terminals (PCs or mobile devices), sensors (cameras and microphones).

[1464] Software: API communication software (REST API, etc.), text processing software (Python libraries), data cleansing software (Pandas), generative artificial intelligence (OpenAI's GPT, etc.), sentiment analysis software (EmotionAnalyzer).

[1465] Specific example

[1466] For example, consider using this invention in a system to support the maintenance and operation of robots used in a factory. When a new technician takes over a task, this system is used to automatically collect data such as emails, maintenance logs, and work procedures left by the previous technician. The collected data is cleansed and standardized in format. A knowledge base is built, and this information is used to train generative artificial intelligence. The technician reviews the automatically generated handover document, and if sentiment analysis indicates confusion or anxiety, the system provides appropriate feedback.

[1467] Example of a prompt

[1468] Design a system for new technicians to take over factory robot maintenance tasks. This system will extract key information from the predecessor's emails, maintenance logs, and work procedures, and automatically generate a handover document. It will also include a function to analyze the technician's emotions and add detailed supplementary explanations if they are feeling anxious or confused.

[1469] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1470] Step 1:

[1471] The user logs into the system and specifies the location where work-related data is stored. In this step, the user can specify an email service, cloud storage, local file server, etc. The input is the user's specified storage location information, and the output is a list of the specified storage locations.

[1472] Step 2:

[1473] The server collects data from a storage location specified by the user. It accesses the storage location and retrieves the data using APIs or the file system. The input is a list of storage locations, and the output is a collection of the collected original data.

[1474] Step 3:

[1475] The server cleanses the collected data. It performs text processing to remove spam emails and unwanted information from meeting logs. The input is the original collected data, and the output is the cleansed data.

[1476] Step 4:

[1477] The server cleanses the data and standardizes its format. It converts data in different formats into a unified format. The input is cleansed data, and the output is data with a unified format.

[1478] Step 5:

[1479] The server extracts important information from standardized data and builds a knowledge base. For example, it extracts key topics, task progress, and important decisions from email content. The input is standardized data, and the output is a knowledge base.

[1480] Step 6:

[1481] The server inputs the knowledge base into a generative artificial intelligence (AI) model and trains it. It learns patterns and templates for generating handover documents from the information in the knowledge base. The input is the knowledge base, and the output is the trained generative AI model.

[1482] Step 7:

[1483] The server automatically generates a handover document using a pre-trained generative artificial intelligence model. The generated handover document includes details of the task, progress, and contact information of the person in charge. The input is the pre-trained generative artificial intelligence model, and the output is the handover document.

[1484] Step 8:

[1485] The user reviews the generated handover document and makes corrections as needed. They access the handover document using a terminal and modify the necessary parts. The input is the handover document, and the output is the corrected handover document.

[1486] Step 9:

[1487] While the server is verifying the handover document, it runs an emotion analysis engine to analyze the user's emotions. It analyzes voice and facial expression data to identify the user's emotional state. The input is the user's voice and facial expression data, and the output is the user's emotional state.

[1488] Step 10:

[1489] The server readjusts or supplements the handover document based on the user's emotional state identified through sentiment analysis. Detailed explanations are added to sections where the user felt confused or anxious. The input is the user's emotional state and the handover document; the output is the supplemented handover document.

[1490] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1491] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1492] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1493] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1494] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. In the upper and lower directions of the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. Also, the upper side of the concentric circles is where "pleasant" emotions are located, and the lower side is where "unpleasant" emotions are located. In this way, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1495] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1496] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1497] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1498] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1499] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1500] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1501] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1502] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1503] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1504] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1505] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1506] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1507] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1508] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1509] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1510] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted as being incorporated by reference.

[1511] The following is further disclosed regarding the embodiments described above.

[1512] (Claim 1)

[1513] A means for users to specify the location where business-related data is stored,

[1514] The means by which the server collects data from the storage location,

[1515] The server provides a means for cleansing and standardizing the format of the collected data,

[1516] A server extracts important information from cleansed and formatted data and provides a means for building a knowledge base.

[1517] A server provides a means for inputting a knowledge base into a generative artificial intelligence and training it,

[1518] A means for a server to generate a handover document using a trained generative artificial intelligence model,

[1519] A means for users to review and modify the generated handover document,

[1520] A system that includes this.

[1521] (Claim 2)

[1522] The system according to claim 1, wherein the server filters collected data for spam using a machine learning algorithm, and then cleanses and formats it.

[1523] (Claim 3)

[1524] The system according to claim 1, in which the server, when building a knowledge base, includes project summaries, progress, and contact information for team members in the database.

[1525] "Example 1"

[1526] (Claim 1)

[1527] A means for users to specify the location where business-related data is stored,

[1528] A means for the server to collect data from storage locations using an API,

[1529] A means for the server to cleanse and standardize the format of the collected data using a Python data processing library,

[1530] A means by which a server uses a natural language processing library to extract important information from cleansed and formatted data, and stores it in a relational database to build a knowledge base,

[1531] A server inputs a knowledge base into a generative artificial intelligence and uses an API to train it,

[1532] A server provides a means for generating a handover document based on a prompt statement using a trained generative artificial intelligence model,

[1533] A means for users to review and modify the generated handover document via a web interface,

[1534] A system that includes this.

[1535] (Claim 2)

[1536] The system according to claim 1, wherein the server filters collected data for spam using a machine learning algorithm, and then cleanses and formats it.

[1537] (Claim 3)

[1538] The system according to claim 1, in which the server, when building a knowledge base, includes project summaries, progress, and contact information for team members in the database.

[1539] "Application Example 1"

[1540] (Claim 1)

[1541] A means for users to specify the location where business-related data is stored,

[1542] The means by which the server collects data from the storage location,

[1543] The server provides a means for cleansing and standardizing the format of the collected data,

[1544] A server extracts important information from cleansed and formatted data and provides a means for building a knowledge base.

[1545] A server provides a means for inputting a knowledge base into a generative artificial intelligence and training it,

[1546] A means for a server to generate a handover document using a trained generative artificial intelligence model,

[1547] A means for users to review and modify the generated handover document,

[1548] A means of specifying the data storage location using a smart device via voice command or code scan,

[1549] A system that includes this.

[1550] (Claim 2)

[1551] The system according to claim 1, wherein the server filters collected data for spam using a machine learning algorithm, and then cleanses and formats it.

[1552] (Claim 3)

[1553] The system according to claim 1, in which the server, when building a knowledge base, includes project summaries, progress, and contact information for team members in the database.

[1554] "Example 2 of combining an emotion engine"

[1555] (Claim 1)

[1556] A means for users to specify the location where business-related data is stored,

[1557] The means by which the server collects data from the storage location,

[1558] The server provides a means for cleansing and standardizing the format of the collected data,

[1559] A server extracts important information from cleansed and formatted data and provides a means for building a knowledge base.

[1560] A server provides a means for inputting a knowledge base into a generative artificial intelligence and training it,

[1561] A means for a server to generate a handover document using a trained generative artificial intelligence model,

[1562] A means for users to review and modify the generated handover document,

[1563] A means by which an emotion analysis engine analyzes the user's emotions and provides feedback,

[1564] The server has a means to readjust or supplement the contents of the handover document based on the results of the emotion analysis engine,

[1565] A system that includes this.

[1566] (Claim 2)

[1567] The system according to claim 1, wherein the server filters collected data for spam using a machine learning algorithm, and then cleanses and formats it.

[1568] (Claim 3)

[1569] The system according to claim 1, in which the server, when building a knowledge base, includes project summaries, progress, and contact information for team members in the database.

[1570] "Application example 2 when combining with an emotional engine"

[1571] (Claim 1)

[1572] A means for users to specify the location where business-related data is stored,

[1573] The means by which the server collects data from the storage location,

[1574] The server provides a means for cleansing and standardizing the format of the collected data,

[1575] A server extracts important information from cleansed and formatted data and provides a means for building a knowledge base.

[1576] A server provides a means for inputting a knowledge base into a generative artificial intelligence and training it,

[1577] A means for a server to generate a handover document using a trained generative artificial intelligence model,

[1578] A means for users to review and modify the generated handover document,

[1579] A sentiment analysis tool that analyzes the user's emotions while reviewing the handover document and provides appropriate feedback,

[1580] A system that includes this.

[1581] (Claim 2)

[1582] The system according to claim 1, wherein the server filters collected data for spam using a machine learning algorithm, and then cleanses and formats it.

[1583] (Claim 3)

[1584] The system according to claim 1, in which the server, when building a knowledge base, includes project summaries, progress, and contact information for team members in the database. [Explanation of symbols]

[1585] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for users to specify the location where business-related data is stored, The means by which the server collects data from the storage location, The server provides a means for cleansing and standardizing the format of the collected data, A server extracts important information from cleansed and formatted data and provides a means for building a knowledge base. A server provides a means for inputting a knowledge base into a generative artificial intelligence and training it, A means for a server to generate a handover document using a trained generative artificial intelligence model, A means for users to review and modify the generated handover document, A system that includes this.

2. The system according to claim 1, wherein the server filters collected data for spam using a machine learning algorithm, and then cleanses and formats the data.

3. The system according to claim 1, in which the server, when building a knowledge base, includes project overviews, progress status, and contact information for team members in the database.

Images