system

Abstract

Provide a system. A method for collecting business-related data; A means of integrating and learning from collected business-related data; A means for providing relevant information from the trained model in response to a user request; A system including:

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In business, employee transfers and retirements are inevitable, and the task of handing over work that occurs at such times requires manpower and depends on the organization of related information and the accuracy of the handover. This can disrupt business continuity and reduce work efficiency. In particular, the task of sorting through the large amount of work-related data (emails, chats, documents, etc.) that accumulates daily and extracting the information necessary for the handover is cumbersome. There is a need for a system that can improve this situation and make work handovers more efficient. [Means for solving the problem]

[0005] The present invention provides a system that improves the efficiency of business handover following employee transfers or retirement by collecting business-related data and training it using natural language processing technology. The present invention is a system that includes the following means: a means for collecting business-related data, a means for integrating and training the collected business-related data, and a means for providing related information from a trained model in response to a user request. This system significantly reduces the work involved in business handover and improves the continuity and efficiency of business operations. Furthermore, the present invention accommodates a wide range of business operations by including email data, chat data, and document data as business-related data. Furthermore, by providing a means for periodically collecting business-related data, it is possible to provide information based on the latest data at all times.

[0006] "Business-related data" refers to all data related to business operations occurring within a company, and specifically includes email data, chat data, document data, and the like.

[0007] "Collection methods" refers to the methods and techniques used to obtain data for a specific purpose, including processes for obtaining data from email servers, communication tools, and file storage systems.

[0008] "Means of integrating and training" refers to methods and technologies for consolidating collected data and training an AI model using natural language processing technology, etc.

[0009] A "user request" refers to an operation or request by a user to request the system to provide information, and specifically includes a request for information about a project.

[0010] "Trained Model" refers to an AI model that has been trained using previously collected data and has the ability to recognize and provide specific business-related information.

[0011] "Means for providing relevant information" refers to the methods and technologies that enable an AI model to extract and provide necessary information from trained data in response to a user request.

[0012] "Email Data" refers to the content of emails sent and received within an enterprise, including all business-related text, attachments, and other metadata.

[0013] "Chat Data" refers to the content of messages exchanged through communication tools, including all chat logs and related metadata related to business.

[0014] "Document data" refers to the content of documents and files created and shared for business purposes, including text files, spreadsheets, presentation materials, etc.

[0015] "Natural language processing technology" refers to technology that enables computers to understand and process human language, and specifically includes text analysis, sentiment analysis, document classification, and other related technologies.

[0016] "Means of collecting business-related data on a regular basis" refers to methods or techniques that automatically collect data at regular intervals, including, for example, daily or weekly data collection processes. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5]FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] The system of this invention allows the server to collect business-related data, integrate it, and learn from it to quickly and accurately provide the business information required by the user. By implementing this system, even in the event of transfer or retirement, business handover can be carried out smoothly, improving the continuity and efficiency of business operations.

[0039] This system mainly consists of the following steps: collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[0040] Collection of work-related data

[0041] The server first connects to the company's email server and collects all emails sent and received by employees for work, then connects to communication tools (e.g., chat apps) and collects all chat logs related to work, and finally connects to the file storage system and collects all documents related to work.

[0042] Examples:

[0043] The server connects to the email server every night at midnight and collects new mail for that day.

[0044] The server connects to the chat tool used by the company at regular intervals (for example, every hour) and retrieves new chat messages sent and received during that time.

[0045] Data Integration and Learning

[0046] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model (natural language processing model) for learning. The server automates this process and periodically updates the learning model.

[0047] Examples:

[0048] The server compiles the latest email data, chat data, and document data into a single database on a daily basis.

[0049] Using this database, the AI ​​model is trained overnight, and by the next morning a model reflecting the latest information is completed.

[0050] Providing information in response to user requests

[0051] When a user needs specific information for work handover or information understanding, the user sends a request to the server via the device, and the server uses the trained model to quickly extract information relevant to the user's request and provide it to the device.

[0052] Examples:

[0053] A user who has been newly assigned to a project requests on their device, "Show me all emails and important chat logs related to Project A from the past three months."

[0054] The server uses the trained model to search for relevant emails and chat logs, extract that information, and display it on the device.

[0055] Summary of embodiments

[0056] The system of this invention combines three major steps: automatic collection of business-related data, data integration and learning, and information provision based on user requests. This significantly simplifies the traditional manual process of business handover, and enables smooth business continuity even in the event of transfers or retirement. Furthermore, by periodically learning the latest data, the system can always provide the latest information.

[0057] The processing flow will be explained below.

[0058] Step 1:

[0059] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts their contents into plain text and stores them in a database. It also deduplicates data already collected in the past.

[0060] Step 2:

[0061] The server connects to the company's communication tools (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, it converts the content into text format and stores it in a database. It then compares it with past logs to incorporate only new data, improving efficiency.

[0062] Step 3:

[0063] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, deleted, or updated by employees. When it finds new or changed documents, it saves the contents to the database in text format or raw format.

[0064] Step 4:

[0065] The server consolidates the collected email, chat, and document data, which involves combining all new and updated data in the database into one unified data set.

[0066] Step 5:

[0067] The server feeds the integrated dataset to an AI model (natural language processing model) to train the model, which learns the details of the business and project and prepares for the next user request.

[0068] Step 6:

[0069] When a user takes over a new project, he or she inputs specific information (e.g., project name, period for which information is desired) from the terminal and sends a request to the server.

[0070] Step 7:

[0071] The server feeds the user's request to the trained model and retrieves the relevant information that the model returns, including emails, chat messages, and documents related to the specified project and time period.

[0072] Step 8:

[0073] The server formats the acquired information and converts it into a format that is easy for the user to understand. The information is sent to the device and displayed in an appropriate format (e.g., list format, timeline format).

[0074] Step 9:

[0075] The user can check the information provided on the terminal and smoothly carry out the necessary handover and work. If there are any further questions or additional requests, they can send a request to the server again using the same procedure.

[0076] Example 1

[0077] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0078] The current business handover process relies on manual work and individual knowledge, resulting in low efficiency and causing problems with business continuity when employees are transferred or leave the company. Furthermore, business-related data exists in a wide variety of formats, making it difficult to quickly and accurately extract the necessary information. This negatively impacts the efficiency and completeness of business operations. Furthermore, the lack of an effective means to quickly obtain the information necessary for business operations can delay the entire business process.

[0079] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0080] In this invention, the server includes means for acquiring collected business-related data at a scheduled time, means for integrating the acquired business-related data into a database, means for training a natural language processing model using the integrated data, means for receiving a request in the form of a prompt sentence from a user, means for searching for information based on the user request using the trained model, and means for providing the search results to a user terminal. This enables efficient collection, integration, and training of business-related data and rapid information provision. As a result, smooth business handover and improved business continuity and efficiency are achieved.

[0081] "Collected business-related data" is a set of data related to a company's business activities, including email data, chat data, and file storage data.

[0082] The "measure for acquiring data at a fixed time" is a mechanism by which the server automatically collects business-related data at a preset time.

[0083] "Means of integrating into a database" refers to the process of compiling and integrating various collected data into a single database.

[0084] "Means for training a natural language processing model" refers to a method for using integrated data to train an AI model to perform natural language processing.

[0085] The "means for receiving a request in the form of a prompt sentence" is an interface for receiving a request input by a user in a specific format and transmitting it to a server.

[0086] "Means for searching for information based on a user request using a trained model" is a process of searching for relevant information in response to a user request using an already trained natural language processing model.

[0087] The "means for providing search results to a user terminal" refers to a mechanism by which the server transmits the search results to the user terminal and provides them in a displayable or usable form.

[0088] This invention is a system in which a server collects business-related data, integrates and learns from it, and quickly and accurately provides the business information required by users. The main functions of this system include collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[0089] Collection of work-related data

[0090] The server first connects to the company's email server and collects all emails sent and received by employees for work. Then, the server connects to communication tools (e.g., chat apps) and collects all chat logs related to work. Furthermore, the server connects to a file storage system and collects all documents related to work. Specifically, the server uses the following hardware and software:

[0091] Email server: Retrieves email data using the IMAP protocol.

[0092] Chat app: Collect chat logs using REST API.

[0093] File storage system: Retrieves documents using the SFTP protocol.

[0094] For example, the server connects to an email server every night at midnight to retrieve new emails for that day. The server also connects to a chat tool used by the company at a fixed time (for example, every hour) to retrieve new chat messages sent and received during that time. For this purpose, a scheduling function (for example, a Cron job) is set up on the server to connect at fixed times.

[0095] Data Integration and Learning

[0096] The server then integrates the collected data into a database. Specifically, the server processes the data as follows:

[0097] Database: A relational database management system such as MySQL (registered trademark) is used, and each piece of data is inserted into the corresponding table.

[0098] ETL scripts: Data mapping and cleansing are performed using temporary tables to ensure data integrity.

[0099] The server then uses the integrated data to train a natural language processing model. Specifically, the server uses the following techniques:

[0100] Python environment: Build and train natural language processing models using TENSORFLOW (registered trademark) and Keras.

[0101] Data preprocessing: Tokenization, text vectorization, etc.

[0102] Save the trained model: Save it to disk in HDF5 format or similar.

[0103] Providing information in response to user requests

[0104] When a user needs specific information for work handover or information gathering, they send a request to the server via their terminal. Specifically, the process is as follows:

[0105] Prompt input: A user accesses a dedicated web application in a browser and inputs a request such as: "Show me all emails and important chat logs related to Project A from the past three months."

[0106] HTTP communication: The terminal sends this prompt text to the server using an HTTP POST request.

[0107] The server uses the trained model to search for relevant information based on the user's request, then sends the search results back to the user's device, where the user can view the results. Specifically, it works as follows:

[0108] Query Generation: A natural language processing model analyzes the user's input and generates an appropriate SQL query.

[0109] Retrieving search results: The server executes the generated query and extracts the relevant data from the database.

[0110] Send results: The search results are converted into JSON format and sent to the device.

[0111] The terminal parses the received JSON data and displays it to the user in a table or report format on the web application.

[0112] As described above, the present invention realizes efficient collection, integration, and learning of business-related data, and rapid information provision, thereby achieving smooth business handover and improved business continuity and efficiency.

[0113] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0114] Step 1: Collect business-related data

[0115] The server connects to the company's email server and collects all emails sent and received by employees for work. The server uses the IMAP protocol to access the mail server and retrieves new emails. Next, the server connects to communication tools (e.g., chat apps) using REST APIs and collects all chat logs related to work. Finally, the server connects to the file storage system using the SFTP protocol and collects all documents related to work.

[0116] Inputs: email servers, chat apps, file storage systems

[0117] Output: New email data, chat log data, document data

[0118] What it does: The server runs a cron job every night at midnight to collect new emails from the email server for the day, connects to the company's chat tool every hour to retrieve new chat messages, and periodically downloads important documents from the file storage system.

[0119] Step 2: Integrate the data

[0120] The server integrates the collected email, chat, and document data into a database. The server connects to a MySQL database and inserts each piece of data into the corresponding table. To maintain data consistency, temporary tables are used, and data mapping and cleaning are performed.

[0121] Input: New email data, chat log data, document data

[0122] Output: Integrated database

[0123] What happens: The server runs the ETL (Extract, Transform, Load) script to insert various data into the database, including storing new email data in the email table, chat logs in the chat table, and document data in the document table.

[0124] Step 3: Train the data

[0125] The server uses the integrated data to train a natural language processing model. The server sets up a Python environment and uses TensorFlow and Keras to build and train the natural language processing model. As data preprocessing, it performs tokenization and text vectorization, and saves the trained model.

[0126] Input: Integrated database

[0127] Output: Trained AI model

[0128] How it works: Every night, the server runs a Python script to train a natural language processing model based on the integrated data. After training is complete, the model is saved in an HDF5 file.

[0129] Step 4: Receiving a User Request

[0130] A user requests specific information for the purpose of task handover or information gathering. The user accesses a dedicated web application and inputs the request in the form of a prompt. The terminal then sends this request to the server using the HTTP POST method.

[0131] Input: A prompt-style request from the user

[0132] Output: HTTP POST request to the server

[0133] Specific operation: A user enters the request "Show me all emails and important chat logs related to Project A from the past three months" into a web form and presses the submit button. The device sends this request to the server.

[0134] Step 5: Search using the trained model

[0135] The server uses the trained model to search for relevant information based on the user's request. A natural language processing model analyzes the user's input and generates an appropriate SQL query. The server executes the query and extracts the relevant data from the database.

[0136] Input: A request in the form of a prompt, a trained AI model

[0137] Output: Search result data

[0138] Specific operation: The server parses the Japanese prompt and generates an SQL query to retrieve email data and chat logs related to "Project A" from the database. The server executes this query to extract the required data.

[0139] Step 6: Serving search results

[0140] The server returns the search results to the user's device. The search results are converted to JSON format and sent as an HTTP response. The device parses the received JSON data and displays it in a table or report format on the web application.

[0141] Input: Search result data

[0142] Output: Data displayed on the user's terminal

[0143] Specific operation: The server converts the search results into JSON format and sends it to the device as an HTTP response. The device then analyzes the received JSON data and displays it to the user on the web application. Specifically, the data is organized in tabular or graph format, allowing the user to see the information they need at a glance.

[0144] (Application example 1)

[0145] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0146] In modern industrial environments, large amounts of data are generated, including equipment operation logs, sensor data, and quality inspection data. Properly managing this data and ensuring smooth handovers when employees are transferred or leave the company are key issues for improving business continuity and efficiency. However, with conventional systems, the collection, integration, and learning of business-related data is often done manually, resulting in poor business efficiency. Furthermore, the inability to provide appropriate information quickly has led to numerous problems with business handover and management.

[0147] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0148] In this invention, the server includes means for collecting business-related data, means for integrating and training the collected business-related data, means for providing related information from a trained model in response to a user request, means for collecting operation logs, sensor data, and quality inspection data from industrial equipment, means for supplying the collected industrial equipment data to a machine learning model for integration and training, and means for using the machine learning model to extract and provide related information in response to inquiries from engineers. This enables the efficient collection and management of large amounts of business-related data in industrial environments, enabling smooth handover in the event of transfers or retirements, as well as improving business continuity and efficiency.

[0149] "Business-related data" refers to data including email data, chat data, document data, and operation logs, sensor data, and quality inspection data generated by industrial equipment when conducting business within a company.

[0150] "Means of collection" refers to the functions and technologies used by servers or other devices to access various data sources via a network and collect the required data.

[0151] "Means of integration and learning" refers to the functions and technologies that centrally manage collected data and supply it to AI models or machine learning models for analysis and learning.

[0152] "Means of providing relevant information from a trained model" refers to functions and technologies that quickly extract and provide necessary information from a pre-trained AI model in response to a user request.

[0153] "Industrial equipment" refers to machines and devices used in manufacturing, testing, processing, or other industrial processes.

[0154] "Operation log" refers to operation history data recorded when industrial equipment is in operation.

[0155] "Sensor data" refers to various measurement data acquired by sensors attached to industrial equipment.

[0156] "Quality inspection data" refers to data acquired by inspection equipment to evaluate the quality of a product or process.

[0157] "Means of supplying data to a machine learning model for integration and learning" refers to the functions and technologies for inputting various collected data into a machine learning algorithm for integration and learning, and extracting useful insights and patterns from the data.

[0158] "Means for extracting and providing relevant information in response to inquiries from engineers" refers to the functions and technologies for extracting appropriate information from collected and learned data based on inquiries from engineers who manage and operate industrial equipment, and providing it to the engineers.

[0159] A system for implementing the present invention utilizes various hardware and software components. Specific embodiments are described below.

[0160] 1. Data Collection Module

[0161] The server collects business-related data such as operation logs of industrial equipment, sensor data, and quality inspection data in real time via the network. For example, it has a function to obtain data from sensors using APIs and record operation logs.

[0162] Specific software used includes a REST API for data acquisition and a database system (e.g., ElasticSearch (registered trademark)). The server periodically collects information from these data sources and stores it in a database.

[0163] 2. Data Integration and Learning Module

[0164] The server centrally manages the various collected data by periodically retrieving data stored in a database system such as Elasticsearch and providing it to the machine learning model.

[0165] The specific software used is the Hugging Face transformers library, which uses natural language processing techniques to analyze the data and periodically train the model.

[0166] 3. User Request Module

[0167] When a user sends a request to the system using an administrative terminal, the server uses a pre-trained machine learning model to quickly search and extract relevant information and provide it to the user.

[0168] For example, if a technician requests "Show me the quality inspection data for the past three months," the server will search for the relevant data in Elasticsearch and provide the relevant information to the technician, enabling them to quickly obtain the information they need and improving work efficiency.

[0169] Examples of prompts:

[0170] "Show me the quality inspection data for the past three months"

[0171] As described above, the embodiment of the present invention is composed of three main modules: data collection, data integration and learning, and user request response, which enable efficient collection and management of industrial equipment data and enable users to quickly obtain the information they need.

[0172] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0173] Step 1: Data collection

[0174] The server accesses various data sources over the network. Specifically, it periodically sends requests to the APIs of industrial equipment to obtain operation logs, sensor data, and quality inspection data. The input is real-time data from the equipment, and the output is the collected data stream. The server converts this data into an appropriate format and stores it in a database.

[0175] Step 2: Data integration

[0176] The server accumulates the collected data in a database such as Elasticsearch. Daily data is linked to create a single centralized database. The input is various business-related data, and the output is the integrated database. Specifically, the server retrieves new data at the end of each day and adds it to the centralized database.

[0177] Step 3: Learning Modules

[0178] The server uses the merged data to train a machine learning model (for example, using Hugging Face's transformers library). The input is the merged database, and the output is the trained model. Specifically, the server periodically retrieves the latest data from the database every night and feeds it to the model for training.

[0179] Step 4: User request reception

[0180] A user uses a terminal to request specific business-related information. The request is entered in natural language. The input is the request sentence from the user, and the output is the analysis result of the request content. In concrete terms, the user enters the request through the terminal interface.

[0181] Step 5: Information Search

[0182] The server uses the trained model to search for relevant information from Elasticsearch based on the user request. The input is the request analysis result, and the output is the searched business-related information. Specifically, the server generates an Elasticsearch query and searches for the appropriate information in the database.

[0183] Step 6: Provide information

[0184] The server organizes the search results and provides them to the user. The input is the business-related information that was searched, and the output is the organized information that is provided to the user. Specifically, the server compactly organizes the search results and displays them on the user's terminal. At this time, the information is converted into the required format according to the content of the request.

[0185] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0186] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[0187] This system consists of the following steps: collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and provision of information in response to user requests.

[0188] Collection of work-related data

[0189] The server connects to the company's email server, communication tools (e.g., chat apps), and file storage system to collect all employees' emails, chat logs, and documents. These data are important sources of information for business activities, and are all stored in a database in text format or in its original form.

[0190] Data Integration and Learning

[0191] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model using natural language processing technology, which learns about the details of the work and the project. This allows it to provide more accurate information in preparation for the next user request.

[0192] Emotion recognition by emotion engine

[0193] The server includes an emotion engine that recognizes the user's emotions from the collected work-related data and user interactions. The emotion engine analyzes text data and voice data and can classify the user's emotional state (e.g., stress, joy, anger, etc.). This emotion data is reflected in the analysis of user requests and the provision of information.

[0194] Examples:

[0195] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[0196] Providing information in response to user requests

[0197] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[0198] Examples:

[0199] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[0200] Summary of embodiments

[0201] The system of the present invention consists of four steps: automatic collection of work-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests. This not only significantly improves the efficiency of work handover compared to conventional manual work handovers, but also enables flexible information provision according to the user's emotional state, improving the continuity and efficiency of work.

[0202] The processing flow will be explained below.

[0203] Step 1:

[0204] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts them into text format and stores them in a database. Collection is performed automatically at a fixed time during the night (e.g., 2:00 AM), and duplicate data is removed.

[0205] Step 2:

[0206] The server connects to the company's communication tool (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, the content is converted into text format and stored in a database. Chat logs are collected automatically at regular intervals (e.g., every hour).

[0207] Step 3:

[0208] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, modified, and deleted by all employees. When it finds new or updated documents, it saves their contents in a database in text format or in their original format.

[0209] Step 4:

[0210] The server then integrates the collected email, chat, and document data, which involves bringing together data from different data sources into a single, unified data set.

[0211] Step 5:

[0212] The server feeds the combined dataset into a natural language processing-based AI model to train it, allowing it to learn the details of the job and project and prepare for the next user request.

[0213] Step 6:

[0214] The server uses an emotion engine to recognize emotions from the collected data and user interactions. The emotion engine analyzes text and voice data to classify the user's emotional state (e.g., joy, anger, stress). This emotion data is fed into a trained model and used to improve the quality of information provided.

[0215] Step 7:

[0216] When a user takes over a new project, they use their device to enter the project name and request details (e.g., the period for which they want information), and then send a request to the server. The user's emotional state is also considered as part of the request.

[0217] Step 8:

[0218] The server uses trained models and an emotion engine to search and extract relevant information that matches the user's request, retrieving emails, chat logs, and documents related to the requested project or timeframe, and tailoring the response based on the user's emotional state.

[0219] Step 9:

[0220] The server then formats the extracted information into an easy-to-understand format, providing responses such as summaries, highlighting key points, or adding additional details, all of which are customized based on the user's emotional state.

[0221] Step 10:

[0222] The terminal displays the formatted information to the user, who can then review it and quickly take over the task or perform any necessary actions. If the user has further questions or requests, they can send a request to the server again using the same procedure.

[0223] Through the above process, this system significantly improves the efficiency of manual work handover and provides flexible information that responds to the user's emotions.

[0224] Example 2

[0225] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0226] In conventional business handover systems, data collection and integration is done manually, which is inefficient and requires time and effort. Furthermore, the system does not provide information that takes into account the user's emotional state, which can result in inappropriate responses. This results in problems such as reduced business continuity and efficiency.

[0227] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0228] In this invention, the server includes a means for collecting business-related data, a means for integrating and learning the collected business-related data, a means for providing related information from a trained model in response to a user request, and a means for recognizing the user's emotional state and adjusting the information provided in accordance with the emotional state, thereby enabling efficient business handover and providing appropriate information based on the user's emotional state.

[0229] "Business-related data" refers to data that includes information about business activities, such as email data, chat data, and document data within a company.

[0230] "Means of collection" refers to the software and hardware configuration used to obtain data from email servers, chat apps, file storage systems, etc.

[0231] An "integration and training tool" is software and processes that convert collected data into a unified format and feed it into a machine learning model for training.

[0232] "Means for providing relevant information in response to a request" refers to a system that has the function of searching for and providing the desired information from a trained model in response to a user request.

[0233] "Means for recognizing emotional states" refers to an emotion analysis engine and related technologies that analyze text and voice data and classify a user's emotions (e.g., stress, joy, anger, etc.).

[0234] The "means for adjusting information provision" is a system that has the functionality to adjust the content and format of responses taking into account the recognized emotional state of the user.

[0235] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[0236] Collection of work-related data

[0237] The server connects to the company's email server, communication tools, and file storage system to collect all employee emails, chat logs, and documents. For example, a commonly used email server is Exchange Server. Slack is also available as a communication tool, and Google Drive is an example of a file storage system. These data are important sources of information for business activities, and are all stored in a database in text format or in their original form.

[0238] Data Integration and Learning

[0239] The server integrates the collected email, chat, and document data. This integrated data is then fed into an AI model that uses natural language processing technology. For example, AI models such as BERT and GPT-3 (registered trademark) are used. This allows the model to learn about the details of the work and projects, enabling it to provide highly accurate information in preparation for the next user request.

[0240] Emotion recognition by emotion engine

[0241] The server includes an emotion engine that recognizes emotions from collected work-related data and user interactions. The emotion engine analyzes text data and voice data to classify the user's emotional state (e.g., stress, joy, anger, etc.). Specifically, it uses the NLP library spaCy and speech analysis tools IBM Watson (registered trademark) and Google Cloud Speech-to-Text. This emotion data is reflected in the analysis of user requests and the information provided.

[0242] Examples:

[0243] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[0244] Providing information in response to user requests

[0245] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[0246] Examples:

[0247] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[0248] Example prompt sentence:

[0249] "Please provide a summary of relevant emails and important chat logs for Project B."

[0250] "Show me the detailed report on Project B."

[0251] As described above, this system sequentially processes each stage of automatic collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests, thereby helping to improve business efficiency and user satisfaction.

[0252] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0253] Step 1:

[0254] The server connects to the company's email server, chat app, and file storage system to collect business-related data. Specifically, it retrieves email data from Microsoft® Exchange Server, collects chat logs using the Slack API, and downloads document data using the Google Drive API.

[0255] Input: APIs for email servers, chat apps, and file storage systems

[0256] Output: A collection of collected email data, chat data, and document data

[0257] Specific operation: The server executes the Python script, collects data through each API, and stores the collected data in a database.

[0258] Step 2:

[0259] The server consolidates the collected email data, chat data, and document data. A data cleaning tool (e.g., Pandas) is used to standardize the data format and remove unnecessary data.

[0260] Input: Email data, chat data, and document data collected in Step 1

[0261] Output: A consolidated dataset

[0262] Specific operation: The server uses a Python script to combine data from different sources into a single data frame, eliminate duplicate data, and convert it into a unified format.

[0263] Step 3:

[0264] The server then feeds this combined dataset into a natural language processing AI model for training, using, for example, BERT or GPT-3 to learn job descriptions and project details from the data.

[0265] Input: The dataset merged in step 2

[0266] Output: Trained AI model

[0267] Specific operation: The server uses TensorFlow or PyTorch to input the collected and integrated datasets into the AI ​​model for training.

[0268] Step 4:

[0269] The server uses an emotion engine to analyze collected task-related data and user interactions to recognize the user's emotional state, using NLP libraries (e.g., spaCy) and speech analysis tools (e.g., IBM Watson, Google Cloud Speech-to-Text).

[0270] Input: Text data, audio data

[0271] Output: Recognized emotional state data

[0272] How it works: The server parses text data using the spaCy library, converts audio data into text using the Google Cloud Speech-to-Text API, and classifies emotions from the text.

[0273] Step 5:

[0274] The user uses a terminal to request information related to a new project from the server, entering information such as the project name and required period into a dedicated form and submitting it.

[0275] Input: Request information from the user (project name, period, etc.)

[0276] Output: Request to server

[0277] Specific operation: The user enters information into a form on the terminal and sends a request to the server.

[0278] Step 6:

[0279] The server receives and analyzes user requests, using trained AI models and an emotion engine to extract the most relevant information for the request and tailor the response to take into account the user's emotional state.

[0280] Input: User request information, trained AI model, emotion data

[0281] Output: Information to be provided to the user

[0282] How it works: The server analyzes the request, uses AI models to search and extract relevant information, analyzes the user's emotions with an emotion engine, and returns the information in an appropriate format. For example, if stress is detected, it will provide a summary, and if joy is detected, it will provide detailed information.

[0283] This makes task handover more efficient and allows flexible provision of information according to the user's emotional state.

[0284] (Application example 2)

[0285] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0286] In workplaces where industrial robots are used, worker rotation and training of new workers requires a lot of time and effort. In particular, if transfers and handover of projects and work to new workers are not carried out promptly, the efficiency and productivity of the entire business may decline. Furthermore, the emotional state of workers can affect the quality of information they provide, and situations where workers feel stressed or anxious increase the risk of work errors. To solve these issues, effective and emotion-aware information provision is required.

[0287] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work-related data, means for integrating and learning the collected work-related data, means for providing related information from a trained model in response to a user request, means for recognizing emotions and adjusting a response, and means for streamlining task handover in an industrial robot in response to a user request. This enables the industrial robot to streamline task handover on site and provide flexible information based on emotion recognition.

[0288] "Business-related data" refers to any digital information related to business activities, such as email data, chat data, and business document data.

[0289] "Means of collection" refers to technology or equipment that automatically obtains the required data from designated data sources (e.g., email servers, chat applications, document management systems, etc.).

[0290] An "integration and learning means" is a process or system that centrally integrates collected business-related data and applies machine learning algorithms to generate a model based on that data.

[0291] "Means for providing relevant information from a trained model in response to a user request" refers to technology or equipment for extracting and providing appropriate information from a pre-trained model based on a request entered by a user.

[0292] "Means for recognizing emotions and tailoring responses" refers to systems or algorithms that analyze a user's text or voice, identify their emotional state, and generate appropriate responses based on those emotions.

[0293] An "industrial robot" is an automated machine or device designed to perform tasks in production or manufacturing settings.

[0294] "Means for streamlining the handover of tasks" refers to technologies and systems that enable users to quickly and accurately communicate details of their work and projects to new personnel.

[0295] To realize the present invention in an industrial robot, the following system configuration and processing are required.

[0296] Hardware Configuration

[0297] 1. Robots: Industrial robots used in production (e.g., humanoid robots)

[0298] 2. Server: A server for data processing and model training

[0299] 3. Network: Network environment for connecting the robot and the server

[0300] 4. Sensors: Sensors to collect audio and video information for emotion recognition

[0301] Software Configuration

[0302] 1. Data collection software: Apache Kafka or similar real-time data streaming platform

[0303] 2. Data search software: Search engines such as Elasticsearch

[0304] 3. Machine Learning Models: OpenAI® GPT-4® and TensorFlow models for sentiment analysis

[0305] 4. Emotion Recognition Software: Emotion analysis software such as Natural Language Toolkit (NLTK) and Nvidia Clara

[0306] Processing Overview

[0307] 1. Data collection: The server collects email, chat, and business document data from various data sources within the company in real time. Apache Kafka can be used to efficiently ingest this data.

[0308] 2. Data integration and training: The collected data is stored in Elasticsearch, and OpenAI GPT-4 is used to integrate and train this data. During this process, business details and relevant information are filtered by the machine learning model.

[0309] 3. Emotion Recognition: The robot's built-in microphone and camera detect the user's voice and facial expressions, and analyzes their emotional state using TensorFlow. This emotional data is sent to the server when the user makes a request.

[0310] 4. Information provision: When a user requests specific information through the robot, the server analyzes the request based on the trained model and emotion recognition results, and adjusts the response according to the user's emotional state to provide appropriate information to the user.

[0311] Specific examples

[0312] If a user requests detailed instructions for the next step in a task, the server generates a prompt like this and provides it to OpenAI GPT-4:

[0313] Example prompt sentence:

[0314] "A worker has requested more details about process X. He seems a little nervous. What information should he provide first? Please keep it concise and focus on five points."

[0315] In response to this prompt, OpenAI GPT-4 provides a concise summary of five key points, allowing users to get the information they need quickly and accurately. Emotion recognition also allows for a more concise explanation if the user is stressed, or more detailed information if the user is emotionally stable.

[0316] In this way, the system of the present invention can improve the work efficiency of industrial robots and provide information flexibly according to the emotional state of the user.

[0317] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0318] Step 1:

[0319] The server collects business-related data from email servers, chat applications, document management systems, etc. Specifically, it uses Apache Kafka to stream this data in real time and store it in Elasticsearch. The input is data from each data source, and the output is the integrated data stored in Elasticsearch.

[0320] Step 2:

[0321] The server uses business-related data stored in Elasticsearch and trains the data using the OpenAI GPT-4 model. At this stage, the consolidated data is fed into a machine learning algorithm, which analyzes the data and learns about the business and project details. The input is the consolidated data from Elasticsearch, and the output is a trained GPT-4 model.

[0322] Step 3:

[0323] The robot's built-in microphones and cameras collect the user's voice and video and send them to a server. The server uses TensorFlow to analyze the voice and video data and identify the user's emotional state. The input is the voice and video data, and the output is a classification of the user's emotional state.

[0324] Step 4:

[0325] The user sends a question or request to the robot. For example, "Please tell me the detailed steps for the next work step." This request is sent to the server. The input is the user's request, and the output is its content.

[0326] Step 5:

[0327] The server analyzes the user's request and generates and supplies a prompt to the generative AI model (GPT-4). For example, the prompt might read, "A worker has requested details about process X. He seems a little nervous. What information should he provide first? Please answer briefly, focusing on five key points." The input is the user's request and emotional state, and the output is the prompt.

[0328] Step 6:

[0329] Based on the generated prompt, the OpenAI GPT-4 model generates appropriate information according to the request. The server analyzes this information and reconstructs it in a form that is easy for the user to understand. The input is the prompt, and the output is the generated information.

[0330] Step 7:

[0331] The server sends the generated information to the robot, which then provides it to the user. For example, five key points can be briefly explained. The input is the generated information, and the output is the specific information provided to the user.

[0332] This series of processes enables industrial robots to provide appropriate and efficient information taking into account the user's emotional state.

[0333] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0334] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0335] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0336] [Second embodiment]

[0337] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0338] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0339] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0340] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0341] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0342] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0343] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0344] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0345] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0346] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0347] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0348] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0349] The system of this invention allows the server to collect business-related data, integrate it, and learn from it to quickly and accurately provide the business information required by the user. By implementing this system, even in the event of transfer or retirement, business handover can be carried out smoothly, improving the continuity and efficiency of business operations.

[0350] This system mainly consists of the following steps: collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[0351] Collection of work-related data

[0352] The server first connects to the company's email server and collects all emails sent and received by employees for work, then connects to communication tools (e.g., chat apps) and collects all chat logs related to work, and finally connects to the file storage system and collects all documents related to work.

[0353] Examples:

[0354] The server connects to the email server every night at midnight and collects new mail for that day.

[0355] The server connects to the chat tool used by the company at regular intervals (for example, every hour) and retrieves new chat messages sent and received during that time.

[0356] Data Integration and Learning

[0357] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model (natural language processing model) for learning. The server automates this process and periodically updates the learning model.

[0358] Examples:

[0359] The server compiles the latest email data, chat data, and document data into a single database on a daily basis.

[0360] Using this database, the AI ​​model is trained overnight, and by the next morning a model reflecting the latest information is completed.

[0361] Providing information in response to user requests

[0362] When a user needs specific information for work handover or information understanding, the user sends a request to the server via the device, and the server uses the trained model to quickly extract information relevant to the user's request and provide it to the device.

[0363] Examples:

[0364] A user who has been newly assigned to a project requests on their device, "Show me all emails and important chat logs related to Project A from the past three months."

[0365] The server uses the trained model to search for relevant emails and chat logs, extract that information, and display it on the device.

[0366] Summary of embodiments

[0367] The system of this invention combines three major steps: automatic collection of business-related data, data integration and learning, and information provision based on user requests. This significantly simplifies the traditional manual process of business handover, and enables smooth business continuity even in the event of transfers or retirement. Furthermore, by periodically learning the latest data, the system can always provide the latest information.

[0368] The processing flow will be explained below.

[0369] Step 1:

[0370] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts their contents into plain text and stores them in a database. It also deduplicates data already collected in the past.

[0371] Step 2:

[0372] The server connects to the company's communication tools (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, it converts the content into text format and stores it in a database. It then compares it with past logs to incorporate only new data, improving efficiency.

[0373] Step 3:

[0374] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, deleted, or updated by employees. When it finds new or changed documents, it saves the contents to a database in text format or raw format.

[0375] Step 4:

[0376] The server consolidates the collected email, chat, and document data, which involves combining all new and updated data in the database into one unified data set.

[0377] Step 5:

[0378] The server feeds the integrated dataset to an AI model (natural language processing model) to train the model, which learns the details of the business and project and prepares for the next user request.

[0379] Step 6:

[0380] When a user takes over a new project, he or she inputs specific information (e.g., project name, period for which information is desired) from the terminal and sends a request to the server.

[0381] Step 7:

[0382] The server feeds the user's request to the trained model and retrieves the relevant information that the model returns, including emails, chat messages, and documents related to the specified project and time period.

[0383] Step 8:

[0384] The server formats the acquired information and converts it into a format that is easy for the user to understand. The information is sent to the device and displayed in an appropriate format (e.g., list format, timeline format).

[0385] Step 9:

[0386] The user can check the information provided on the terminal and smoothly carry out the necessary handover and work. If there are any further questions or additional requests, they can send a request to the server again using the same procedure.

[0387] Example 1

[0388] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0389] The current business handover process relies on manual work and individual knowledge, resulting in low efficiency and causing problems with business continuity when employees are transferred or leave the company. Furthermore, business-related data exists in a wide variety of formats, making it difficult to quickly and accurately extract the necessary information. This negatively impacts the efficiency and completeness of business operations. Furthermore, the lack of an effective means to quickly obtain the information necessary for business operations can delay the entire business process.

[0390] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0391] In this invention, the server includes means for acquiring collected business-related data at a scheduled time, means for integrating the acquired business-related data into a database, means for training a natural language processing model using the integrated data, means for receiving a request in the form of a prompt sentence from a user, means for searching for information based on the user request using the trained model, and means for providing the search results to a user terminal. This enables efficient collection, integration, and training of business-related data and rapid information provision. As a result, smooth business handover and improved business continuity and efficiency are achieved.

[0392] "Collected business-related data" is a set of data related to a company's business activities, including email data, chat data, and file storage data.

[0393] The "measure for acquiring data at a fixed time" is a mechanism by which the server automatically collects business-related data at a preset time.

[0394] "Means of integrating into a database" refers to the process of compiling and integrating various collected data into a single database.

[0395] "Means for training a natural language processing model" refers to a method for using integrated data to train an AI model to perform natural language processing.

[0396] The "means for receiving a request in the form of a prompt sentence" is an interface for receiving a request input by a user in a specific format and transmitting it to a server.

[0397] "Means for searching for information based on a user request using a trained model" is a process of searching for relevant information in response to a user request using an already trained natural language processing model.

[0398] The "means for providing search results to a user terminal" refers to a mechanism by which the server transmits the search results to the user terminal and provides them in a displayable or usable form.

[0399] This invention is a system in which a server collects business-related data, integrates and learns from it, and quickly and accurately provides the business information required by users. The main functions of this system include collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[0400] Collection of work-related data

[0401] The server first connects to the company's email server and collects all emails sent and received by employees for work. Then, the server connects to communication tools (e.g., chat apps) and collects all chat logs related to work. Furthermore, the server connects to a file storage system and collects all documents related to work. Specifically, the server uses the following hardware and software:

[0402] Email server: Retrieves email data using the IMAP protocol.

[0403] Chat app: Collect chat logs using REST API.

[0404] File storage system: Retrieves documents using the SFTP protocol.

[0405] For example, the server connects to an email server every night at midnight to retrieve new emails for that day. The server also connects to a chat tool used by the company at a fixed time (for example, every hour) to retrieve new chat messages sent and received during that time. For this purpose, a scheduling function (for example, a Cron job) is set up on the server to connect at fixed times.

[0406] Data Integration and Learning

[0407] The server then integrates the collected data into a database. Specifically, the server processes the data as follows:

[0408] Database: Use a relational database management system such as MySQL and insert each piece of data into the corresponding table.

[0409] ETL scripts: Data mapping and cleansing are performed using temporary tables to ensure data integrity.

[0410] The server then uses the integrated data to train a natural language processing model. Specifically, the server uses the following techniques:

[0411] Python environment: Build and train natural language processing models using TensorFlow and Keras.

[0412] Data preprocessing: Tokenization, text vectorization, etc.

[0413] Save the trained model: Save it to disk in HDF5 format or similar.

[0414] Providing information in response to user requests

[0415] When a user needs specific information for work handover or information gathering, they send a request to the server via their terminal. Specifically, the process is as follows:

[0416] Prompt input: A user accesses a dedicated web application in a browser and inputs a request such as: "Show me all emails and important chat logs related to Project A from the past three months."

[0417] HTTP communication: The terminal sends this prompt text to the server using an HTTP POST request.

[0418] The server uses the trained model to search for relevant information based on the user's request, then sends the search results back to the user's device, where the user can view the results. Specifically, it works as follows:

[0419] Query Generation: A natural language processing model analyzes the user's input and generates an appropriate SQL query.

[0420] Retrieving search results: The server executes the generated query and extracts the relevant data from the database.

[0421] Send results: The search results are converted into JSON format and sent to the device.

[0422] The terminal parses the received JSON data and displays it to the user in a table or report format on the web application.

[0423] As described above, the present invention realizes efficient collection, integration, and learning of business-related data, and rapid information provision, thereby achieving smooth business handover and improved business continuity and efficiency.

[0424] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0425] Step 1: Collect business-related data

[0426] The server connects to the company's email server and collects all emails sent and received by employees for work. The server uses the IMAP protocol to access the mail server and retrieves new emails. Next, the server connects to communication tools (e.g., chat apps) using REST APIs and collects all chat logs related to work. Finally, the server connects to the file storage system using the SFTP protocol and collects all documents related to work.

[0427] Inputs: email servers, chat apps, file storage systems

[0428] Output: New email data, chat log data, document data

[0429] What it does: The server runs a cron job every night at midnight to collect new emails from the email server for the day, connects to the company's chat tool every hour to retrieve new chat messages, and periodically downloads important documents from the file storage system.

[0430] Step 2: Integrate the data

[0431] The server integrates the collected email, chat, and document data into a database. The server connects to a MySQL database and inserts each piece of data into the corresponding table. To maintain data consistency, temporary tables are used, and data mapping and cleaning are performed.

[0432] Input: New email data, chat log data, document data

[0433] Output: Integrated database

[0434] What happens: The server runs the ETL (Extract, Transform, Load) script to insert various data into the database, including storing new email data in the email table, chat logs in the chat table, and document data in the document table.

[0435] Step 3: Train the data

[0436] The server uses the integrated data to train a natural language processing model. The server sets up a Python environment and uses TensorFlow and Keras to build and train the natural language processing model. As data preprocessing, it performs tokenization and text vectorization, and saves the trained model.

[0437] Input: Integrated database

[0438] Output: Trained AI model

[0439] How it works: Every night, the server runs a Python script to train a natural language processing model based on the integrated data. After training is complete, the model is saved in an HDF5 file.

[0440] Step 4: Receiving a User Request

[0441] A user requests specific information for the purpose of task handover or information gathering. The user accesses a dedicated web application and inputs the request in the form of a prompt. The terminal then sends this request to the server using the HTTP POST method.

[0442] Input: A prompt-style request from the user

[0443] Output: HTTP POST request to the server

[0444] Specific operation: A user enters the request "Show me all emails and important chat logs related to Project A from the past three months" into a web form and presses the submit button. The device sends this request to the server.

[0445] Step 5: Search using the trained model

[0446] The server uses the trained model to search for relevant information based on the user's request. A natural language processing model analyzes the user's input and generates an appropriate SQL query. The server executes the query and extracts the relevant data from the database.

[0447] Input: A request in the form of a prompt, a trained AI model

[0448] Output: Search result data

[0449] Specific operation: The server parses the Japanese prompt and generates an SQL query to retrieve email data and chat logs related to "Project A" from the database. The server executes this query to extract the required data.

[0450] Step 6: Serving search results

[0451] The server returns the search results to the user's device. The search results are converted to JSON format and sent as an HTTP response. The device parses the received JSON data and displays it in a table or report format on the web application.

[0452] Input: Search result data

[0453] Output: Data displayed on the user's terminal

[0454] Specific operation: The server converts the search results into JSON format and sends it to the device as an HTTP response. The device then analyzes the received JSON data and displays it to the user on the web application. Specifically, the data is organized in tabular or graph format, allowing the user to see the information they need at a glance.

[0455] (Application example 1)

[0456] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0457] In modern industrial environments, large amounts of data are generated, including equipment operation logs, sensor data, and quality inspection data. Properly managing this data and ensuring smooth handovers when employees are transferred or leave the company are key issues for improving business continuity and efficiency. However, with conventional systems, the collection, integration, and learning of business-related data is often done manually, resulting in poor business efficiency. Furthermore, the inability to provide appropriate information quickly has led to numerous problems with business handover and management.

[0458] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0459] In this invention, the server includes means for collecting business-related data, means for integrating and training the collected business-related data, means for providing related information from a trained model in response to a user request, means for collecting operation logs, sensor data, and quality inspection data from industrial equipment, means for supplying the collected industrial equipment data to a machine learning model for integration and training, and means for using the machine learning model to extract and provide related information in response to inquiries from engineers. This enables the efficient collection and management of large amounts of business-related data in industrial environments, enabling smooth handover in the event of transfers or retirements, as well as improving business continuity and efficiency.

[0460] "Business-related data" refers to data including email data, chat data, document data, and operation logs, sensor data, and quality inspection data generated by industrial equipment when conducting business within a company.

[0461] "Means of collection" refers to the functions and technologies used by servers or other devices to access various data sources via a network and collect the required data.

[0462] "Means of integration and learning" refers to the functions and technologies that centrally manage collected data and supply it to AI models or machine learning models for analysis and learning.

[0463] "Means of providing relevant information from a trained model" refers to functions and technologies that quickly extract and provide necessary information from a pre-trained AI model in response to a user request.

[0464] "Industrial equipment" refers to machines and devices used in manufacturing, testing, processing, or other industrial processes.

[0465] "Operation log" refers to operation history data recorded when industrial equipment is in operation.

[0466] "Sensor data" refers to various measurement data acquired by sensors attached to industrial equipment.

[0467] "Quality inspection data" refers to data acquired by inspection equipment to evaluate the quality of a product or process.

[0468] "Means of supplying data to a machine learning model for integration and learning" refers to the functions and technologies for inputting various collected data into a machine learning algorithm for integration and learning, and extracting useful insights and patterns from the data.

[0469] "Means for extracting and providing relevant information in response to inquiries from engineers" refers to the functions and technologies for extracting appropriate information from collected and learned data based on inquiries from engineers who manage and operate industrial equipment, and providing it to the engineers.

[0470] A system for implementing the present invention utilizes various hardware and software components. Specific embodiments are described below.

[0471] 1. Data Collection Module

[0472] The server collects business-related data such as operation logs of industrial equipment, sensor data, and quality inspection data in real time via the network. For example, it has a function to obtain data from sensors using APIs and record operation logs.

[0473] The specific software used includes a REST API for data retrieval and a database system (e.g., Elasticsearch). The server periodically collects information from these data sources and stores it in a database.

[0474] 2. Data Integration and Learning Module

[0475] The server centrally manages the various collected data by periodically retrieving data stored in a database system such as Elasticsearch and providing it to the machine learning model.

[0476] The specific software used is the Hugging Face transformers library, which uses natural language processing techniques to analyze the data and periodically train the model.

[0477] 3. User Request Module

[0478] When a user sends a request to the system using an administrative terminal, the server uses a pre-trained machine learning model to quickly search and extract relevant information and provide it to the user.

[0479] For example, if a technician requests "Show me the quality inspection data for the past three months," the server will search for the relevant data in Elasticsearch and provide the relevant information to the technician, enabling them to quickly obtain the information they need and improving work efficiency.

[0480] Examples of prompts:

[0481] "Show me the quality inspection data for the past three months"

[0482] As described above, the embodiment of the present invention is composed of three main modules: data collection, data integration and learning, and user request response, which enable efficient collection and management of industrial equipment data and enable users to quickly obtain the information they need.

[0483] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0484] Step 1: Data collection

[0485] The server accesses various data sources over the network. Specifically, it periodically sends requests to the APIs of industrial equipment to obtain operation logs, sensor data, and quality inspection data. The input is real-time data from the equipment, and the output is the collected data stream. The server converts this data into an appropriate format and stores it in a database.

[0486] Step 2: Data integration

[0487] The server accumulates the collected data in a database such as Elasticsearch. Daily data is linked to create a single centralized database. The input is various business-related data, and the output is the integrated database. Specifically, the server retrieves new data at the end of each day and adds it to the centralized database.

[0488] Step 3: Learning Modules

[0489] The server uses the merged data to train a machine learning model (for example, using Hugging Face's transformers library). The input is the merged database, and the output is the trained model. Specifically, the server periodically retrieves the latest data from the database every night and feeds it to the model for training.

[0490] Step 4: User request reception

[0491] A user uses a terminal to request specific business-related information. The request is entered in natural language. The input is the request sentence from the user, and the output is the analysis result of the request content. In concrete terms, the user enters the request through the terminal interface.

[0492] Step 5: Information Search

[0493] The server uses the trained model to search for relevant information from Elasticsearch based on the user request. The input is the request analysis result, and the output is the searched business-related information. Specifically, the server generates an Elasticsearch query and searches for the appropriate information in the database.

[0494] Step 6: Provide information

[0495] The server organizes the search results and provides them to the user. The input is the business-related information that was searched, and the output is the organized information that is provided to the user. Specifically, the server compactly organizes the search results and displays them on the user's terminal. At this time, the information is converted into the required format according to the content of the request.

[0496] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0497] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[0498] This system consists of the following steps: collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and provision of information in response to user requests.

[0499] Collection of work-related data

[0500] The server connects to the company's email server, communication tools (e.g., chat apps), and file storage system to collect all employees' emails, chat logs, and documents. These data are important sources of information for business activities, and are all stored in a database in text format or in its original form.

[0501] Data Integration and Learning

[0502] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model using natural language processing technology, which learns about the details of the work and the project. This allows it to provide more accurate information in preparation for the next user request.

[0503] Emotion recognition by emotion engine

[0504] The server includes an emotion engine that recognizes the user's emotions from the collected work-related data and user interactions. The emotion engine analyzes text data and voice data and can classify the user's emotional state (e.g., stress, joy, anger, etc.). This emotion data is reflected in the analysis of user requests and the provision of information.

[0505] Examples:

[0506] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[0507] Providing information in response to user requests

[0508] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[0509] Examples:

[0510] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[0511] Summary of embodiments

[0512] The system of the present invention consists of four steps: automatic collection of work-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests. This not only significantly improves the efficiency of work handover compared to conventional manual work handovers, but also enables flexible information provision according to the user's emotional state, improving the continuity and efficiency of work.

[0513] The processing flow will be explained below.

[0514] Step 1:

[0515] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts them into text format and stores them in a database. Collection is performed automatically at a fixed time during the night (e.g., 2:00 AM), and duplicate data is removed.

[0516] Step 2:

[0517] The server connects to the company's communication tool (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, the content is converted into text format and stored in a database. Chat logs are collected automatically at regular intervals (e.g., every hour).

[0518] Step 3:

[0519] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, modified, and deleted by all employees. When it finds new or updated documents, it saves their contents in a database in text format or in their original format.

[0520] Step 4:

[0521] The server then integrates the collected email, chat, and document data, which involves bringing together data from different data sources into a single, unified data set.

[0522] Step 5:

[0523] The server feeds the combined dataset into a natural language processing-based AI model to train it, allowing it to learn the details of the job and project and prepare for the next user request.

[0524] Step 6:

[0525] The server uses an emotion engine to recognize emotions from the collected data and user interactions. The emotion engine analyzes text and voice data to classify the user's emotional state (e.g., joy, anger, stress). This emotion data is fed into a trained model and used to improve the quality of information provided.

[0526] Step 7:

[0527] When a user takes over a new project, they use their device to enter the project name and request details (e.g., the period for which they want information), and then send a request to the server. The user's emotional state is also considered as part of the request.

[0528] Step 8:

[0529] The server uses trained models and an emotion engine to search and extract relevant information that matches the user's request, retrieving emails, chat logs, and documents related to the requested project or timeframe, and tailoring the response based on the user's emotional state.

[0530] Step 9:

[0531] The server then formats the extracted information into an easy-to-understand format, providing responses such as summaries, highlighting key points, or adding additional details, all of which are customized based on the user's emotional state.

[0532] Step 10:

[0533] The terminal displays the formatted information to the user, who can then review it and quickly take over the task or perform any necessary actions. If the user has further questions or requests, they can send a request to the server again using the same procedure.

[0534] Through the above process, this system significantly improves the efficiency of manual work handover and provides flexible information that responds to the user's emotions.

[0535] Example 2

[0536] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0537] In conventional business handover systems, data collection and integration is done manually, which is inefficient and requires time and effort. Furthermore, the system does not provide information that takes into account the user's emotional state, which can result in inappropriate responses. This results in problems such as reduced business continuity and efficiency.

[0538] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0539] In this invention, the server includes a means for collecting business-related data, a means for integrating and learning the collected business-related data, a means for providing related information from a trained model in response to a user request, and a means for recognizing the user's emotional state and adjusting the information provided in accordance with the emotional state, thereby enabling efficient business handover and providing appropriate information based on the user's emotional state.

[0540] "Business-related data" refers to data that includes information about business activities, such as email data, chat data, and document data within a company.

[0541] "Means of collection" refers to the software and hardware configuration used to obtain data from email servers, chat apps, file storage systems, etc.

[0542] An "integration and training tool" is software and processes that convert collected data into a unified format and feed it into a machine learning model for training.

[0543] "Means for providing relevant information in response to a request" refers to a system that has the function of searching for and providing the desired information from a trained model in response to a user request.

[0544] "Means for recognizing emotional states" refers to an emotion analysis engine and related technologies that analyze text and voice data and classify a user's emotions (e.g., stress, joy, anger, etc.).

[0545] The "means for adjusting information provision" is a system that has the functionality to adjust the content and format of responses taking into account the recognized emotional state of the user.

[0546] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[0547] Collection of work-related data

[0548] The server connects to the company's email server, communication tools, and file storage system to collect all employee emails, chat logs, and documents. For example, a commonly used email server is Exchange Server. Slack is also available as a communication tool, and Google Drive is an example of a file storage system. These data are important sources of information for business activities, and are all stored in a database in text format or in their original form.

[0549] Data Integration and Learning

[0550] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model that uses natural language processing technology. For example, AI models such as BERT and GPT-3 are used. This allows the model to learn about the details of the work and projects, enabling it to provide more accurate information in preparation for the next user request.

[0551] Emotion recognition by emotion engine

[0552] The server includes an emotion engine that recognizes emotions from collected work-related data and user interactions. The emotion engine analyzes text and voice data to classify the user's emotional state (e.g., stress, joy, anger, etc.). Specifically, it uses the NLP library spaCy and speech analysis tools IBM Watson and Google Cloud Speech-to-Text. This emotion data is reflected in the analysis of user requests and the information provided.

[0553] Examples:

[0554] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[0555] Providing information in response to user requests

[0556] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[0557] Examples:

[0558] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[0559] Example prompt sentence:

[0560] "Please provide a summary of relevant emails and important chat logs for Project B."

[0561] "Show me the detailed report on Project B."

[0562] As described above, this system sequentially processes each stage of automatic collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests, thereby helping to improve business efficiency and user satisfaction.

[0563] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0564] Step 1:

[0565] The server connects to the company's email server, chat app, and file storage system to collect business-related data, specifically retrieving email data from Microsoft Exchange Server, collecting chat logs using the Slack API, and downloading document data using the Google Drive API.

[0566] Input: APIs for email servers, chat apps, and file storage systems

[0567] Output: A collection of collected email data, chat data, and document data

[0568] Specific operation: The server executes the Python script, collects data through each API, and stores the collected data in a database.

[0569] Step 2:

[0570] The server consolidates the collected email data, chat data, and document data. A data cleaning tool (e.g., Pandas) is used to standardize the data format and remove unnecessary data.

[0571] Input: Email data, chat data, and document data collected in Step 1

[0572] Output: A consolidated dataset

[0573] Specific operation: The server uses a Python script to combine data from different sources into a single data frame, eliminate duplicate data, and convert it into a unified format.

[0574] Step 3:

[0575] The server then feeds this combined dataset into a natural language processing AI model for training, using, for example, BERT or GPT-3 to learn job descriptions and project details from the data.

[0576] Input: The dataset merged in step 2

[0577] Output: Trained AI model

[0578] Specific operation: The server uses TensorFlow or PyTorch to input the collected and integrated datasets into the AI ​​model for training.

[0579] Step 4:

[0580] The server uses an emotion engine to analyze collected task-related data and user interactions to recognize the user's emotional state, using NLP libraries (e.g., spaCy) and speech analysis tools (e.g., IBM Watson, Google Cloud Speech-to-Text).

[0581] Input: Text data, audio data

[0582] Output: Recognized emotional state data

[0583] How it works: The server parses text data using the spaCy library, converts audio data into text using the Google Cloud Speech-to-Text API, and classifies emotions from the text.

[0584] Step 5:

[0585] The user uses a terminal to request information related to a new project from the server, entering information such as the project name and required period into a dedicated form and submitting it.

[0586] Input: Request information from the user (project name, period, etc.)

[0587] Output: Request to server

[0588] Specific operation: The user enters information into a form on the terminal and sends a request to the server.

[0589] Step 6:

[0590] The server receives and analyzes user requests, using trained AI models and an emotion engine to extract the most relevant information for the request and tailor the response to take into account the user's emotional state.

[0591] Input: User request information, trained AI model, emotion data

[0592] Output: Information to be provided to the user

[0593] How it works: The server analyzes the request, uses AI models to search and extract relevant information, analyzes the user's emotions with an emotion engine, and returns the information in an appropriate format. For example, if stress is detected, it will provide a summary, and if joy is detected, it will provide detailed information.

[0594] This makes task handover more efficient and allows flexible provision of information according to the user's emotional state.

[0595] (Application example 2)

[0596] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0597] In workplaces where industrial robots are used, worker rotation and training of new workers requires a lot of time and effort. In particular, if transfers and handover of projects and work to new workers are not carried out promptly, the efficiency and productivity of the entire business may decline. Furthermore, the emotional state of workers can affect the quality of information they provide, and situations where workers feel stressed or anxious increase the risk of work errors. To solve these issues, effective and emotion-aware information provision is required.

[0598] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work-related data, means for integrating and learning the collected work-related data, means for providing related information from a trained model in response to a user request, means for recognizing emotions and adjusting a response, and means for streamlining task handover in an industrial robot in response to a user request. This enables the industrial robot to streamline task handover on site and provide flexible information based on emotion recognition.

[0599] "Business-related data" refers to any digital information related to business activities, such as email data, chat data, and business document data.

[0600] "Means of collection" refers to technology or equipment that automatically obtains the required data from designated data sources (e.g., email servers, chat applications, document management systems, etc.).

[0601] An "integration and learning means" is a process or system that centrally integrates collected business-related data and applies machine learning algorithms to generate a model based on that data.

[0602] "Means for providing relevant information from a trained model in response to a user request" refers to technology or equipment for extracting and providing appropriate information from a pre-trained model based on a request entered by a user.

[0603] "Means for recognizing emotions and tailoring responses" refers to systems or algorithms that analyze a user's text or voice, identify their emotional state, and generate appropriate responses based on those emotions.

[0604] An "industrial robot" is an automated machine or device designed to perform tasks in production or manufacturing settings.

[0605] "Means for streamlining the handover of tasks" refers to technologies and systems that enable users to quickly and accurately communicate details of their work and projects to new personnel.

[0606] To realize the present invention in an industrial robot, the following system configuration and processing are required.

[0607] Hardware Configuration

[0608] 1. Robots: Industrial robots used in production (e.g., humanoid robots)

[0609] 2. Server: A server for data processing and model training

[0610] 3. Network: Network environment for connecting the robot and the server

[0611] 4. Sensors: Sensors to collect audio and video information for emotion recognition

[0612] Software Configuration

[0613] 1. Data collection software: Apache Kafka or a similar real-time data streaming platform

[0614] 2. Data search software: Search engines such as Elasticsearch

[0615] 3. Machine learning models: OpenAI GPT-4 and TensorFlow models for sentiment analysis

[0616] 4. Emotion Recognition Software: Emotion analysis software such as Natural Language Toolkit (NLTK) and Nvidia Clara

[0617] Processing Overview

[0618] 1. Data collection: The server collects email, chat, and business document data from various data sources within the company in real time. Apache Kafka can be used to efficiently ingest this data.

[0619] 2. Data integration and training: The collected data is stored in Elasticsearch, and OpenAI GPT-4 is used to integrate and train this data. During this process, business details and relevant information are filtered by the machine learning model.

[0620] 3. Emotion Recognition: The robot's built-in microphone and camera detect the user's voice and facial expressions, and analyzes their emotional state using TensorFlow. This emotional data is sent to the server when the user makes a request.

[0621] 4. Information provision: When a user requests specific information through the robot, the server analyzes the request based on the trained model and emotion recognition results, and adjusts the response according to the user's emotional state to provide appropriate information to the user.

[0622] Specific examples

[0623] If a user requests detailed instructions for the next step in a task, the server generates a prompt like this and provides it to OpenAI GPT-4:

[0624] Example prompt sentence:

[0625] "A worker has requested more details about process X. He seems a little nervous. What information should he provide first? Please keep it concise and focus on five points."

[0626] In response to this prompt, OpenAI GPT-4 provides a concise summary of five key points, allowing users to get the information they need quickly and accurately. Emotion recognition also allows for a more concise explanation if the user is stressed, or more detailed information if the user is emotionally stable.

[0627] In this way, the system of the present invention can improve the work efficiency of industrial robots and provide information flexibly according to the emotional state of the user.

[0628] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0629] Step 1:

[0630] The server collects business-related data from email servers, chat applications, document management systems, etc. Specifically, it uses Apache Kafka to stream this data in real time and store it in Elasticsearch. The input is data from each data source, and the output is the integrated data stored in Elasticsearch.

[0631] Step 2:

[0632] The server uses business-related data stored in Elasticsearch and trains the data using the OpenAI GPT-4 model. At this stage, the consolidated data is fed into a machine learning algorithm, which analyzes the data and learns about the business and project details. The input is the consolidated data from Elasticsearch, and the output is a trained GPT-4 model.

[0633] Step 3:

[0634] The robot's built-in microphones and cameras collect the user's voice and video and send them to a server. The server uses TensorFlow to analyze the voice and video data and identify the user's emotional state. The input is the voice and video data, and the output is a classification of the user's emotional state.

[0635] Step 4:

[0636] The user sends a question or request to the robot. For example, "Please tell me the detailed steps for the next work step." This request is sent to the server. The input is the user's request, and the output is its content.

[0637] Step 5:

[0638] The server analyzes the user's request and generates and supplies a prompt to the generative AI model (GPT-4). For example, the prompt might read, "A worker has requested details about process X. He seems a little nervous. What information should he provide first? Please answer briefly, focusing on five key points." The input is the user's request and emotional state, and the output is the prompt.

[0639] Step 6:

[0640] Based on the generated prompt, the OpenAI GPT-4 model generates appropriate information according to the request. The server analyzes this information and reconstructs it in a form that is easy for the user to understand. The input is the prompt, and the output is the generated information.

[0641] Step 7:

[0642] The server sends the generated information to the robot, which then provides it to the user. For example, five key points can be briefly explained. The input is the generated information, and the output is the specific information provided to the user.

[0643] This series of processes enables industrial robots to provide appropriate and efficient information taking into account the user's emotional state.

[0644] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0645] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0646] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0647] [Third embodiment]

[0648] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0649] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0650] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0651] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0652] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0653] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0654] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0655] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0656] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0657] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0658] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0659] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0660] The system of this invention allows the server to collect business-related data, integrate it, and learn from it to quickly and accurately provide the business information required by the user. By implementing this system, even in the event of transfer or retirement, business handover can be carried out smoothly, improving the continuity and efficiency of business operations.

[0661] This system mainly consists of the following steps: collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[0662] Collection of work-related data

[0663] The server first connects to the company's email server and collects all emails sent and received by employees for work, then connects to communication tools (e.g., chat apps) and collects all chat logs related to work, and finally connects to the file storage system and collects all documents related to work.

[0664] Examples:

[0665] The server connects to the email server every night at midnight and collects new mail for that day.

[0666] The server connects to the chat tool used by the company at regular intervals (for example, every hour) and retrieves new chat messages sent and received during that time.

[0667] Data Integration and Learning

[0668] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model (natural language processing model) for learning. The server automates this process and periodically updates the learning model.

[0669] Examples:

[0670] The server compiles the latest email data, chat data, and document data into a single database on a daily basis.

[0671] Using this database, the AI ​​model is trained overnight, and by the next morning a model reflecting the latest information is completed.

[0672] Providing information in response to user requests

[0673] When a user needs specific information for work handover or information understanding, the user sends a request to the server via the device, and the server uses the trained model to quickly extract information relevant to the user's request and provide it to the device.

[0674] Examples:

[0675] A user who has been newly assigned to a project requests on their device, "Show me all emails and important chat logs related to Project A from the past three months."

[0676] The server uses the trained model to search for relevant emails and chat logs, extract that information, and display it on the device.

[0677] Summary of embodiments

[0678] The system of this invention combines three major steps: automatic collection of business-related data, data integration and learning, and information provision based on user requests. This significantly simplifies the traditional manual process of business handover, and enables smooth business continuity even in the event of transfers or retirement. Furthermore, by periodically learning the latest data, the system can always provide the latest information.

[0679] The processing flow will be explained below.

[0680] Step 1:

[0681] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts their contents into plain text and stores them in a database. It also deduplicates data already collected in the past.

[0682] Step 2:

[0683] The server connects to the company's communication tools (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, it converts the content into text format and stores it in a database. It then compares it with past logs to incorporate only new data, improving efficiency.

[0684] Step 3:

[0685] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, deleted, or updated by employees. When it finds new or changed documents, it saves the contents to a database in text format or raw format.

[0686] Step 4:

[0687] The server consolidates the collected email, chat, and document data, which involves combining all new and updated data in the database into one unified data set.

[0688] Step 5:

[0689] The server feeds the integrated dataset to an AI model (natural language processing model) to train the model, which learns the details of the business and project and prepares for the next user request.

[0690] Step 6:

[0691] When a user takes over a new project, he or she inputs specific information (e.g., project name, period for which information is desired) from the terminal and sends a request to the server.

[0692] Step 7:

[0693] The server feeds the user's request to the trained model and retrieves the relevant information that the model returns, including emails, chat messages, and documents related to the specified project and time period.

[0694] Step 8:

[0695] The server formats the acquired information and converts it into a format that is easy for the user to understand. The information is sent to the device and displayed in an appropriate format (e.g., list format, timeline format).

[0696] Step 9:

[0697] The user can check the information provided on the terminal and smoothly carry out the necessary handover and work. If there are any further questions or additional requests, they can send a request to the server again using the same procedure.

[0698] Example 1

[0699] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0700] The current business handover process relies on manual work and individual knowledge, resulting in low efficiency and causing problems with business continuity when employees are transferred or leave the company. Furthermore, business-related data exists in a wide variety of formats, making it difficult to quickly and accurately extract the necessary information. This negatively impacts the efficiency and completeness of business operations. Furthermore, the lack of an effective means to quickly obtain the information necessary for business operations can delay the entire business process.

[0701] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0702] In this invention, the server includes means for acquiring collected business-related data at a scheduled time, means for integrating the acquired business-related data into a database, means for training a natural language processing model using the integrated data, means for receiving a request in the form of a prompt sentence from a user, means for searching for information based on the user request using the trained model, and means for providing the search results to a user terminal. This enables efficient collection, integration, and training of business-related data and rapid information provision. As a result, smooth business handover and improved business continuity and efficiency are achieved.

[0703] "Collected business-related data" is a set of data related to a company's business activities, including email data, chat data, and file storage data.

[0704] The "measure for acquiring data at a fixed time" is a mechanism by which the server automatically collects business-related data at a preset time.

[0705] "Means of integrating into a database" refers to the process of compiling and integrating various collected data into a single database.

[0706] "Means for training a natural language processing model" refers to a method for using integrated data to train an AI model to perform natural language processing.

[0707] The "means for receiving a request in the form of a prompt sentence" is an interface for receiving a request input by a user in a specific format and transmitting it to a server.

[0708] "Means for searching for information based on a user request using a trained model" is a process of searching for relevant information in response to a user request using an already trained natural language processing model.

[0709] The "means for providing search results to a user terminal" refers to a mechanism by which the server transmits the search results to the user terminal and provides them in a displayable or usable form.

[0710] This invention is a system in which a server collects business-related data, integrates and learns from it, and quickly and accurately provides the business information required by users. The main functions of this system include collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[0711] Collection of work-related data

[0712] The server first connects to the company's email server and collects all emails sent and received by employees for work. Then, the server connects to communication tools (e.g., chat apps) and collects all chat logs related to work. Furthermore, the server connects to a file storage system and collects all documents related to work. Specifically, the server uses the following hardware and software:

[0713] Email server: Retrieves email data using the IMAP protocol.

[0714] Chat app: Collect chat logs using REST API.

[0715] File storage system: Retrieves documents using the SFTP protocol.

[0716] For example, the server connects to an email server every night at midnight to retrieve new emails for that day. The server also connects to a chat tool used by the company at a fixed time (for example, every hour) to retrieve new chat messages sent and received during that time. For this purpose, a scheduling function (for example, a Cron job) is set up on the server to connect at fixed times.

[0717] Data Integration and Learning

[0718] The server then integrates the collected data into a database. Specifically, the server processes the data as follows:

[0719] Database: Use a relational database management system such as MySQL and insert each piece of data into the corresponding table.

[0720] ETL scripts: Data mapping and cleansing are performed using temporary tables to ensure data integrity.

[0721] The server then uses the integrated data to train a natural language processing model. Specifically, the server uses the following techniques:

[0722] Python environment: Build and train natural language processing models using TensorFlow and Keras.

[0723] Data preprocessing: Tokenization, text vectorization, etc.

[0724] Save the trained model: Save it to disk in HDF5 format or similar.

[0725] Providing information in response to user requests

[0726] When a user needs specific information for work handover or information gathering, they send a request to the server via their terminal. Specifically, the process is as follows:

[0727] Prompt input: A user accesses a dedicated web application in a browser and inputs a request such as: "Show me all emails and important chat logs related to Project A from the past three months."

[0728] HTTP communication: The terminal sends this prompt text to the server using an HTTP POST request.

[0729] The server uses the trained model to search for relevant information based on the user's request, then sends the search results back to the user's device, where the user can view the results. Specifically, it works as follows:

[0730] Query Generation: A natural language processing model analyzes the user's input and generates an appropriate SQL query.

[0731] Retrieving search results: The server executes the generated query and extracts the relevant data from the database.

[0732] Send results: The search results are converted into JSON format and sent to the device.

[0733] The terminal parses the received JSON data and displays it to the user in a table or report format on the web application.

[0734] As described above, the present invention realizes efficient collection, integration, and learning of business-related data, and rapid information provision, thereby achieving smooth business handover and improved business continuity and efficiency.

[0735] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0736] Step 1: Collect business-related data

[0737] The server connects to the company's email server and collects all emails sent and received by employees for work. The server uses the IMAP protocol to access the mail server and retrieves new emails. Next, the server connects to communication tools (e.g., chat apps) using REST APIs and collects all chat logs related to work. Finally, the server connects to the file storage system using the SFTP protocol and collects all documents related to work.

[0738] Inputs: email servers, chat apps, file storage systems

[0739] Output: New email data, chat log data, document data

[0740] What it does: The server runs a cron job every night at midnight to collect new emails from the email server for the day, connects to the company's chat tool every hour to retrieve new chat messages, and periodically downloads important documents from the file storage system.

[0741] Step 2: Integrate the data

[0742] The server integrates the collected email, chat, and document data into a database. The server connects to a MySQL database and inserts each piece of data into the corresponding table. To maintain data consistency, temporary tables are used, and data mapping and cleaning are performed.

[0743] Input: New email data, chat log data, document data

[0744] Output: Integrated database

[0745] What happens: The server runs the ETL (Extract, Transform, Load) script to insert various data into the database, including storing new email data in the email table, chat logs in the chat table, and document data in the document table.

[0746] Step 3: Train the data

[0747] The server uses the integrated data to train a natural language processing model. The server sets up a Python environment and uses TensorFlow and Keras to build and train the natural language processing model. As data preprocessing, it performs tokenization and text vectorization, and saves the trained model.

[0748] Input: Integrated database

[0749] Output: Trained AI model

[0750] How it works: Every night, the server runs a Python script to train a natural language processing model based on the integrated data. After training is complete, the model is saved in an HDF5 file.

[0751] Step 4: Receiving a User Request

[0752] A user requests specific information for the purpose of task handover or information gathering. The user accesses a dedicated web application and inputs the request in the form of a prompt. The terminal then sends this request to the server using the HTTP POST method.

[0753] Input: A prompt-style request from the user

[0754] Output: HTTP POST request to the server

[0755] Specific operation: A user enters the request "Show me all emails and important chat logs related to Project A from the past three months" into a web form and presses the submit button. The device sends this request to the server.

[0756] Step 5: Search using the trained model

[0757] The server uses the trained model to search for relevant information based on the user's request. A natural language processing model analyzes the user's input and generates an appropriate SQL query. The server executes the query and extracts the relevant data from the database.

[0758] Input: A request in the form of a prompt, a trained AI model

[0759] Output: Search result data

[0760] Specific operation: The server parses the Japanese prompt and generates an SQL query to retrieve email data and chat logs related to "Project A" from the database. The server executes this query to extract the required data.

[0761] Step 6: Serving search results

[0762] The server returns the search results to the user's device. The search results are converted to JSON format and sent as an HTTP response. The device parses the received JSON data and displays it in a table or report format on the web application.

[0763] Input: Search result data

[0764] Output: Data displayed on the user's terminal

[0765] Specific operation: The server converts the search results into JSON format and sends it to the device as an HTTP response. The device then analyzes the received JSON data and displays it to the user on the web application. Specifically, the data is organized in tabular or graph format, allowing the user to see the information they need at a glance.

[0766] (Application example 1)

[0767] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0768] In modern industrial environments, large amounts of data are generated, including equipment operation logs, sensor data, and quality inspection data. Properly managing this data and ensuring smooth handovers when employees are transferred or leave the company are key issues for improving business continuity and efficiency. However, with conventional systems, the collection, integration, and learning of business-related data is often done manually, resulting in poor business efficiency. Furthermore, the inability to provide appropriate information quickly has led to numerous problems with business handover and management.

[0769] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0770] In this invention, the server includes means for collecting business-related data, means for integrating and training the collected business-related data, means for providing related information from a trained model in response to a user request, means for collecting operation logs, sensor data, and quality inspection data from industrial equipment, means for supplying the collected industrial equipment data to a machine learning model for integration and training, and means for using the machine learning model to extract and provide related information in response to inquiries from engineers. This enables the efficient collection and management of large amounts of business-related data in industrial environments, enabling smooth handover in the event of transfers or retirements, as well as improving business continuity and efficiency.

[0771] "Business-related data" refers to data including email data, chat data, document data, and operation logs, sensor data, and quality inspection data generated by industrial equipment when conducting business within a company.

[0772] "Means of collection" refers to the functions and technologies used by servers or other devices to access various data sources via a network and collect the required data.

[0773] "Means of integration and learning" refers to the functions and technologies that centrally manage collected data and supply it to AI models or machine learning models for analysis and learning.

[0774] "Means of providing relevant information from a trained model" refers to functions and technologies that quickly extract and provide necessary information from a pre-trained AI model in response to a user request.

[0775] "Industrial equipment" refers to machines and devices used in manufacturing, testing, processing, or other industrial processes.

[0776] "Operation log" refers to operation history data recorded when industrial equipment is in operation.

[0777] "Sensor data" refers to various measurement data acquired by sensors attached to industrial equipment.

[0778] "Quality inspection data" refers to data acquired by inspection equipment to evaluate the quality of a product or process.

[0779] "Means of supplying data to a machine learning model for integration and learning" refers to the functions and technologies for inputting various collected data into a machine learning algorithm for integration and learning, and extracting useful insights and patterns from the data.

[0780] "Means for extracting and providing relevant information in response to inquiries from engineers" refers to the functions and technologies for extracting appropriate information from collected and learned data based on inquiries from engineers who manage and operate industrial equipment, and providing it to the engineers.

[0781] A system for implementing the present invention utilizes various hardware and software components. Specific embodiments are described below.

[0782] 1. Data Collection Module

[0783] The server collects business-related data such as operation logs of industrial equipment, sensor data, and quality inspection data in real time via the network. For example, it has a function to obtain data from sensors using APIs and record operation logs.

[0784] The specific software used includes a REST API for data retrieval and a database system (e.g., Elasticsearch). The server periodically collects information from these data sources and stores it in a database.

[0785] 2. Data Integration and Learning Module

[0786] The server centrally manages the various collected data by periodically retrieving data stored in a database system such as Elasticsearch and providing it to the machine learning model.

[0787] The specific software used is the Hugging Face transformers library, which uses natural language processing techniques to analyze the data and periodically train the model.

[0788] 3. User Request Module

[0789] When a user sends a request to the system using an administrative terminal, the server uses a pre-trained machine learning model to quickly search and extract relevant information and provide it to the user.

[0790] For example, if a technician requests "Show me the quality inspection data for the past three months," the server will search for the relevant data in Elasticsearch and provide the relevant information to the technician, enabling them to quickly obtain the information they need and improving work efficiency.

[0791] Examples of prompts:

[0792] "Show me the quality inspection data for the past three months"

[0793] As described above, the embodiment of the present invention is composed of three main modules: data collection, data integration and learning, and user request response, which enable efficient collection and management of industrial equipment data and enable users to quickly obtain the information they need.

[0794] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0795] Step 1: Data collection

[0796] The server accesses various data sources over the network. Specifically, it periodically sends requests to the APIs of industrial equipment to obtain operation logs, sensor data, and quality inspection data. The input is real-time data from the equipment, and the output is the collected data stream. The server converts this data into an appropriate format and stores it in a database.

[0797] Step 2: Data integration

[0798] The server accumulates the collected data in a database such as Elasticsearch. Daily data is linked to create a single centralized database. The input is various business-related data, and the output is the integrated database. Specifically, the server retrieves new data at the end of each day and adds it to the centralized database.

[0799] Step 3: Learning Modules

[0800] The server uses the merged data to train a machine learning model (for example, using Hugging Face's transformers library). The input is the merged database, and the output is the trained model. Specifically, the server periodically retrieves the latest data from the database every night and feeds it to the model for training.

[0801] Step 4: User request reception

[0802] A user uses a terminal to request specific business-related information. The request is entered in natural language. The input is the request sentence from the user, and the output is the analysis result of the request content. In concrete terms, the user enters the request through the terminal interface.

[0803] Step 5: Information Search

[0804] The server uses the trained model to search for relevant information from Elasticsearch based on the user request. The input is the request analysis result, and the output is the searched business-related information. Specifically, the server generates an Elasticsearch query and searches for the appropriate information in the database.

[0805] Step 6: Provide information

[0806] The server organizes the search results and provides them to the user. The input is the business-related information that was searched, and the output is the organized information that is provided to the user. Specifically, the server compactly organizes the search results and displays them on the user's terminal. At this time, the information is converted into the required format according to the content of the request.

[0807] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0808] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[0809] This system consists of the following steps: collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and provision of information in response to user requests.

[0810] Collection of work-related data

[0811] The server connects to the company's email server, communication tools (e.g., chat apps), and file storage system to collect all employees' emails, chat logs, and documents. These data are important sources of information for business activities, and are all stored in a database in text format or in its original form.

[0812] Data Integration and Learning

[0813] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model using natural language processing technology, which learns about the details of the work and the project. This allows it to provide more accurate information in preparation for the next user request.

[0814] Emotion recognition by emotion engine

[0815] The server includes an emotion engine that recognizes the user's emotions from the collected work-related data and user interactions. The emotion engine analyzes text data and voice data and can classify the user's emotional state (e.g., stress, joy, anger, etc.). This emotion data is reflected in the analysis of user requests and the provision of information.

[0816] Examples:

[0817] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[0818] Providing information in response to user requests

[0819] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[0820] Examples:

[0821] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[0822] Summary of embodiments

[0823] The system of the present invention consists of four steps: automatic collection of work-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests. This not only significantly improves the efficiency of work handover compared to conventional manual work handovers, but also enables flexible information provision according to the user's emotional state, improving the continuity and efficiency of work.

[0824] The processing flow will be explained below.

[0825] Step 1:

[0826] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts them into text format and stores them in a database. Collection is performed automatically at a fixed time during the night (e.g., 2:00 AM), and duplicate data is removed.

[0827] Step 2:

[0828] The server connects to the company's communication tool (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, the content is converted into text format and stored in a database. Chat logs are collected automatically at regular intervals (e.g., every hour).

[0829] Step 3:

[0830] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, modified, and deleted by all employees. When it finds new or updated documents, it saves their contents in a database in text format or in their original format.

[0831] Step 4:

[0832] The server then integrates the collected email, chat, and document data, which involves bringing together data from different data sources into a single, unified data set.

[0833] Step 5:

[0834] The server feeds the combined dataset into a natural language processing-based AI model to train it, allowing it to learn the details of the job and project and prepare for the next user request.

[0835] Step 6:

[0836] The server uses an emotion engine to recognize emotions from the collected data and user interactions. The emotion engine analyzes text and voice data to classify the user's emotional state (e.g., joy, anger, stress). This emotion data is fed into a trained model and used to improve the quality of information provided.

[0837] Step 7:

[0838] When a user takes over a new project, they use their device to enter the project name and request details (e.g., the period for which they want information), and then send a request to the server. The user's emotional state is also considered as part of the request.

[0839] Step 8:

[0840] The server uses trained models and an emotion engine to search and extract relevant information that matches the user's request, retrieving emails, chat logs, and documents related to the requested project or timeframe, and tailoring the response based on the user's emotional state.

[0841] Step 9:

[0842] The server then formats the extracted information into an easy-to-understand format, providing responses such as summaries, highlighting key points, or adding additional details, all of which are customized based on the user's emotional state.

[0843] Step 10:

[0844] The terminal displays the formatted information to the user, who can then review it and quickly take over the task or perform any necessary actions. If the user has further questions or requests, they can send a request to the server again using the same procedure.

[0845] Through the above process, this system significantly improves the efficiency of manual work handover and provides flexible information that responds to the user's emotions.

[0846] Example 2

[0847] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0848] In conventional business handover systems, data collection and integration is done manually, which is inefficient and requires time and effort. Furthermore, the system does not provide information that takes into account the user's emotional state, which can result in inappropriate responses. This results in problems such as reduced business continuity and efficiency.

[0849] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0850] In this invention, the server includes a means for collecting business-related data, a means for integrating and learning the collected business-related data, a means for providing related information from a trained model in response to a user request, and a means for recognizing the user's emotional state and adjusting the information provided in accordance with the emotional state, thereby enabling efficient business handover and providing appropriate information based on the user's emotional state.

[0851] "Business-related data" refers to data that includes information about business activities, such as email data, chat data, and document data within a company.

[0852] "Means of collection" refers to the software and hardware configuration used to obtain data from email servers, chat apps, file storage systems, etc.

[0853] An "integration and training tool" is software and processes that convert collected data into a unified format and feed it into a machine learning model for training.

[0854] "Means for providing relevant information in response to a request" refers to a system that has the function of searching for and providing the desired information from a trained model in response to a user request.

[0855] "Means for recognizing emotional states" refers to an emotion analysis engine and related technologies that analyze text and voice data and classify a user's emotions (e.g., stress, joy, anger, etc.).

[0856] The "means for adjusting information provision" is a system that has the functionality to adjust the content and format of responses taking into account the recognized emotional state of the user.

[0857] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[0858] Collection of work-related data

[0859] The server connects to the company's email server, communication tools, and file storage system to collect all employee emails, chat logs, and documents. For example, a commonly used email server is Exchange Server. Slack is also available as a communication tool, and Google Drive is an example of a file storage system. These data are important sources of information for business activities, and are all stored in a database in text format or in their original form.

[0860] Data Integration and Learning

[0861] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model that uses natural language processing technology. For example, AI models such as BERT and GPT-3 are used. This allows the model to learn about the details of the work and projects, enabling it to provide more accurate information in preparation for the next user request.

[0862] Emotion recognition by emotion engine

[0863] The server includes an emotion engine that recognizes emotions from collected work-related data and user interactions. The emotion engine analyzes text and voice data to classify the user's emotional state (e.g., stress, joy, anger, etc.). Specifically, it uses the NLP library spaCy and speech analysis tools IBM Watson and Google Cloud Speech-to-Text. This emotion data is reflected in the analysis of user requests and the information provided.

[0864] Examples:

[0865] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[0866] Providing information in response to user requests

[0867] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[0868] Examples:

[0869] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[0870] Example prompt sentence:

[0871] "Please provide a summary of relevant emails and important chat logs for Project B."

[0872] "Show me the detailed report on Project B."

[0873] As described above, this system sequentially processes each stage of automatic collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests, thereby helping to improve business efficiency and user satisfaction.

[0874] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0875] Step 1:

[0876] The server connects to the company's email server, chat app, and file storage system to collect business-related data, specifically retrieving email data from Microsoft Exchange Server, collecting chat logs using the Slack API, and downloading document data using the Google Drive API.

[0877] Input: APIs for email servers, chat apps, and file storage systems

[0878] Output: A collection of collected email data, chat data, and document data

[0879] Specific operation: The server executes the Python script, collects data through each API, and stores the collected data in a database.

[0880] Step 2:

[0881] The server consolidates the collected email data, chat data, and document data. A data cleaning tool (e.g., Pandas) is used to standardize the data format and remove unnecessary data.

[0882] Input: Email data, chat data, and document data collected in Step 1

[0883] Output: A consolidated dataset

[0884] Specific operation: The server uses a Python script to combine data from different sources into a single data frame, eliminate duplicate data, and convert it into a unified format.

[0885] Step 3:

[0886] The server then feeds this combined dataset into a natural language processing AI model for training, using, for example, BERT or GPT-3 to learn job descriptions and project details from the data.

[0887] Input: The dataset merged in step 2

[0888] Output: Trained AI model

[0889] Specific operation: The server uses TensorFlow or PyTorch to input the collected and integrated datasets into the AI ​​model for training.

[0890] Step 4:

[0891] The server uses an emotion engine to analyze collected task-related data and user interactions to recognize the user's emotional state, using NLP libraries (e.g., spaCy) and speech analysis tools (e.g., IBM Watson, Google Cloud Speech-to-Text).

[0892] Input: Text data, audio data

[0893] Output: Recognized emotional state data

[0894] How it works: The server parses text data using the spaCy library, converts audio data into text using the Google Cloud Speech-to-Text API, and classifies emotions from the text.

[0895] Step 5:

[0896] The user uses a terminal to request information related to a new project from the server, entering information such as the project name and required period into a dedicated form and submitting it.

[0897] Input: Request information from the user (project name, period, etc.)

[0898] Output: Request to server

[0899] Specific operation: The user enters information into a form on the terminal and sends a request to the server.

[0900] Step 6:

[0901] The server receives and analyzes user requests, using trained AI models and an emotion engine to extract the most relevant information for the request and tailor the response to take into account the user's emotional state.

[0902] Input: User request information, trained AI model, emotion data

[0903] Output: Information to be provided to the user

[0904] How it works: The server analyzes the request, uses AI models to search and extract relevant information, analyzes the user's emotions with an emotion engine, and returns the information in an appropriate format. For example, if stress is detected, it will provide a summary, and if joy is detected, it will provide detailed information.

[0905] This makes task handover more efficient and allows flexible provision of information according to the user's emotional state.

[0906] (Application example 2)

[0907] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[0908] In workplaces where industrial robots are used, worker rotation and training of new workers requires a lot of time and effort. In particular, if transfers and handover of projects and work to new workers are not carried out promptly, the efficiency and productivity of the entire business may decline. Furthermore, the emotional state of workers can affect the quality of information they provide, and situations where workers feel stressed or anxious increase the risk of work errors. To solve these issues, effective and emotion-aware information provision is required.

[0909] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work-related data, means for integrating and learning the collected work-related data, means for providing related information from a trained model in response to a user request, means for recognizing emotions and adjusting a response, and means for streamlining task handover in an industrial robot in response to a user request. This enables the industrial robot to streamline task handover on site and provide flexible information based on emotion recognition.

[0910] "Business-related data" refers to any digital information related to business activities, such as email data, chat data, and business document data.

[0911] "Means of collection" refers to technology or equipment that automatically obtains the required data from designated data sources (e.g., email servers, chat applications, document management systems, etc.).

[0912] An "integration and learning means" is a process or system that centrally integrates collected business-related data and applies machine learning algorithms to generate a model based on that data.

[0913] "Means for providing relevant information from a trained model in response to a user request" refers to technology or equipment for extracting and providing appropriate information from a pre-trained model based on a request entered by a user.

[0914] "Means for recognizing emotions and tailoring responses" refers to systems or algorithms that analyze a user's text or voice, identify their emotional state, and generate appropriate responses based on those emotions.

[0915] An "industrial robot" is an automated machine or device designed to perform tasks in production or manufacturing settings.

[0916] "Means for streamlining the handover of tasks" refers to technologies and systems that enable users to quickly and accurately communicate details of their work and projects to new personnel.

[0917] To realize the present invention in an industrial robot, the following system configuration and processing are required.

[0918] Hardware Configuration

[0919] 1. Robots: Industrial robots used in production (e.g., humanoid robots)

[0920] 2. Server: A server for data processing and model training

[0921] 3. Network: Network environment for connecting the robot and the server

[0922] 4. Sensors: Sensors to collect audio and video information for emotion recognition

[0923] Software Configuration

[0924] 1. Data collection software: Apache Kafka or a similar real-time data streaming platform

[0925] 2. Data search software: Search engines such as Elasticsearch

[0926] 3. Machine learning models: OpenAI GPT-4 and TensorFlow models for sentiment analysis

[0927] 4. Emotion Recognition Software: Emotion analysis software such as Natural Language Toolkit (NLTK) and Nvidia Clara

[0928] Processing Overview

[0929] 1. Data collection: The server collects email, chat, and business document data from various data sources within the company in real time. Apache Kafka can be used to efficiently ingest this data.

[0930] 2. Data integration and training: The collected data is stored in Elasticsearch, and OpenAI GPT-4 is used to integrate and train this data. During this process, business details and relevant information are filtered by the machine learning model.

[0931] 3. Emotion Recognition: The robot's built-in microphone and camera detect the user's voice and facial expressions, and analyzes their emotional state using TensorFlow. This emotional data is sent to the server when the user makes a request.

[0932] 4. Information provision: When a user requests specific information through the robot, the server analyzes the request based on the trained model and emotion recognition results, and adjusts the response according to the user's emotional state to provide appropriate information to the user.

[0933] Specific examples

[0934] If a user requests detailed instructions for the next step in a task, the server generates a prompt like this and provides it to OpenAI GPT-4:

[0935] Example prompt sentence:

[0936] "A worker has requested more details about process X. He seems a little nervous. What information should he provide first? Please keep it concise and focus on five points."

[0937] In response to this prompt, OpenAI GPT-4 provides a concise summary of five key points, allowing users to get the information they need quickly and accurately. Emotion recognition also allows for a more concise explanation if the user is stressed, or more detailed information if the user is emotionally stable.

[0938] In this way, the system of the present invention can improve the work efficiency of industrial robots and provide information flexibly according to the emotional state of the user.

[0939] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0940] Step 1:

[0941] The server collects business-related data from email servers, chat applications, document management systems, etc. Specifically, it uses Apache Kafka to stream this data in real time and store it in Elasticsearch. The input is data from each data source, and the output is the integrated data stored in Elasticsearch.

[0942] Step 2:

[0943] The server uses business-related data stored in Elasticsearch and trains the data using the OpenAI GPT-4 model. At this stage, the consolidated data is fed into a machine learning algorithm, which analyzes the data and learns about the business and project details. The input is the consolidated data from Elasticsearch, and the output is a trained GPT-4 model.

[0944] Step 3:

[0945] The robot's built-in microphones and cameras collect the user's voice and video and send them to a server. The server uses TensorFlow to analyze the voice and video data and identify the user's emotional state. The input is the voice and video data, and the output is a classification of the user's emotional state.

[0946] Step 4:

[0947] The user sends a question or request to the robot. For example, "Please tell me the detailed steps for the next work step." This request is sent to the server. The input is the user's request, and the output is its content.

[0948] Step 5:

[0949] The server analyzes the user's request and generates and supplies a prompt to the generative AI model (GPT-4). For example, the prompt might read, "A worker has requested details about process X. He seems a little nervous. What information should he provide first? Please answer briefly, focusing on five key points." The input is the user's request and emotional state, and the output is the prompt.

[0950] Step 6:

[0951] Based on the generated prompt, the OpenAI GPT-4 model generates appropriate information according to the request. The server analyzes this information and reconstructs it in a form that is easy for the user to understand. The input is the prompt, and the output is the generated information.

[0952] Step 7:

[0953] The server sends the generated information to the robot, which then provides it to the user. For example, five key points can be briefly explained. The input is the generated information, and the output is the specific information provided to the user.

[0954] This series of processes enables industrial robots to provide appropriate and efficient information taking into account the user's emotional state.

[0955] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0956] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0957] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[0958] [Fourth embodiment]

[0959] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0960] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0961] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0962] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0963] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0964] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0965] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0966] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0967] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0968] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0969] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0970] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0971] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[0972] The system of this invention allows the server to collect business-related data, integrate it, and learn from it to quickly and accurately provide the business information required by the user. By implementing this system, even in the event of transfer or retirement, business handover can be carried out smoothly, improving the continuity and efficiency of business operations.

[0973] This system mainly consists of the following steps: collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[0974] Collection of work-related data

[0975] The server first connects to the company's email server and collects all emails sent and received by employees for work, then connects to communication tools (e.g., chat apps) and collects all chat logs related to work, and finally connects to the file storage system and collects all documents related to work.

[0976] Examples:

[0977] The server connects to the email server every night at midnight and collects new mail for that day.

[0978] The server connects to the chat tool used by the company at regular intervals (for example, every hour) and retrieves new chat messages sent and received during that time.

[0979] Data Integration and Learning

[0980] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model (natural language processing model) for learning. The server automates this process and periodically updates the learning model.

[0981] Examples:

[0982] The server compiles the latest email data, chat data, and document data into a single database on a daily basis.

[0983] Using this database, the AI ​​model is trained overnight, and by the next morning a model reflecting the latest information is completed.

[0984] Providing information in response to user requests

[0985] When a user needs specific information for work handover or information understanding, the user sends a request to the server via the device, and the server uses the trained model to quickly extract information relevant to the user's request and provide it to the device.

[0986] Examples:

[0987] A user who has been newly assigned to a project requests on their device, "Show me all emails and important chat logs related to Project A from the past three months."

[0988] The server uses the trained model to search for relevant emails and chat logs, extract that information, and display it on the device.

[0989] Summary of embodiments

[0990] The system of this invention combines three major steps: automatic collection of business-related data, data integration and learning, and information provision based on user requests. This significantly simplifies the traditional manual process of business handover, and enables smooth business continuity even in the event of transfers or retirement. Furthermore, by periodically learning the latest data, the system can always provide the latest information.

[0991] The processing flow will be explained below.

[0992] Step 1:

[0993] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts their contents into plain text and stores them in a database. It also deduplicates data already collected in the past.

[0994] Step 2:

[0995] The server connects to the company's communication tools (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, it converts the content into text format and stores it in a database. It then compares it with past logs to incorporate only new data, improving efficiency.

[0996] Step 3:

[0997] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, deleted, or updated by employees. When it finds new or changed documents, it saves the contents to a database in text format or raw format.

[0998] Step 4:

[0999] The server consolidates the collected email, chat, and document data, which involves combining all new and updated data in the database into one unified data set.

[1000] Step 5:

[1001] The server feeds the integrated dataset to an AI model (natural language processing model) to train the model, which learns the details of the business and project and prepares for the next user request.

[1002] Step 6:

[1003] When a user takes over a new project, he or she inputs specific information (e.g., project name, period for which information is desired) from the terminal and sends a request to the server.

[1004] Step 7:

[1005] The server feeds the user's request to the trained model and retrieves the relevant information that the model returns, including emails, chat messages, and documents related to the specified project and time period.

[1006] Step 8:

[1007] The server formats the acquired information and converts it into a format that is easy for the user to understand. The information is sent to the device and displayed in an appropriate format (e.g., list format, timeline format).

[1008] Step 9:

[1009] The user can check the information provided on the terminal and smoothly carry out the necessary handover and work. If there are any further questions or additional requests, they can send a request to the server again using the same procedure.

[1010] Example 1

[1011] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1012] The current business handover process relies on manual work and individual knowledge, resulting in low efficiency and causing problems with business continuity when employees are transferred or leave the company. Furthermore, business-related data exists in a wide variety of formats, making it difficult to quickly and accurately extract the necessary information. This negatively impacts the efficiency and completeness of business operations. Furthermore, the lack of an effective means to quickly obtain the information necessary for business operations can delay the entire business process.

[1013] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1014] In this invention, the server includes means for acquiring collected business-related data at a scheduled time, means for integrating the acquired business-related data into a database, means for training a natural language processing model using the integrated data, means for receiving a request in the form of a prompt sentence from a user, means for searching for information based on the user request using the trained model, and means for providing the search results to a user terminal. This enables efficient collection, integration, and training of business-related data and rapid information provision. As a result, smooth business handover and improved business continuity and efficiency are achieved.

[1015] "Collected business-related data" is a set of data related to a company's business activities, including email data, chat data, and file storage data.

[1016] The "measure for acquiring data at a fixed time" is a mechanism by which the server automatically collects business-related data at a preset time.

[1017] "Means of integrating into a database" refers to the process of compiling and integrating various collected data into a single database.

[1018] "Means for training a natural language processing model" refers to a method for using integrated data to train an AI model to perform natural language processing.

[1019] The "means for receiving a request in the form of a prompt sentence" is an interface for receiving a request input by a user in a specific format and transmitting it to a server.

[1020] "Means for searching for information based on a user request using a trained model" is a process of searching for relevant information in response to a user request using an already trained natural language processing model.

[1021] The "means for providing search results to a user terminal" refers to a mechanism by which the server transmits the search results to the user terminal and provides them in a displayable or usable form.

[1022] This invention is a system in which a server collects business-related data, integrates and learns from it, and quickly and accurately provides the business information required by users. The main functions of this system include collecting business-related data, integrating and learning from the data, and providing information in response to user requests.

[1023] Collection of work-related data

[1024] The server first connects to the company's email server and collects all emails sent and received by employees for work. Then, the server connects to communication tools (e.g., chat apps) and collects all chat logs related to work. Furthermore, the server connects to a file storage system and collects all documents related to work. Specifically, the server uses the following hardware and software:

[1025] Email server: Retrieves email data using the IMAP protocol.

[1026] Chat app: Collect chat logs using REST API.

[1027] File storage system: Retrieves documents using the SFTP protocol.

[1028] For example, the server connects to an email server every night at midnight to retrieve new emails for that day. The server also connects to a chat tool used by the company at a fixed time (for example, every hour) to retrieve new chat messages sent and received during that time. For this purpose, a scheduling function (for example, a Cron job) is set up on the server to connect at fixed times.

[1029] Data Integration and Learning

[1030] The server then integrates the collected data into a database. Specifically, the server processes the data as follows:

[1031] Database: Use a relational database management system such as MySQL and insert each piece of data into the corresponding table.

[1032] ETL scripts: Data mapping and cleansing are performed using temporary tables to ensure data integrity.

[1033] The server then uses the integrated data to train a natural language processing model. Specifically, the server uses the following techniques:

[1034] Python environment: Build and train natural language processing models using TensorFlow and Keras.

[1035] Data preprocessing: Tokenization, text vectorization, etc.

[1036] Save the trained model: Save it to disk in HDF5 format or similar.

[1037] Providing information in response to user requests

[1038] When a user needs specific information for work handover or information gathering, they send a request to the server via their terminal. Specifically, the process is as follows:

[1039] Prompt input: A user accesses a dedicated web application in a browser and inputs a request such as: "Show me all emails and important chat logs related to Project A from the past three months."

[1040] HTTP communication: The terminal sends this prompt text to the server using an HTTP POST request.

[1041] The server uses the trained model to search for relevant information based on the user's request, then sends the search results back to the user's device, where the user can view the results. Specifically, it works as follows:

[1042] Query Generation: A natural language processing model analyzes the user's input and generates an appropriate SQL query.

[1043] Retrieving search results: The server executes the generated query and extracts the relevant data from the database.

[1044] Send results: The search results are converted into JSON format and sent to the device.

[1045] The terminal parses the received JSON data and displays it to the user in a table or report format on the web application.

[1046] As described above, the present invention realizes efficient collection, integration, and learning of business-related data, and rapid information provision, thereby achieving smooth business handover and improved business continuity and efficiency.

[1047] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1048] Step 1: Collect business-related data

[1049] The server connects to the company's email server and collects all emails sent and received by employees for work. The server uses the IMAP protocol to access the mail server and retrieves new emails. Next, the server connects to communication tools (e.g., chat apps) using REST APIs and collects all chat logs related to work. Finally, the server connects to the file storage system using the SFTP protocol and collects all documents related to work.

[1050] Inputs: email servers, chat apps, file storage systems

[1051] Output: New email data, chat log data, document data

[1052] What it does: The server runs a cron job every night at midnight to collect new emails from the email server for the day, connects to the company's chat tool every hour to retrieve new chat messages, and periodically downloads important documents from the file storage system.

[1053] Step 2: Integrate the data

[1054] The server integrates the collected email, chat, and document data into a database. The server connects to a MySQL database and inserts each piece of data into the corresponding table. To maintain data consistency, temporary tables are used, and data mapping and cleaning are performed.

[1055] Input: New email data, chat log data, document data

[1056] Output: Integrated database

[1057] What happens: The server runs the ETL (Extract, Transform, Load) script to insert various data into the database, including storing new email data in the email table, chat logs in the chat table, and document data in the document table.

[1058] Step 3: Train the data

[1059] The server uses the integrated data to train a natural language processing model. The server sets up a Python environment and uses TensorFlow and Keras to build and train the natural language processing model. As data preprocessing, it performs tokenization and text vectorization, and saves the trained model.

[1060] Input: Integrated database

[1061] Output: Trained AI model

[1062] How it works: Every night, the server runs a Python script to train a natural language processing model based on the integrated data. After training is complete, the model is saved in an HDF5 file.

[1063] Step 4: Receiving a User Request

[1064] A user requests specific information for the purpose of task handover or information gathering. The user accesses a dedicated web application and inputs the request in the form of a prompt. The terminal then sends this request to the server using the HTTP POST method.

[1065] Input: A prompt-style request from the user

[1066] Output: HTTP POST request to the server

[1067] Specific operation: A user enters the request "Show me all emails and important chat logs related to Project A from the past three months" into a web form and presses the submit button. The device sends this request to the server.

[1068] Step 5: Search using the trained model

[1069] The server uses the trained model to search for relevant information based on the user's request. A natural language processing model analyzes the user's input and generates an appropriate SQL query. The server executes the query and extracts the relevant data from the database.

[1070] Input: A request in the form of a prompt, a trained AI model

[1071] Output: Search result data

[1072] Specific operation: The server parses the Japanese prompt and generates an SQL query to retrieve email data and chat logs related to "Project A" from the database. The server executes this query to extract the required data.

[1073] Step 6: Serving search results

[1074] The server returns the search results to the user's device. The search results are converted to JSON format and sent as an HTTP response. The device parses the received JSON data and displays it in a table or report format on the web application.

[1075] Input: Search result data

[1076] Output: Data displayed on the user's terminal

[1077] Specific operation: The server converts the search results into JSON format and sends it to the device as an HTTP response. The device then analyzes the received JSON data and displays it to the user on the web application. Specifically, the data is organized in tabular or graph format, allowing the user to see the information they need at a glance.

[1078] (Application example 1)

[1079] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1080] In modern industrial environments, large amounts of data are generated, including equipment operation logs, sensor data, and quality inspection data. Properly managing this data and ensuring smooth handovers when employees are transferred or leave the company are key issues for improving business continuity and efficiency. However, with conventional systems, the collection, integration, and learning of business-related data is often done manually, resulting in poor business efficiency. Furthermore, the inability to provide appropriate information quickly has led to numerous problems with business handover and management.

[1081] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1082] In this invention, the server includes means for collecting business-related data, means for integrating and training the collected business-related data, means for providing related information from a trained model in response to a user request, means for collecting operation logs, sensor data, and quality inspection data from industrial equipment, means for supplying the collected industrial equipment data to a machine learning model for integration and training, and means for using the machine learning model to extract and provide related information in response to inquiries from engineers. This enables the efficient collection and management of large amounts of business-related data in industrial environments, enabling smooth handover in the event of transfers or retirements, as well as improving business continuity and efficiency.

[1083] "Business-related data" refers to data including email data, chat data, document data, and operation logs, sensor data, and quality inspection data generated by industrial equipment when conducting business within a company.

[1084] "Means of collection" refers to the functions and technologies used by servers or other devices to access various data sources via a network and collect the required data.

[1085] "Means of integration and learning" refers to the functions and technologies that centrally manage collected data and supply it to AI models or machine learning models for analysis and learning.

[1086] "Means of providing relevant information from a trained model" refers to functions and technologies that quickly extract and provide necessary information from a pre-trained AI model in response to a user request.

[1087] "Industrial equipment" refers to machines and devices used in manufacturing, testing, processing, or other industrial processes.

[1088] "Operation log" refers to operation history data recorded when industrial equipment is in operation.

[1089] "Sensor data" refers to various measurement data acquired by sensors attached to industrial equipment.

[1090] "Quality inspection data" refers to data acquired by inspection equipment to evaluate the quality of a product or process.

[1091] "Means of supplying data to a machine learning model for integration and learning" refers to the functions and technologies for inputting various collected data into a machine learning algorithm for integration and learning, and extracting useful insights and patterns from the data.

[1092] "Means for extracting and providing relevant information in response to inquiries from engineers" refers to the functions and technologies for extracting appropriate information from collected and learned data based on inquiries from engineers who manage and operate industrial equipment, and providing it to the engineers.

[1093] A system for implementing the present invention utilizes various hardware and software components. Specific embodiments are described below.

[1094] 1. Data Collection Module

[1095] The server collects business-related data such as operation logs of industrial equipment, sensor data, and quality inspection data in real time via the network. For example, it has a function to obtain data from sensors using APIs and record operation logs.

[1096] The specific software used includes a REST API for data retrieval and a database system (e.g., Elasticsearch). The server periodically collects information from these data sources and stores it in a database.

[1097] 2. Data Integration and Learning Module

[1098] The server centrally manages the various collected data by periodically retrieving data stored in a database system such as Elasticsearch and providing it to the machine learning model.

[1099] The specific software used is the Hugging Face transformers library, which uses natural language processing techniques to analyze the data and periodically train the model.

[1100] 3. User Request Module

[1101] When a user sends a request to the system using an administrative terminal, the server uses a pre-trained machine learning model to quickly search and extract relevant information and provide it to the user.

[1102] For example, if a technician requests "Show me the quality inspection data for the past three months," the server will search for the relevant data in Elasticsearch and provide the relevant information to the technician, enabling them to quickly obtain the information they need and improving work efficiency.

[1103] Examples of prompts:

[1104] "Show me the quality inspection data for the past three months"

[1105] As described above, the embodiment of the present invention is composed of three main modules: data collection, data integration and learning, and user request response, which enable efficient collection and management of industrial equipment data and enable users to quickly obtain the information they need.

[1106] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1107] Step 1: Data collection

[1108] The server accesses various data sources over the network. Specifically, it periodically sends requests to the APIs of industrial equipment to obtain operation logs, sensor data, and quality inspection data. The input is real-time data from the equipment, and the output is the collected data stream. The server converts this data into an appropriate format and stores it in a database.

[1109] Step 2: Data integration

[1110] The server accumulates the collected data in a database such as Elasticsearch. Daily data is linked to create a single centralized database. The input is various business-related data, and the output is the integrated database. Specifically, the server retrieves new data at the end of each day and adds it to the centralized database.

[1111] Step 3: Learning Modules

[1112] The server uses the merged data to train a machine learning model (for example, using Hugging Face's transformers library). The input is the merged database, and the output is the trained model. Specifically, the server periodically retrieves the latest data from the database every night and feeds it to the model for training.

[1113] Step 4: User request reception

[1114] A user uses a terminal to request specific business-related information. The request is entered in natural language. The input is the request sentence from the user, and the output is the analysis result of the request content. In concrete terms, the user enters the request through the terminal interface.

[1115] Step 5: Information Search

[1116] The server uses the trained model to search for relevant information from Elasticsearch based on the user request. The input is the request analysis result, and the output is the searched business-related information. Specifically, the server generates an Elasticsearch query and searches for the appropriate information in the database.

[1117] Step 6: Provide information

[1118] The server organizes the search results and provides them to the user. The input is the business-related information that was searched, and the output is the organized information that is provided to the user. Specifically, the server compactly organizes the search results and displays them on the user's terminal. At this time, the information is converted into the required format according to the content of the request.

[1119] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1120] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[1121] This system consists of the following steps: collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and provision of information in response to user requests.

[1122] Collection of work-related data

[1123] The server connects to the company's email server, communication tools (e.g., chat apps), and file storage system to collect all employees' emails, chat logs, and documents. These data are important sources of information for business activities, and are all stored in a database in text format or in its original form.

[1124] Data Integration and Learning

[1125] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model using natural language processing technology, which learns about the details of the work and the project. This allows it to provide more accurate information in preparation for the next user request.

[1126] Emotion recognition by emotion engine

[1127] The server includes an emotion engine that recognizes the user's emotions from the collected work-related data and user interactions. The emotion engine analyzes text data and voice data and can classify the user's emotional state (e.g., stress, joy, anger, etc.). This emotion data is reflected in the analysis of user requests and the provision of information.

[1128] Examples:

[1129] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[1130] Providing information in response to user requests

[1131] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[1132] Examples:

[1133] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[1134] Summary of embodiments

[1135] The system of the present invention consists of four steps: automatic collection of work-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests. This not only significantly improves the efficiency of work handover compared to conventional manual work handovers, but also enables flexible information provision according to the user's emotional state, improving the continuity and efficiency of work.

[1136] The processing flow will be explained below.

[1137] Step 1:

[1138] The server connects to the company's email server using the IMAP protocol and scans all employees' mailboxes. When it finds new emails, it converts them into text format and stores them in a database. Collection is performed automatically at a fixed time during the night (e.g., 2:00 AM), and duplicate data is removed.

[1139] Step 2:

[1140] The server connects to the company's communication tool (e.g., Slack) using an API and scans all channel and private message logs. When a new message is found, the content is converted into text format and stored in a database. Chat logs are collected automatically at regular intervals (e.g., every hour).

[1141] Step 3:

[1142] The server connects to the company's file storage system (e.g., Google Drive) using an API and scans all documents created, modified, and deleted by all employees. When it finds new or updated documents, it saves their contents in a database in text format or in their original format.

[1143] Step 4:

[1144] The server then integrates the collected email, chat, and document data, which involves bringing together data from different data sources into a single, unified data set.

[1145] Step 5:

[1146] The server feeds the combined dataset into a natural language processing-based AI model to train it, allowing it to learn the details of the job and project and prepare for the next user request.

[1147] Step 6:

[1148] The server uses an emotion engine to recognize emotions from the collected data and user interactions. The emotion engine analyzes text and voice data to classify the user's emotional state (e.g., joy, anger, stress). This emotion data is fed into a trained model and used to improve the quality of information provided.

[1149] Step 7:

[1150] When a user takes over a new project, they use their device to enter the project name and request details (e.g., the period for which they want information), and then send a request to the server. The user's emotional state is also considered as part of the request.

[1151] Step 8:

[1152] The server uses trained models and an emotion engine to search and extract relevant information that matches the user's request, retrieving emails, chat logs, and documents related to the requested project or timeframe, and tailoring the response based on the user's emotional state.

[1153] Step 9:

[1154] The server then formats the extracted information into an easy-to-understand format, providing responses such as summaries, highlighting key points, or adding additional details, all of which are customized based on the user's emotional state.

[1155] Step 10:

[1156] The terminal displays the formatted information to the user, who can then review it and quickly take over the task or perform any necessary actions. If the user has further questions or requests, they can send a request to the server again using the same procedure.

[1157] Through the above process, this system significantly improves the efficiency of manual work handover and provides flexible information that responds to the user's emotions.

[1158] Example 2

[1159] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1160] In conventional business handover systems, data collection and integration is done manually, which is inefficient and requires time and effort. Furthermore, the system does not provide information that takes into account the user's emotional state, which can result in inappropriate responses. This results in problems such as reduced business continuity and efficiency.

[1161] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1162] In this invention, the server includes a means for collecting business-related data, a means for integrating and learning the collected business-related data, a means for providing related information from a trained model in response to a user request, and a means for recognizing the user's emotional state and adjusting the information provided in accordance with the emotional state, thereby enabling efficient business handover and providing appropriate information based on the user's emotional state.

[1163] "Business-related data" refers to data that includes information about business activities, such as email data, chat data, and document data within a company.

[1164] "Means of collection" refers to the software and hardware configuration used to obtain data from email servers, chat apps, file storage systems, etc.

[1165] An "integration and training tool" is software and processes that convert collected data into a unified format and feed it into a machine learning model for training.

[1166] "Means for providing relevant information in response to a request" refers to a system that has the function of searching for and providing the desired information from a trained model in response to a user request.

[1167] "Means for recognizing emotional states" refers to an emotion analysis engine and related technologies that analyze text and voice data and classify a user's emotions (e.g., stress, joy, anger, etc.).

[1168] The "means for adjusting information provision" is a system that has the functionality to adjust the content and format of responses taking into account the recognized emotional state of the user.

[1169] This invention is a system in which a server collects, integrates, and learns from work-related data, recognizes user requests and emotions, and provides appropriate information. This system not only streamlines the handover of work when an employee is transferred or leaves the company, but also adjusts responses based on the user's emotional state, providing more appropriate information.

[1170] Collection of work-related data

[1171] The server connects to the company's email server, communication tools, and file storage system to collect all employee emails, chat logs, and documents. For example, a commonly used email server is Exchange Server. Slack is also available as a communication tool, and Google Drive is an example of a file storage system. These data are important sources of information for business activities, and are all stored in a database in text format or in their original form.

[1172] Data Integration and Learning

[1173] The server integrates the collected email, chat, and document data. This integrated data is fed into an AI model that uses natural language processing technology. For example, AI models such as BERT and GPT-3 are used. This allows the model to learn about the details of the work and projects, enabling it to provide more accurate information in preparation for the next user request.

[1174] Emotion recognition by emotion engine

[1175] The server includes an emotion engine that recognizes emotions from collected work-related data and user interactions. The emotion engine analyzes text and voice data to classify the user's emotional state (e.g., stress, joy, anger, etc.). Specifically, it uses the NLP library spaCy and speech analysis tools IBM Watson and Google Cloud Speech-to-Text. This emotion data is reflected in the analysis of user requests and the information provided.

[1176] Examples:

[1177] The server analyzes past emails and chat logs to learn changes in emotions (e.g., increases or decreases in stress) associated with specific projects.

[1178] Providing information in response to user requests

[1179] When a user wants to take over a new project, they input information such as the project name and required duration from their device and send a request to the server. The server uses a trained model and an emotion engine to extract information that adapts to the request and adjusts the response, taking into account the user's emotional state.

[1180] Examples:

[1181] When a user requests, "Show me the latest emails and important chat logs about Project B," the server analyzes the request with its emotion engine. If the server determines that the user is feeling stressed, it provides a summary of the key points in an easy-to-understand format. If the emotion engine detects joy, it can also provide detailed information and additional materials.

[1182] Example prompt sentence:

[1183] "Please provide a summary of relevant emails and important chat logs for Project B."

[1184] "Show me the detailed report on Project B."

[1185] As described above, this system sequentially processes each stage of automatic collection of business-related data, data integration and learning, emotion recognition using an emotion engine, and information provision based on user requests, thereby helping to improve business efficiency and user satisfaction.

[1186] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1187] Step 1:

[1188] The server connects to the company's email server, chat app, and file storage system to collect business-related data, specifically retrieving email data from Microsoft Exchange Server, collecting chat logs using the Slack API, and downloading document data using the Google Drive API.

[1189] Input: APIs for email servers, chat apps, and file storage systems

[1190] Output: A collection of collected email data, chat data, and document data

[1191] Specific operation: The server executes the Python script, collects data through each API, and stores the collected data in a database.

[1192] Step 2:

[1193] The server consolidates the collected email data, chat data, and document data. A data cleaning tool (e.g., Pandas) is used to standardize the data format and remove unnecessary data.

[1194] Input: Email data, chat data, and document data collected in Step 1

[1195] Output: A consolidated dataset

[1196] Specific operation: The server uses a Python script to combine data from different sources into a single data frame, eliminate duplicate data, and convert it into a unified format.

[1197] Step 3:

[1198] The server then feeds this combined dataset into a natural language processing AI model for training, using, for example, BERT or GPT-3 to learn job descriptions and project details from the data.

[1199] Input: The dataset merged in step 2

[1200] Output: Trained AI model

[1201] Specific operation: The server uses TensorFlow or PyTorch to input the collected and integrated datasets into the AI ​​model for training.

[1202] Step 4:

[1203] The server uses an emotion engine to analyze collected task-related data and user interactions to recognize the user's emotional state, using NLP libraries (e.g., spaCy) and speech analysis tools (e.g., IBM Watson, Google Cloud Speech-to-Text).

[1204] Input: Text data, audio data

[1205] Output: Recognized emotional state data

[1206] How it works: The server parses text data using the spaCy library, converts audio data into text using the Google Cloud Speech-to-Text API, and classifies emotions from the text.

[1207] Step 5:

[1208] The user uses a terminal to request information related to a new project from the server, entering information such as the project name and required period into a dedicated form and submitting it.

[1209] Input: Request information from the user (project name, period, etc.)

[1210] Output: Request to server

[1211] Specific operation: The user enters information into a form on the terminal and sends a request to the server.

[1212] Step 6:

[1213] The server receives and analyzes user requests, using trained AI models and an emotion engine to extract the most relevant information for the request and tailor the response to take into account the user's emotional state.

[1214] Input: User request information, trained AI model, emotion data

[1215] Output: Information to be provided to the user

[1216] How it works: The server analyzes the request, uses AI models to search and extract relevant information, analyzes the user's emotions with an emotion engine, and returns the information in an appropriate format. For example, if stress is detected, it will provide a summary, and if joy is detected, it will provide detailed information.

[1217] This makes task handover more efficient and allows flexible provision of information according to the user's emotional state.

[1218] (Application example 2)

[1219] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1220] In workplaces where industrial robots are used, worker rotation and training of new workers requires a lot of time and effort. In particular, if transfers and handover of projects and work to new workers are not carried out promptly, the efficiency and productivity of the entire business may decline. Furthermore, the emotional state of workers can affect the quality of information they provide, and situations where workers feel stressed or anxious increase the risk of work errors. To solve these issues, effective and emotion-aware information provision is required.

[1221] The identification process by the identification processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for collecting work-related data, means for integrating and learning the collected work-related data, means for providing related information from a trained model in response to a user request, means for recognizing emotions and adjusting a response, and means for streamlining task handover in an industrial robot in response to a user request. This enables the industrial robot to streamline task handover on site and provide flexible information based on emotion recognition.

[1222] "Business-related data" refers to any digital information related to business activities, such as email data, chat data, and business document data.

[1223] "Means of collection" refers to technology or equipment that automatically obtains the required data from designated data sources (e.g., email servers, chat applications, document management systems, etc.).

[1224] An "integration and learning means" is a process or system that centrally integrates collected business-related data and applies machine learning algorithms to generate a model based on that data.

[1225] "Means for providing relevant information from a trained model in response to a user request" refers to technology or equipment for extracting and providing appropriate information from a pre-trained model based on a request entered by a user.

[1226] "Means for recognizing emotions and tailoring responses" refers to systems or algorithms that analyze a user's text or voice, identify their emotional state, and generate appropriate responses based on those emotions.

[1227] An "industrial robot" is an automated machine or device designed to perform tasks in production or manufacturing settings.

[1228] "Means for streamlining the handover of tasks" refers to technologies and systems that enable users to quickly and accurately communicate details of their work and projects to new personnel.

[1229] To realize the present invention in an industrial robot, the following system configuration and processing are required.

[1230] Hardware Configuration

[1231] 1. Robots: Industrial robots used in production (e.g., humanoid robots)

[1232] 2. Server: A server for data processing and model training

[1233] 3. Network: Network environment for connecting the robot and the server

[1234] 4. Sensors: Sensors to collect audio and video information for emotion recognition

[1235] Software Configuration

[1236] 1. Data collection software: Apache Kafka or a similar real-time data streaming platform

[1237] 2. Data search software: Search engines such as Elasticsearch

[1238] 3. Machine learning models: OpenAI GPT-4 and TensorFlow models for sentiment analysis

[1239] 4. Emotion Recognition Software: Emotion analysis software such as Natural Language Toolkit (NLTK) and Nvidia Clara

[1240] Processing Overview

[1241] 1. Data collection: The server collects email, chat, and business document data from various data sources within the company in real time. Apache Kafka can be used to efficiently ingest this data.

[1242] 2. Data integration and training: The collected data is stored in Elasticsearch, and OpenAI GPT-4 is used to integrate and train this data. During this process, business details and relevant information are filtered by the machine learning model.

[1243] 3. Emotion Recognition: The robot's built-in microphone and camera detect the user's voice and facial expressions, and analyzes their emotional state using TensorFlow. This emotional data is sent to the server when the user makes a request.

[1244] 4. Information provision: When a user requests specific information through the robot, the server analyzes the request based on the trained model and emotion recognition results, and adjusts the response according to the user's emotional state to provide appropriate information to the user.

[1245] Specific examples

[1246] If a user requests detailed instructions for the next step in a task, the server generates a prompt like this and provides it to OpenAI GPT-4:

[1247] Example prompt sentence:

[1248] "A worker has requested more details about process X. He seems a little nervous. What information should he provide first? Please keep it concise and focus on five points."

[1249] In response to this prompt, OpenAI GPT-4 provides a concise summary of five key points, allowing users to get the information they need quickly and accurately. Emotion recognition also allows for a more concise explanation if the user is stressed, or more detailed information if the user is emotionally stable.

[1250] In this way, the system of the present invention can improve the work efficiency of industrial robots and provide information flexibly according to the emotional state of the user.

[1251] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1252] Step 1:

[1253] The server collects business-related data from email servers, chat applications, document management systems, etc. Specifically, it uses Apache Kafka to stream this data in real time and store it in Elasticsearch. The input is data from each data source, and the output is the integrated data stored in Elasticsearch.

[1254] Step 2:

[1255] The server uses business-related data stored in Elasticsearch and trains the data using the OpenAI GPT-4 model. At this stage, the consolidated data is fed into a machine learning algorithm, which analyzes the data and learns about the business and project details. The input is the consolidated data from Elasticsearch, and the output is a trained GPT-4 model.

[1256] Step 3:

[1257] The robot's built-in microphones and cameras collect the user's voice and video and send them to a server. The server uses TensorFlow to analyze the voice and video data and identify the user's emotional state. The input is the voice and video data, and the output is a classification of the user's emotional state.

[1258] Step 4:

[1259] The user sends a question or request to the robot. For example, "Please tell me the detailed steps for the next work step." This request is sent to the server. The input is the user's request, and the output is its content.

[1260] Step 5:

[1261] The server analyzes the user's request and generates and supplies a prompt to the generative AI model (GPT-4). For example, the prompt might read, "A worker has requested details about process X. He seems a little nervous. What information should he provide first? Please answer briefly, focusing on five key points." The input is the user's request and emotional state, and the output is the prompt.

[1262] Step 6:

[1263] Based on the generated prompt, the OpenAI GPT-4 model generates appropriate information according to the request. The server analyzes this information and reconstructs it in a form that is easy for the user to understand. The input is the prompt, and the output is the generated information.

[1264] Step 7:

[1265] The server sends the generated information to the robot, which then provides it to the user. For example, five key points can be briefly explained. The input is the generated information, and the output is the specific information provided to the user.

[1266] This series of processes enables industrial robots to provide appropriate and efficient information taking into account the user's emotional state.

[1267] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1268] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1269] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1270] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1271] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1272] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1273] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1274] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1275] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1276] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1277] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1278] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1279] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1280] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1281] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1282] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1283] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1284] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1285] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1286] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1287] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1288] The following is further disclosed regarding the above embodiment.

[1289] (Claim 1)

[1290] a means of collecting business-related data;

[1291] A means of integrating and learning from collected business-related data;

[1292] A means for providing relevant information from the trained model in response to a user request;

[1293] A system including:

[1294] (Claim 2)

[1295] 2. The system of claim 1, wherein the business-related data includes email data, chat data, and document data.

[1296] (Claim 3)

[1297] 2. The system of claim 1, wherein the user request is for taking over a new project.

[1298] (Claim 4)

[1299] 10. The system of claim 1, further comprising means for training the business-related data using natural language processing techniques.

[1300] (Claim 5)

[1301] 10. The system of claim 1, further comprising means for periodically collecting business-related data.

[1302] "Example 1"

[1303] (Claim 1)

[1304] a means for retrieving the collected business-related data at regular intervals;

[1305] means for integrating the acquired business-related data into a database;

[1306] a means for training a natural language processing model using the integrated data; and

[1307] means for receiving a request in the form of a prompt sentence from a user;

[1308] A means for searching for information based on a user request using the trained model;

[1309] means for providing search results to a user terminal;

[1310] A system including:

[1311] (Claim 2)

[1312] 10. The system of claim 1, wherein the collected work-related data includes email data, chat data, and file storage data.

[1313] (Claim 3)

[1314] 2. The system according to claim 1, wherein the user request is related to a project handover.

[1315] "Application Example 1"

[1316] (Claim 1)

[1317] a means of collecting business-related data;

[1318] A means of integrating and learning from collected business-related data;

[1319] A means for providing relevant information from the trained model in response to a user request;

[1320] A means for collecting operation logs, sensor data, and quality inspection data of industrial equipment;

[1321] A means for feeding the collected industrial equipment data to a machine learning model for integration and learning;

[1322] a means for extracting and providing relevant information in response to a technician's query using a machine learning model;

[1323] A system including:

[1324] (Claim 2)

[1325] 2. The system of claim 1, wherein the business-related data includes email data, chat data, and document data.

[1326] (Claim 3)

[1327] 2. The system of claim 1, wherein the user request is for taking over a new project.

[1328] (Claim 4)

[1329] 10. The system of claim 1, wherein the industrial equipment data includes sensor data and quality inspection data.

[1330] "Example 2: Combining Emotion Engines"

[1331] (Claim 1)

[1332] a means of collecting business-related data;

[1333] A means of integrating and learning from collected business-related data;

[1334] A means for providing relevant information from the trained model in response to a user request;

[1335] means for recognizing the emotional state of a user and adjusting the provision of information in response to the emotional state;

[1336] A system including:

[1337] (Claim 2)

[1338] 2. The system of claim 1, wherein the business-related data includes email data, chat data, and document data.

[1339] (Claim 3)

[1340] 2. The system of claim 1, wherein the user request is for taking over a new project.

[1341] "Application example 2 when combining emotion engines"

[1342] (Claim 1)

[1343] a means of collecting business-related data;

[1344] A means of integrating and learning from collected business-related data;

[1345] A means for providing relevant information from the trained model in response to a user request;

[1346] a means of recognizing emotions and adjusting responses;

[1347] A means for efficiently taking over tasks in an industrial robot in response to a user request;

[1348] A system including:

[1349] (Claim 2)

[1350] 2. The system of claim 1, wherein the business-related data includes email data, chat data, and business document data.

[1351] (Claim 3)

[1352] 2. The system of claim 1, wherein the user request is for taking over a new task and the industrial robot adapts. [Explanation of symbols]

[1353] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. a means of collecting business-related data; A means of integrating and learning from collected business-related data; A means for providing relevant information from the trained model in response to a user request; A system including:

2. 2. The system of claim 1, wherein the business-related data includes email data, chat data, and document data.

3. 2. The system of claim 1, wherein the user request is for taking over a new project.

4. 10. The system of claim 1, further comprising means for training the business-related data using natural language processing techniques.

5. 10. The system of claim 1, further comprising means for periodically collecting business-related data.

Images