System

Abstract

A system is provided.SOLUTION: A system comprising: means for installing an agent in a computer of a user and collecting contents of daily tasks and meetings; means for analyzing the collected data and extracting main points; means for automatically generating a task manual, a task flow, a case collection, a glossary, and a task takeover document based on the extracted main points; and means for storing the generated materials in a company-wide portal site and providing a recommendation function to the user.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Modern companies are required to standardize and streamline their operations, but they face challenges such as the difficulty of new graduates, transferred employees, and employees on loan becoming immediately productive. The details of on-site work are often difficult to understand, and training manuals are often lacking, resulting in a waste of time and resources. Furthermore, the location of documents is unclear, and work processes are not fully visualized, making it difficult to transfer work and train new employees. To solve these issues, a system is needed that records, analyzes, and automates work, and provides appropriate content. [Means for solving the problem]

[0005] This system has a means for installing an agent on a user's computer to collect the contents of daily work and meetings, a means for analyzing the collected data and extracting key points, a means for automatically generating work manuals, work flows, case studies, glossaries, and work handover documents based on the extracted key points, and a means for storing the generated materials on a company-wide portal site and providing a recommendation function to users.By including a means for recommending optimal materials based on the user's work history and search behavior, and a means for converting collected voice data into text data and applying natural language processing algorithms to extract key points, the system enables standardization and efficiency of work, as well as rapid handover and training.

[0006] An "agent" is software that is installed on a user's computer and collects the contents of daily tasks and meetings.

[0007] "Data collection" is the process of using an agent to obtain business-related information, such as PC operations and audio during meetings.

[0008] "Data analysis" refers to the techniques and methods used to process collected data and extract necessary information and key points.

[0009] "Key point extraction" refers to identifying and extracting important information and keywords during the data analysis process.

[0010] A "business manual" is a document that details business procedures and processes.

[0011] A "business flow" is a diagram or written explanation that shows the order in which a particular business process proceeds.

[0012] A "case study" is a document that compiles past business operations and response examples and is used as a reference.

[0013] A "glossary" is a document that compiles definitions and explanations of technical terms and abbreviations used in business.

[0014] A "business handover document" is a document used to hand over the content and progress of a business to a successor when the person in charge of that business changes.

[0015] A "portal site" is a website accessible to the entire company that stores business-related materials and information.

[0016] The "recommendation function" is a function that suggests the most suitable materials and information to a specific user based on the user's profile and work history.

[0017] A "natural language processing algorithm" is a technology that enables computers to understand, interpret, and generate human language, and is used to analyze text data and extract key points. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9]1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0019] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0020] First, the terms used in the following description will be explained.

[0021] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0022] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0023] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0024] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0025] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0026] [First embodiment]

[0027] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0028] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0029] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0030] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0031] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0032] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0033] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0034] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0035] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0036] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0037] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0038] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0039] The present invention is a system for achieving standardization and efficiency in business operations. This system is mainly composed of the following means:

[0040] 1. Agent installation and data collection:

[0041] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. For example, the agent collects the content of a user's presentation about a new product.

[0042] 2. Data analysis and key points:

[0043] The server runs algorithms to analyze the collected data. The collected voice data is first converted into text data, and then natural language processing (NLP) algorithms are applied to extract key points. For example, key information such as "new product features" and "market strategy" can be identified from the presentation content.

[0044] 3. Automatic generation of materials:

[0045] Based on the key point extraction results, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents by embedding data into predefined templates. For example, a business manual describing the features of a new product can be automatically generated.

[0046] 4. Portal site management and recommendations:

[0047] The generated materials are stored on a company-wide portal site. The server recommends appropriate materials based on the user's profile, past work history, and search history. For example, when a new employee accesses the portal, they are recommended a new product manual or case study.

[0048] Program processing

[0049] Device:

[0050] The agent sends user operations and voice data to the server in real time. Users simply carry out their usual work or meetings without any special operations.

[0051] server:

[0052] The server first converts the audio data into text, then uses NLP to extract key points. Based on the analyzed key points, various documents are automatically generated using templates and uploaded to the portal site.

[0053] User:

[0054] Users can access the portal site and check the recommended materials to deepen their understanding of their work. For example, when assigned to a new job, they can instantly access the manuals and workflows they need.

[0055] Specific examples

[0056] When a user gives a presentation on a new product, the agent collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[0057] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload for users.

[0058] The processing flow will be explained below.

[0059] Step 1:

[0060] Installing and Configuring the Agent

[0061] server:

[0062] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[0063] Device:

[0064] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[0065] User:

[0066] The user confirms that the agent software is installed and configured.

[0067] Step 2:

[0068] Starting Data Collection

[0069] server:

[0070] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[0071] Device:

[0072] Based on specified conditions, the agent collects data on PC operations and audio data during meetings and sends it to the server in real time.

[0073] User:

[0074] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[0075] Step 3:

[0076] Transcription of audio data

[0077] server:

[0078] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[0079] Step 4:

[0080] Extracting key points using natural language processing (NLP)

[0081] server:

[0082] The server applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[0083] Step 5:

[0084] Analysis of operation log data

[0085] server:

[0086] The server analyzes the collected operation log data to identify frequently occurring operation patterns and business flows. This data will be used as the basis for later generation of documents.

[0087] Step 6:

[0088] Automatic generation of materials

[0089] server:

[0090] Based on the results of key point extraction and operation log analysis, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates. For example, information about the features of a new product can be automatically added to the business manual.

[0091] Step 7:

[0092] Preservation and management of generated materials

[0093] server:

[0094] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[0095] Step 8:

[0096] Uploading to the portal site and making recommendations

[0097] server:

[0098] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on the user profile and work history to provide the most suitable documents to specific users.

[0099] Device:

[0100] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[0101] User:

[0102] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[0103] By sequentially executing these steps, the present invention realizes automatic recording, analysis, automation, and efficiency of business operations.

[0104] Example 1

[0105] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0106] In conventional business processes, a great deal of effort was required to standardize and streamline operations. Manually recording meeting content and work procedures and then creating work manuals and workflows based on those records was particularly tedious work. New employees and transferees also faced the problem of being unable to quickly obtain the information they needed. A system that could automate and streamline a series of tasks was needed.

[0107] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0108] In this invention, the server includes means for installing agent software on a terminal and collecting the contents of daily work and meetings, means for periodically transmitting the collected operation logs and voice data to the server, means for converting the transmitted voice data into text data, means for extracting key points by applying a natural language processing algorithm, means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points, and means for storing the generated materials on a company-wide portal site and providing a recommendation function to users. This makes it possible to automate a series of processes from collecting work data to generating materials and making recommendations.

[0109] "Agent software" is software that is installed on the user's terminal and collects PC operations and conference audio in real time.

[0110] An "operation log" refers to the history of operations performed by a user on a PC, such as keyboard input, mouse clicks, and slide switching.

[0111] "Audio data" refers to data that records what a user says during a meeting or presentation.

[0112] A "server" is a computer system that receives data sent from a terminal, analyzes, converts, and processes the data to generate materials, and stores the materials on a portal site.

[0113] "Text data" refers to character information that has been converted by analyzing voice data.

[0114] "Natural language processing algorithm" is a general term for programs and methods for extracting key points from text data.

[0115] "Key points" refer to important information or keywords that are analyzed and extracted from the collected data.

[0116] A "business manual" refers to a document that describes business procedures and methods.

[0117] "Business flow" refers to a diagram of a series of business processes and procedures.

[0118] A "case study" refers to a document that compiles past business success stories and examples of implementation.

[0119] A "glossary" refers to a document that explains business-related terminology.

[0120] A "business handover document" refers to a document that contains the information necessary to ensure a smooth handover of business operations.

[0121] "Portal site" refers to a website where generated materials are stored and can be accessed and viewed by users.

[0122] The "recommendation function" refers to a function that suggests appropriate materials based on the user's work history and search behavior.

[0123] This invention is a system for standardizing and streamlining user operations. This system mainly includes the following components: agent software, a server, and a portal site. Each component will be described in detail below, along with its operation.

[0124] Agent software installation and data collection

[0125] Device:

[0126] First, agent software is installed on the user's device. This agent software collects in real time the user's PC operations (keyboard input, mouse clicks, switching slides, etc.) and the voices spoken during the meeting. For example, when a user gives a presentation about a new product, the agent software captures the operation log and voice data.

[0127] Data transmission and text conversion

[0128] Device:

[0129] The collected data is sent to the server at regular intervals (for example, every minute) or when an event occurs, allowing for real-time processing of the data.

[0130] server:

[0131] The server takes the received audio data and uses a speech recognition service such as the Google Cloud Speech-to-Text API to convert it into text data, which is then used in the next processing step.

[0132] Extracting key points using natural language processing

[0133] server:

[0134] The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract key information (key points). For example, keywords and phrases such as "new product features" and "market strategy" can be extracted from the text of a new product presentation.

[0135] Automatic generation of materials

[0136] server:

[0137] Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates. For example, a business manual describing the features and market strategy of a new product can be generated.

[0138] Management and recommendations on the portal site

[0139] server:

[0140] The generated materials are stored on a company-wide portal site. The server recommends the most appropriate materials based on the user's work history and search behavior. For example, when a new employee accesses the portal site, a new product manual and case studies are displayed on the top page.

[0141] User:

[0142] Users can deepen their understanding of the business by accessing the portal site and checking the recommended materials, which will enable them to respond quickly to new business tasks.

[0143] Specific examples

[0144] When a user gives a presentation about a new product, the agent software collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice data into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[0145] Example prompts for generative AI models

[0146] 1. "Please summarize the main points of this presentation. The key themes are the features of the new product and the market strategy."

[0147] 2. "Please generate an operations manual from the following text data. Use Microsoft Word as the template."

[0148] 3. "Please extract keywords from the audio data of this conference and upload related resources to the portal site."

[0149] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload of users.

[0150] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0151] Step 1:

[0152] Agent software installation and data collection

[0153] Terminal: Agent software is installed on the user's terminal. The agent software collects in real time the operations performed by the user on the PC (keyboard input, mouse clicks, switching slides, etc.) and the voice uttered during the meeting. The collected data is saved as an operation log and voice data.

[0154] Input: User's PC operation, conference audio

[0155] Output: Operation log, audio data

[0156] Specific operation: Once the agent software is installed and the user begins to operate the device, the operation log and recorded audio are captured in real time and temporarily stored on the device.

[0157] Step 2:

[0158] Data transmission and text conversion

[0159] Terminal: The collected operation log and voice data are sent to the server periodically (for example, every minute).

[0160] Input: Operation log, audio data

[0161] Output: Operation log and audio data sent to the server

[0162] Specific operation: The terminal uploads all the data collected by the agent software to the server at regular intervals or at specific events (e.g., the end of a meeting).

[0163] Server: The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[0164] Input: Audio data

[0165] Output: Text data

[0166] Specific operation: The server sequentially analyzes the received audio files, calls the speech recognition API to convert the audio into text, and passes the result to the next processing step.

[0167] Step 3:

[0168] Extracting key points using natural language processing

[0169] Server: The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract important information and key points.

[0170] Input: Text data

[0171] Output: Key information

[0172] Specific operations: Analyze text data using NLP tools to extract important keywords and phrases such as "new product features" and "market strategy."

[0173] Step 4:

[0174] Automatic generation of materials

[0175] Server: Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates.

[0176] Input: Key information

[0177] Output: Business manuals, business flow, case studies, glossary, business handover documents

[0178] Specific operation: The server embeds key information into a pre-prepared template and generates a document file in a format such as "New Product Manual.docx."

[0179] Step 5:

[0180] Management and recommendations on the portal site

[0181] Server: The generated documents are stored on the company's portal site. The server recommends appropriate documents based on the user's work history and search behavior.

[0182] Input: Generated materials, user work history, search behavior

[0183] Output: Recommended materials

[0184] Specific operation: Documents are uploaded to the portal site, and when a user accesses it, an algorithm is run based on the user's previous work history and search behavior, and documents appropriate for the user are displayed on the top page.

[0185] Step 6:

[0186] Check the materials

[0187] User: The user accesses the portal site and checks the recommended materials, which allows them to deepen their understanding of the business.

[0188] Input: Recommended materials

[0189] Output: Understanding business operations by referring to materials

[0190] Specific operation: A user logs in to the portal site, clicks on the link to recommended materials displayed on the top page, and views the manuals and flow charts.

[0191] (Application example 1)

[0192] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0193] At logistics centers, product handling and picking route procedures change frequently, making it time-consuming to create instructions and manuals to accommodate these changes. In addition, real-time, reliable information provision is required so that workers can immediately understand new instructions and procedures and work efficiently. Conventional methods lack the means to efficiently resolve these issues, making it difficult to streamline and standardize operations.

[0194] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0195] In this invention, the server includes: means for installing an agent on a user's computer and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating work manuals, work flows, case studies, glossaries, and work handover documents based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means including a smartphone agent application for collecting voice data and operation data in real time and sending it to the server; means for analyzing the collected data on the server using voice recognition and natural language processing algorithms and extracting important information; means for automatically generating efficient work instructions based on the extracted key points and notifying the smartphone in real time; and means for providing a portal site that recommends materials based on the user's work history and profile. This enables fast and accurate information provision and work standardization.

[0196] An "agent" is a program that is installed on a user's computer or smartphone and collects information about daily work and meetings.

[0197] "Audio data" refers to data that records the voice generated during a user's work or a meeting.

[0198] "Operation data" refers to data that records the operations performed by a user on a computer or smartphone.

[0199] A "server" is a device or system that has the function of analyzing collected data, extracting key points, and automatically generating and managing materials.

[0200] A "natural language processing algorithm" is an analytical method for extracting key points from text data such as voice data.

[0201] A "business manual" is a guide that describes the procedures and methods for a specific business operation.

[0202] A "business flow" is a diagram or text that shows the flow or process of a business.

[0203] A "case study collection" is a collection of specific past business cases that have been collected and compiled.

[0204] A "glossary" is a document that collects technical terms and their definitions related to a particular field or task.

[0205] A "business handover document" is a document that describes the handover details when a business person in charge is changed.

[0206] A "portal site" is a website that stores generated materials and provides users with a recommendation function.

[0207] The "agent application for smartphones" is software that is installed on a smartphone and collects voice data and operation data in real time and transmits them to a server.

[0208] "Real-time notification" is a function that instantly sends generated work instructions to a smartphone.

[0209] "Key point extraction" is the process of extracting important information or points from collected data.

[0210] The "recommendation function" is a function that recommends the most appropriate materials and information based on the user's work history and profile.

[0211] This invention provides a system that includes a smartphone application and back-end system used in logistics centers and other locations. Smartphones with installed agents collect the daily work details of workers and send them to a server, thereby generating and notifying work instructions in real time. Below, we will explain each component of the system and its processing.

[0212] Terminal part (smartphone)

[0213] Hardware / software used: Smartphone, Android OS, iOS, agent app

[0214] Voice data collection: The voice of the worker is collected in real time using the microphone on the smartphone. For example, when a worker explains how to handle a new product, the voice data is collected.

[0215] Operation data collection: Includes a screen capture function for recording screen operations. For example, when operating a screen to confirm product picking instructions, that operation data is collected.

[0216] Server part

[0217] Hardware / software used: Cloud servers (AWS, Google Cloud, etc.), speech recognition APIs (Google Speech-to-Text API, etc.), NLP libraries (SpaCy, NLTK)

[0218] Speech-to-text conversion: Converts collected voice data into text data. For example, a voice explaining a new product is converted into text data.

[0219] Key point extraction: Using NLP algorithms to extract important information from text data, such as the features and handling precautions of a new product.

[0220] Automatic generation of work instructions: Based on the extracted key points, work instructions are automatically generated using templates. For example, a handling procedure manual for a new product is generated.

[0221] Uploading materials: The generated materials are uploaded to a portal site and recommendations are made based on the user profile. For example, when a worker accesses the portal site, a handling manual for a new product is recommended.

[0222] Real-time notification section

[0223] Hardware / software used: Smartphone, push notification API (Firebase Cloud Messaging, etc.)

[0224] Notifications: Generated work instructions are sent via push notification in real time. For example, the handling procedure for a new product is immediately sent to the worker's smartphone.

[0225] Examples and prompts

[0226] Examples:

[0227] Scenario: A presentation is being given to explain how to handle a new product.

[0228] Data collection: The agent app collects the audio of the presentation.

[0229] Data analysis: The audio is converted into text and the key point, "How to handle the new product," is extracted.

[0230] Instruction generation: Generate handling instructions for new products based on the extracted key points.

[0231] Notification: The procedure manual is pushed to the picking worker's smartphone.

[0232] Example prompt for a generative AI model:

[0233] Analyze the audio recording about how to handle new products and extract the following information:

[0234] 1. New product handling procedures

[0235] 2. Materials and Equipment Needed

[0236] 3. Points to note

[0237] Based on the extracted information, please automatically generate a work instruction in the following format:

[0238] 1. Product name

[0239] 2. Handling Procedures

[0240] 3. Required Materials

[0241] 4. Points to note

[0242] Please notify the smartphone app of the generated work instructions in the following format:

[0243] 1. Title: New Product Handling Instructions

[0244] 2. Content: Extracted work instructions

[0245] This system, configured in this way, will enable the efficiency and standardization of operations at logistics centers, and by providing appropriate instructions to workers in real time, it is expected to reduce errors and improve overall work efficiency.

[0246] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0247] Step 1:

[0248] The agent app on the smartphone collects the worker's voice data and operation data in real time. The voice data is recorded through the smartphone's microphone, and the operation data is recorded using the screen capture function. The voice and operation data are collected as input and sent to the server for later analysis.

[0249] Step 2:

[0250] The collected voice data and operation data are sent to the server. The server receives this data and first converts the voice data into text using a voice recognition API (e.g., Google Speech-to-Text API). The input is the voice data, and the output is the converted text data. Specifically, voice data is given as input, and the voice recognition API outputs text data.

[0251] Step 3:

[0252] The server applies a natural language processing (NLP) algorithm (e.g., SpaCy, NLTK) to the converted text data to extract key points. The input is text data obtained by speech recognition, and the extracted results containing important information (key points) are output. Specifically, text data is given as input, and the NLP algorithm outputs key points such as "features of the new product" and "how to use it."

[0253] Step 4:

[0254] The server automatically generates work instructions based on the key point extraction results. The input is the extracted key point information, and a document containing the work instructions (e.g., handling procedures for a new product) is generated using a template. Specifically, the key point information is embedded in the template, and the generated work instructions are output.

[0255] Step 5:

[0256] The server uploads the generated work instructions to a portal site and recommends related materials based on the user's profile and work history. The input is the generated work instructions and user profile, and the portal site's recommendation information is output. Specifically, the work instructions are stored on the portal site, and appropriate materials are displayed according to the user.

[0257] Step 6:

[0258] The generated work instructions are sent in real time using a smartphone push notification API (e.g., Firebase Cloud Messaging). The input is the work instructions and information about the recipient smartphone, and the output is a notification message sent to the smartphone. Specifically, the work instructions are sent as a push notification message and are instantly displayed on the worker's smartphone.

[0259] Step 7:

[0260] The user (worker) checks the work instructions notified on their smartphone and carries out the work. The input is the work instructions displayed on the smartphone, and the output is the work procedure to be carried out based on them. Specifically, the user taps the notification to open the work instructions and carries out the work according to the instructed procedure.

[0261] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0262] This invention combines an emotion engine with a system that promotes the standardization and efficiency of users' work. This system is composed of the following means:

[0263] 1. Install and configure the agent:

[0264] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. When a user gives a presentation about a new product, the agent collects the content.

[0265] 2. Data collection and analysis:

[0266] The server analyzes the collected voice data and operation data. The voice data is converted into text data using a transcription tool, and key points are extracted using a natural language processing (NLP) algorithm. An emotion engine then recognizes the user's emotions from the voice and operation data. For example, not only is important information such as "new product features" and "market strategy" identified, but the emotions (e.g., excitement, stress) expressed when the user describes that information are also recognized.

[0267] 3. Automatic generation of materials:

[0268] The server automatically generates a business manual, business flow, case studies, glossary, and business handover document based on the extracted key information and emotion data. For example, a business manual may be generated that particularly emphasizes the features of a new product that the user excitedly explained.

[0269] 4. Portal site management and recommendations:

[0270] The generated materials are stored on a company-wide portal site. The server recommends the most suitable materials based on each user's profile, work history, search history, and even emotional data. For example, when a new employee accesses the portal site, they are presented with an interesting work manual recommended based on their emotional data.

[0271] Program processing

[0272] Device:

[0273] The agent collects user operations and voice data in real time and sends it to the server. It also acquires the user's emotional data using an emotion engine and sends this information to the server. The user simply needs to carry out their normal work or meetings.

[0274] server:

[0275] 1. Transcription of audio data:

[0276] The server converts the collected voice data into text data, using a transcription tool to accurately convert the voice data into text format.

[0277] 2. Key Points Extraction Using Natural Language Processing (NLP):

[0278] The server applies NLP algorithms to the text data to extract key points and keywords, such as "new product features" and "target market."

[0279] 3. Emotional Data Analysis:

[0280] The server analyzes the collected emotion data using the emotion engine to detect the user's emotional state (e.g., stress level and motivation) from the tone of voice and operation logs.

[0281] 4. Automatic generation of materials:

[0282] The server combines key information and emotion data to automatically generate business manuals, workflows, case studies, glossaries, and business handover documents. The generated materials reflect the user's emotions.

[0283] 5. Portal site management and recommendations:

[0284] The server uploads the generated documents to a company-wide portal site and organizes them by category. It also recommends the most suitable documents based on user profiles and emotional data. It also notifies users of the access information for the documents.

[0285] User:

[0286] Users receive notifications and access recommended materials from the portal site, which allows them to efficiently learn and deepen their understanding of their work.

[0287] Specific examples

[0288] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server then converts the voice data into text and uses NLP to extract key points. The emotion engine also analyzes whether the user is excited. Based on the extracted key points and emotional data, a business manual is automatically generated with sections that need to be emphasized highlighted. When new employees access the portal site, this business manual is recommended, and information is provided in a fun and easy-to-learn format.

[0289] In this way, this system supports the standardization and efficiency of business operations, and by utilizing user emotion data, it is possible to provide more appropriate and effective content.

[0290] The processing flow will be explained below.

[0291] Step 1:

[0292] Installing and Configuring the Agent

[0293] server:

[0294] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[0295] Device:

[0296] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[0297] User:

[0298] The user confirms that the agent software is installed and configured.

[0299] Step 2:

[0300] Starting Data Collection

[0301] server:

[0302] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[0303] Device:

[0304] The agent collects data on PC operations and voice data during meetings based on specified conditions and sends it to the server in real time. The emotion engine also analyzes the user's voice tone and operation patterns.

[0305] User:

[0306] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[0307] Step 3:

[0308] Transcription of audio data

[0309] server:

[0310] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[0311] Step 4:

[0312] Extracting key points using natural language processing (NLP)

[0313] server:

[0314] The server then applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[0315] Step 5:

[0316] Emotional Data Analysis

[0317] server:

[0318] The server analyzes the voice tones and operation patterns collected by the emotion engine to determine the user's emotional state. For example, it detects whether the user is excited when explaining a new product. The emotional data is stored together with other business data.

[0319] Step 6:

[0320] Automatic generation of materials

[0321] server:

[0322] The server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates, using the key point extraction results and emotion data. Materials are created in a way that emphasizes parts that strongly reflect the user's emotions (for example, the features of a new product that was explained in an excited state).

[0323] Step 7:

[0324] Preservation and management of generated materials

[0325] server:

[0326] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[0327] Step 8:

[0328] Uploading to the portal site and making recommendations

[0329] server:

[0330] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on user profiles and emotional data to provide the most suitable documents to specific users.

[0331] Device:

[0332] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[0333] User:

[0334] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[0335] By executing each step in this way, the present invention realizes automatic recording, analysis, document generation, and efficiency improvement of business operations, and further utilizes emotion data to provide more appropriate and effective content.

[0336] Example 2

[0337] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0338] Conventional business management systems lack the functionality to efficiently collect users' daily work and meeting details, extract key points, and automatically generate materials based on them. Furthermore, materials are not created taking into account the emotional state of the user, making it difficult to achieve standardized and efficient business processes. Furthermore, they lack the functionality to optimally recommend generated materials, which makes it difficult to provide appropriate information to specific users.

[0339] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for installing an agent in a user's information device and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means for converting collected voice data into text data using a transcription tool; means for recognizing and analyzing the user's emotional state from the voice data and operation data using an emotion engine; and means for automatically generating materials by combining key point information and emotion data. This enables standardization and efficiency of work and the provision of information that takes emotion data into consideration.

[0340] An "agent" is software that is installed on a user's information device and collects the contents of daily tasks and meetings.

[0341] "Information devices" is a general term for electronic devices such as computers, tablets, and smartphones that users use for work and meetings.

[0342] An "emotion engine" is software or algorithms that recognize and analyze a user's emotional state from voice data and operation data.

[0343] "Transcription tool" is a general term for software and services that convert audio data into text data.

[0344] "Natural language processing algorithm" is a general term for algorithms used to extract important points and keywords from text data.

[0345] A "company-wide portal site" is a web-based platform that users can access for sharing and managing information within a company.

[0346] The "recommendation function" is a function that suggests the most suitable materials and information based on the user's profile and behavioral data.

[0347] A "business manual" is a document that contains procedures and guidelines for carrying out a specific business or process.

[0348] A "business flow" is a diagram or table that visually represents the steps and flow of a specific business or process.

[0349] A "case study collection" is a document or data that collects and organizes past cases and case studies.

[0350] A "glossary" is a list of technical terms and definitions related to a particular field or task.

[0351] A "business handover document" is a document that lists the necessary information and procedures to be used when handing over business operations.

[0352] The present invention combines an emotion engine with a system that promotes the standardization and efficiency of user work. Specific embodiments for implementing this system are described below.

[0353] This system is mainly composed of three entities: a server, a terminal, and a user.

[0354] Device:

[0355] 1. Install and configure the agent:

[0356] The server installs agent software on the user's information device. This agent collects in real time the operations performed by the user on a PC or tablet, as well as audio data during a meeting. For example, when a user gives a presentation about a new product, the agent collects detailed information about the presentation.

[0357] 2. Data collection and transmission:

[0358] The terminal agent collects user operation data and conference audio data and sends them to the server. The emotion engine also collects the user's emotion data (e.g., excitement or stress) and sends this data to the server. The user simply carries out their normal work, and data collection is carried out in the background.

[0359] server:

[0360] 1. Transcription of audio data:

[0361] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically analyzed and the dialogue is recorded in text format. For example, a log stating "Voice data text conversion completed" is generated.

[0362] 2. Key Points Extraction Using Natural Language Processing (NLP):

[0363] The server applies an NLP algorithm (e.g., spaCy) to the text data to extract key points and keywords. For example, keywords such as "new product features" and "market strategy" are identified. At this step, the "extracted key points data" are passed on to the next step of processing.

[0364] 3. Emotional Data Analysis:

[0365] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the collected emotion data. It detects the user's emotional state (e.g., excitement, stress, etc.) from the tone of the voice and operation logs. For example, if a user becomes excited during a presentation, this state is recorded as "user is excited."

[0366] 4. Automatic generation of materials:

[0367] The server combines key information and emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that highlights the features of a new product that a user was particularly excited about. Materials are generated using a template engine (e.g., Jinja2).

[0368] 5. Portal site management and recommendations:

[0369] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, they receive a notification that a "business manual highlighting the features of a new product" is displayed.

[0370] User:

[0371] Users can receive notifications and access recommended materials from the portal site, allowing them to efficiently learn about new job content and deepen their understanding.

[0372] Specific examples

[0373] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server converts the collected voice data into text and extracts the main points using an NLP algorithm. The emotion engine also analyzes whether the user is excited.

[0374] For example, if the system detects that the user is excited when explaining the features of a new product, the system will automatically generate an operational manual that emphasizes those features.When a new employee accesses the portal site, the system will present them with an operational manual that is fun and easy to learn, recommended based on emotional data.

[0375] Prompt Sentence Examples

[0376] Prompt statement:

[0377] Collect voice and operation data from users giving a presentation on a new product, extract key points using an NLP algorithm, and analyze the user's level of excitement using an emotion engine. Then, automatically generate an operations manual that highlights the parts that caused the user to feel particularly excited.

[0378] This prompt sentence is used to instruct the generative AI model, ensuring efficient operation of the system.

[0379] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0380] Step 1:

[0381] Installing and Configuring the Agent

[0382] The server installs agent software on the user's device. This agent collects in real time the operations and voice data performed on the device, such as a PC or tablet. For example, when a user gives a presentation about a new product, the agent is ready to collect the content of the presentation. The input is the agent software, and the output is a notification that the installation is complete. Specifically, the agent is installed and initial settings are performed.

[0383] Step 2:

[0384] Data collection and transmission

[0385] The agent on the terminal collects user operation data and voice data during the conference in real time and sends this data to the server. For example, when a user starts a presentation, the content of the presentation is collected. Emotional data (excitement and stress) is also collected at the same time. The input is voice data and operation data, and the output is the data sent to the server. Specifically, the agent records the voice during the conference and keeps an operation log.

[0386] Step 3:

[0387] Transcription of audio data

[0388] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically converted into text format. The input is the voice data sent to the server, and the output is the converted text data. Specifically, the transcription tool analyzes the voice data and outputs the dialogue as text.

[0389] Step 4:

[0390] Extracting key points using natural language processing (NLP)

[0391] The server analyzes the text data converted by the transcription tool using an NLP algorithm (e.g., spaCy) to extract important key points and keywords. For example, key points such as "new product features" and "market strategy" are identified. The converted text data is the input, and the extracted key point data is the output. Specifically, the NLP algorithm automatically analyzes and extracts keywords and key points within the text.

[0392] Step 5:

[0393] Emotional Data Analysis

[0394] The server analyzes the collected emotion data using an emotion engine (e.g., IBM Watson Tone Analyzer) and recognizes the user's emotional state (e.g., excitement, stress, etc.). The input is emotion data including voice tone and operation logs, and the output is analyzed emotional state information. Specifically, the emotion engine analyzes the voice tone and detects the user's emotional state.

[0395] Step 6:

[0396] Automatic generation of materials

[0397] The server combines key information extracted by NLP with emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that emphasizes the features of a new product that a user described with particular enthusiasm. The input is the extracted key information and analyzed emotion data, and the output is automatically generated materials. Specifically, various materials are generated using a template engine (for example, Jinja2).

[0398] Step 7:

[0399] Management and recommendations on the portal site

[0400] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, a business manual with the characteristics that the employee described with excitement is recommended. The input is the generated materials and the user's profile information, and the output is the recommended material information. Specifically, the server uploads the materials and executes the recommendation function.

[0401] (Application example 2)

[0402] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0403] Lack of work efficiency and standardization in traditional logistics centers is a problem. In particular, it takes time to train new workers and understand their work, and it is difficult to respond immediately to their emotional state. This invention is designed to solve these problems and improve work efficiency and the training process.

[0404] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0405] In this invention, the server includes means for installing an agent in a user's information processing device and collecting the contents of daily work and meetings, means for analyzing the collected data and extracting key points, means for automatically generating work guides, work processes, case studies, glossaries, and work handover documents based on the extracted key points, means for storing the generated materials on an information sharing site and providing users with a recommendation function, means for collecting and analyzing worker emotion data in real time, and means for recommending optimal work guides and training materials based on the collected emotion data, thereby making it possible to improve work efficiency and speed up the training process.

[0406] An "agent" is software that is installed on a user's information processing device and collects the contents of daily tasks and meetings.

[0407] An "information processing device" is an electronic device for collecting, analyzing, and communicating data.

[0408] "Daily work and meeting contents" is information about the work and meeting contents that the user performs on a daily basis.

[0409] "Analyzing data" refers to processing collected information to find useful patterns and takeaways.

[0410] "Gist extraction" refers to extracting important information or key content from the analyzed data.

[0411] A "work guide" is a procedure manual for performing specific tasks efficiently and accurately.

[0412] A "business process" is a series of work steps to achieve a specific goal.

[0413] A "case study collection" is a document that compiles past examples and case studies.

[0414] A "glossary" is a document that lists definitions of technical terms and terms.

[0415] A "business handover document" is a document that contains the information necessary when a business person in charge changes.

[0416] An "information sharing site" is a web platform that allows generated materials and information to be stored and shared.

[0417] The "recommended function" is a function that provides the most suitable materials and information to the user.

[0418] "Emotion data" is data that represents the emotional state of the user.

[0419] "Real-time collection" refers to obtaining data immediately.

[0420] "Analysis" refers to examining collected data in detail to find meaning and patterns.

[0421] "Recommending optimal work guides and educational materials according to the user's emotional state" refers to providing the most appropriate work instructions and educational materials in consideration of the user's emotional state.

[0422] This invention is a work support system using smart glasses that aims to improve work efficiency and standardize work in logistics centers.

[0423] Hardware and Software Configuration

[0424] Device:

[0425] The smart glasses have a built-in microphone to capture the worker's voice, allowing them to collect voice input in real time, and a built-in display that can display information on the screen.

[0426] server:

[0427] The server uses the following software to analyze the collected voice data:

[0428] 1. Speech Recognition: A speech recognition library for converting collected voice data into text.

[0429] 2. Transformers (Hugging Face's NLP library): Analyzes text data after speech recognition using natural language processing (NLP) to extract key points.

[0430] 3. Emotion Recognition: An emotion recognition library for recognizing the user's emotional state from collected voice data.

[0431] Specific processing flow:

[0432] 1. Collecting voice data: The microphone built into the smart glasses captures the voice of the worker and sends the data to the server.

[0433] 2. Transcription of voice data: The server converts the collected voice data into text using the Speech Recognition library.

[0434] 3. Gist Extraction: Use the Transformers NLP library on the text data to extract key points.

[0435] 4. Emotion Data Analysis: Use the Emotion Recognition library to analyze the worker's emotional state from the tone and content of their voice.

[0436] 5. Automatic generation of work guides and educational materials: Automatically generate appropriate work guides and educational materials based on key information and emotion data.

[0437] 6. Information display: The generated materials and recommended information are displayed in real time on the smart glasses display.

[0438] User:

[0439] By wearing the smart glasses, workers can receive feedback and guidance information tailored to their own working conditions, which is expected to improve work efficiency and enhance proficiency.

[0440] Specific examples

[0441] For example, if a worker picking a new product says, "I don't know how to pick this product," the voice is captured by the microphone in the smart glasses. The voice data is sent to the server, converted into text, and analyzed, and the optimal picking method and handling instructions for that product are displayed in real time on the worker's smart glasses.

[0442] Example prompts to input to the generative AI model:

[0443] "Please tell me how to analyze voice data and recognize emotions to provide appropriate feedback to workers who are feeling stressed while working, and provide work guidance in real time."

[0444] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0445] Step 1:

[0446] The microphone in the smart glasses captures the worker's voice.

[0447] Input: Worker's voice

[0448] How it works: A microphone built into the smart glasses captures the worker's voice and transmits the voice data to a server in real time.

[0449] Output: Audio data

[0450] Step 2:

[0451] The server converts the received voice data into text using the Speech Recognition library.

[0452] Input: Audio data

[0453] Operation: The server inputs the voice data into the Speech Recognition library and converts it into text data.

[0454] Output: Text data

[0455] Step 3:

[0456] The server uses the Transformers NLP library to parse the text data and extract key points.

[0457] Input: Text data

[0458] How it works: The server inputs text data into an NLP library and applies natural language processing algorithms to extract important key information and concepts.

[0459] Output: Key information

[0460] Step 4:

[0461] The server uses the Emotion Recognition library to recognize emotional states from text data.

[0462] Input: Text data

[0463] How it works: The server inputs the collected text data into the Emotion Recognition library, which analyzes the user's emotional state from the tone and content of the voice.

[0464] Output: Emotion data

[0465] Step 5:

[0466] The server automatically generates work guides and educational materials based on the key information and emotion data.

[0467] Input: Key points, emotion data

[0468] Operation: The server inputs key information and emotion data into the DocumentGenerator module, and automatically generates optimal work guides and educational materials based on that information.

[0469] Output: Work guides, educational materials

[0470] Step 6:

[0471] The generated materials and recommended information are displayed on the smart glasses display.

[0472] Input: Work guides, educational materials

[0473] How it works: The server sends the generated materials and recommended information to the smart glasses, which then display it on their screens. Workers can then perform their work while viewing the information displayed in real time.

[0474] Output: Information displayed on the smart glasses display

[0475] keyword:

[0476] Generative AI model, prompt sentence

[0477] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0478] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0479] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0480] [Second embodiment]

[0481] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0482] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0483] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0484] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0485] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0486] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0487] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0488] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0489] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0490] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0491] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0492] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0493] The present invention is a system for achieving standardization and efficiency in business operations. This system is mainly composed of the following means:

[0494] 1. Agent installation and data collection:

[0495] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. For example, the agent collects the content of a user's presentation about a new product.

[0496] 2. Data analysis and key points:

[0497] The server runs algorithms to analyze the collected data. The collected voice data is first converted into text data, and then natural language processing (NLP) algorithms are applied to extract key points. For example, key information such as "new product features" and "market strategy" can be identified from the presentation content.

[0498] 3. Automatic generation of materials:

[0499] Based on the key point extraction results, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents by embedding data into predefined templates. For example, a business manual describing the features of a new product can be automatically generated.

[0500] 4. Portal site management and recommendations:

[0501] The generated materials are stored on a company-wide portal site. The server recommends appropriate materials based on the user's profile, past work history, and search history. For example, when a new employee accesses the portal, they are recommended a new product manual or case study.

[0502] Program processing

[0503] Device:

[0504] The agent sends user operations and voice data to the server in real time. Users simply carry out their usual work or meetings without any special operations.

[0505] server:

[0506] The server first converts the audio data into text, then uses NLP to extract key points. Based on the analyzed key points, various documents are automatically generated using templates and uploaded to the portal site.

[0507] User:

[0508] Users can access the portal site and check the recommended materials to deepen their understanding of their work. For example, when assigned to a new job, they can instantly access the manuals and workflows they need.

[0509] Specific examples

[0510] When a user gives a presentation on a new product, the agent collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[0511] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload for users.

[0512] The processing flow will be explained below.

[0513] Step 1:

[0514] Installing and Configuring the Agent

[0515] server:

[0516] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[0517] Device:

[0518] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[0519] User:

[0520] The user confirms that the agent software is installed and configured.

[0521] Step 2:

[0522] Starting Data Collection

[0523] server:

[0524] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[0525] Device:

[0526] Based on specified conditions, the agent collects data on PC operations and audio data during meetings and sends it to the server in real time.

[0527] User:

[0528] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[0529] Step 3:

[0530] Transcription of audio data

[0531] server:

[0532] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[0533] Step 4:

[0534] Extracting key points using natural language processing (NLP)

[0535] server:

[0536] The server applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[0537] Step 5:

[0538] Analysis of operation log data

[0539] server:

[0540] The server analyzes the collected operation log data to identify frequently occurring operation patterns and business flows. This data will be used as the basis for later generation of documents.

[0541] Step 6:

[0542] Automatic generation of materials

[0543] server:

[0544] Based on the results of key point extraction and operation log analysis, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates. For example, information about the features of a new product can be automatically added to the business manual.

[0545] Step 7:

[0546] Preservation and management of generated materials

[0547] server:

[0548] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[0549] Step 8:

[0550] Uploading to the portal site and making recommendations

[0551] server:

[0552] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on the user profile and work history to provide the most suitable documents to specific users.

[0553] Device:

[0554] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[0555] User:

[0556] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[0557] By sequentially executing these steps, the present invention realizes automatic recording, analysis, automation, and efficiency of business operations.

[0558] Example 1

[0559] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0560] In conventional business processes, a great deal of effort was required to standardize and streamline operations. Manually recording meeting content and work procedures and then creating work manuals and workflows based on those records was particularly tedious work. New employees and transferees also faced the problem of being unable to quickly obtain the information they needed. A system that could automate and streamline a series of tasks was needed.

[0561] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0562] In this invention, the server includes means for installing agent software on a terminal and collecting the contents of daily work and meetings, means for periodically transmitting the collected operation logs and voice data to the server, means for converting the transmitted voice data into text data, means for extracting key points by applying a natural language processing algorithm, means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points, and means for storing the generated materials on a company-wide portal site and providing a recommendation function to users. This makes it possible to automate a series of processes from collecting work data to generating materials and making recommendations.

[0563] "Agent software" is software that is installed on the user's terminal and collects PC operations and conference audio in real time.

[0564] An "operation log" refers to the history of operations performed by a user on a PC, such as keyboard input, mouse clicks, and slide switching.

[0565] "Audio data" refers to data that records what a user says during a meeting or presentation.

[0566] A "server" is a computer system that receives data sent from a terminal, analyzes, converts, and processes the data to generate materials, and stores the materials on a portal site.

[0567] "Text data" refers to character information that has been converted by analyzing voice data.

[0568] "Natural language processing algorithm" is a general term for programs and methods for extracting key points from text data.

[0569] "Key points" refer to important information or keywords that are analyzed and extracted from the collected data.

[0570] A "business manual" refers to a document that describes business procedures and methods.

[0571] "Business flow" refers to a diagram of a series of business processes and procedures.

[0572] A "case study" refers to a document that compiles past business success stories and examples of implementation.

[0573] A "glossary" refers to a document that explains business-related terminology.

[0574] A "business handover document" refers to a document that contains the information necessary to ensure a smooth handover of business operations.

[0575] "Portal site" refers to a website where generated materials are stored and can be accessed and viewed by users.

[0576] The "recommendation function" refers to a function that suggests appropriate materials based on the user's work history and search behavior.

[0577] This invention is a system for standardizing and streamlining user operations. This system mainly includes the following components: agent software, a server, and a portal site. Each component will be described in detail below, along with its operation.

[0578] Agent software installation and data collection

[0579] Device:

[0580] First, agent software is installed on the user's device. This agent software collects in real time the user's PC operations (keyboard input, mouse clicks, switching slides, etc.) and the voices spoken during the meeting. For example, when a user gives a presentation about a new product, the agent software captures the operation log and voice data.

[0581] Data transmission and text conversion

[0582] Device:

[0583] The collected data is sent to the server at regular intervals (for example, every minute) or when an event occurs, allowing for real-time processing of the data.

[0584] server:

[0585] The server takes the received audio data and uses a speech recognition service such as the Google Cloud Speech-to-Text API to convert it into text data, which is then used in the next processing step.

[0586] Extracting key points using natural language processing

[0587] server:

[0588] The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract key information (key points). For example, keywords and phrases such as "new product features" and "market strategy" can be extracted from the text of a new product presentation.

[0589] Automatic generation of materials

[0590] server:

[0591] Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates. For example, a business manual describing the features and market strategy of a new product can be generated.

[0592] Management and recommendations on the portal site

[0593] server:

[0594] The generated materials are stored on a company-wide portal site. The server recommends the most appropriate materials based on the user's work history and search behavior. For example, when a new employee accesses the portal site, a new product manual and case studies are displayed on the top page.

[0595] User:

[0596] Users can deepen their understanding of the business by accessing the portal site and checking the recommended materials, which will enable them to respond quickly to new business tasks.

[0597] Specific examples

[0598] When a user gives a presentation about a new product, the agent software collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice data into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[0599] Example prompts for generative AI models

[0600] 1. "Please summarize the main points of this presentation. The key themes are the features of the new product and the market strategy."

[0601] 2. "Please generate an operations manual from the following text data. Use Microsoft Word as the template."

[0602] 3. "Please extract keywords from the audio data of this conference and upload related resources to the portal site."

[0603] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload of users.

[0604] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0605] Step 1:

[0606] Agent software installation and data collection

[0607] Terminal: Agent software is installed on the user's terminal. The agent software collects in real time the operations performed by the user on the PC (keyboard input, mouse clicks, switching slides, etc.) and the voice uttered during the meeting. The collected data is saved as an operation log and voice data.

[0608] Input: User's PC operation, conference audio

[0609] Output: Operation log, audio data

[0610] Specific operation: Once the agent software is installed and the user begins to operate the device, the operation log and recorded audio are captured in real time and temporarily stored on the device.

[0611] Step 2:

[0612] Data transmission and text conversion

[0613] Terminal: The collected operation log and voice data are sent to the server periodically (for example, every minute).

[0614] Input: Operation log, audio data

[0615] Output: Operation log and audio data sent to the server

[0616] Specific operation: The terminal uploads all the data collected by the agent software to the server at regular intervals or at specific events (e.g., the end of a meeting).

[0617] Server: The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[0618] Input: Audio data

[0619] Output: Text data

[0620] Specific operation: The server sequentially analyzes the received audio files, calls the speech recognition API to convert the audio into text, and passes the result to the next processing step.

[0621] Step 3:

[0622] Extracting key points using natural language processing

[0623] Server: The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract important information and key points.

[0624] Input: Text data

[0625] Output: Key information

[0626] Specific operations: Analyze text data using NLP tools to extract important keywords and phrases such as "new product features" and "market strategy."

[0627] Step 4:

[0628] Automatic generation of materials

[0629] Server: Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates.

[0630] Input: Key information

[0631] Output: Business manuals, business flow, case studies, glossary, business handover documents

[0632] Specific operation: The server embeds key information into a pre-prepared template and generates a document file in a format such as "New Product Manual.docx."

[0633] Step 5:

[0634] Management and recommendations on the portal site

[0635] Server: The generated documents are stored on the company's portal site. The server recommends appropriate documents based on the user's work history and search behavior.

[0636] Input: Generated materials, user work history, search behavior

[0637] Output: Recommended materials

[0638] Specific operation: Documents are uploaded to the portal site, and when a user accesses it, an algorithm is run based on the user's previous work history and search behavior, and documents appropriate for the user are displayed on the top page.

[0639] Step 6:

[0640] Check the materials

[0641] User: The user accesses the portal site and checks the recommended materials, which allows them to deepen their understanding of the business.

[0642] Input: Recommended materials

[0643] Output: Understanding business operations by referring to materials

[0644] Specific operation: A user logs in to the portal site, clicks on the link to recommended materials displayed on the top page, and views the manuals and flow charts.

[0645] (Application example 1)

[0646] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0647] At logistics centers, product handling and picking route procedures change frequently, making it time-consuming to create instructions and manuals to accommodate these changes. In addition, real-time, reliable information provision is required so that workers can immediately understand new instructions and procedures and work efficiently. Conventional methods lack the means to efficiently resolve these issues, making it difficult to streamline and standardize operations.

[0648] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0649] In this invention, the server includes: means for installing an agent on a user's computer and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating work manuals, work flows, case studies, glossaries, and work handover documents based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means including a smartphone agent application for collecting voice data and operation data in real time and sending it to the server; means for analyzing the collected data on the server using voice recognition and natural language processing algorithms and extracting important information; means for automatically generating efficient work instructions based on the extracted key points and notifying the smartphone in real time; and means for providing a portal site that recommends materials based on the user's work history and profile. This enables fast and accurate information provision and work standardization.

[0650] An "agent" is a program that is installed on a user's computer or smartphone and collects information about daily work and meetings.

[0651] "Audio data" refers to data that records the voice generated during a user's work or a meeting.

[0652] "Operation data" refers to data that records the operations performed by a user on a computer or smartphone.

[0653] A "server" is a device or system that has the function of analyzing collected data, extracting key points, and automatically generating and managing materials.

[0654] A "natural language processing algorithm" is an analytical method for extracting key points from text data such as voice data.

[0655] A "business manual" is a guide that describes the procedures and methods for a specific business operation.

[0656] A "business flow" is a diagram or text that shows the flow or process of a business.

[0657] A "case study collection" is a collection of specific past business cases that have been collected and compiled.

[0658] A "glossary" is a document that collects technical terms and their definitions related to a particular field or task.

[0659] A "business handover document" is a document that describes the handover details when a business person in charge is changed.

[0660] A "portal site" is a website that stores generated materials and provides users with a recommendation function.

[0661] The "agent application for smartphones" is software that is installed on a smartphone and collects voice data and operation data in real time and transmits them to a server.

[0662] "Real-time notification" is a function that instantly sends generated work instructions to a smartphone.

[0663] "Key point extraction" is the process of extracting important information or points from collected data.

[0664] The "recommendation function" is a function that recommends the most appropriate materials and information based on the user's work history and profile.

[0665] This invention provides a system that includes a smartphone application and back-end system used in logistics centers and other locations. Smartphones with installed agents collect the daily work details of workers and send them to a server, thereby generating and notifying work instructions in real time. Below, we will explain each component of the system and its processing.

[0666] Terminal part (smartphone)

[0667] Hardware / software used: Smartphone, Android OS, iOS, agent app

[0668] Voice data collection: The voice of the worker is collected in real time using the microphone on the smartphone. For example, when a worker explains how to handle a new product, the voice data is collected.

[0669] Operation data collection: Includes a screen capture function for recording screen operations. For example, when operating a screen to confirm product picking instructions, that operation data is collected.

[0670] Server part

[0671] Hardware / software used: Cloud servers (AWS, Google Cloud, etc.), speech recognition APIs (Google Speech-to-Text API, etc.), NLP libraries (SpaCy, NLTK)

[0672] Speech-to-text conversion: Converts collected voice data into text data. For example, a voice explaining a new product is converted into text data.

[0673] Key point extraction: Using NLP algorithms to extract important information from text data, such as the features and handling precautions of a new product.

[0674] Automatic generation of work instructions: Based on the extracted key points, work instructions are automatically generated using templates. For example, a handling procedure manual for a new product is generated.

[0675] Uploading materials: The generated materials are uploaded to a portal site and recommendations are made based on the user profile. For example, when a worker accesses the portal site, a handling manual for a new product is recommended.

[0676] Real-time notification section

[0677] Hardware / software used: Smartphone, push notification API (Firebase Cloud Messaging, etc.)

[0678] Notifications: Generated work instructions are sent via push notification in real time. For example, the handling procedure for a new product is immediately sent to the worker's smartphone.

[0679] Examples and prompts

[0680] Examples:

[0681] Scenario: A presentation is being given to explain how to handle a new product.

[0682] Data collection: The agent app collects the audio of the presentation.

[0683] Data analysis: The audio is converted into text and the key point, "How to handle the new product," is extracted.

[0684] Instruction generation: Generate handling instructions for new products based on the extracted key points.

[0685] Notification: The procedure manual is pushed to the picking worker's smartphone.

[0686] Example prompt for a generative AI model:

[0687] Analyze the audio recording about how to handle new products and extract the following information:

[0688] 1. New product handling procedures

[0689] 2. Materials and Equipment Needed

[0690] 3. Points to note

[0691] Based on the extracted information, please automatically generate a work instruction in the following format:

[0692] 1. Product name

[0693] 2. Handling Procedures

[0694] 3. Required Materials

[0695] 4. Points to note

[0696] Please notify the smartphone app of the generated work instructions in the following format:

[0697] 1. Title: New Product Handling Instructions

[0698] 2. Content: Extracted work instructions

[0699] This system, configured in this way, will enable the efficiency and standardization of operations at logistics centers, and by providing appropriate instructions to workers in real time, it is expected to reduce errors and improve overall work efficiency.

[0700] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0701] Step 1:

[0702] The agent app on the smartphone collects the worker's voice data and operation data in real time. The voice data is recorded through the smartphone's microphone, and the operation data is recorded using the screen capture function. The voice and operation data are collected as input and sent to the server for later analysis.

[0703] Step 2:

[0704] The collected voice data and operation data are sent to the server. The server receives this data and first converts the voice data into text using a voice recognition API (e.g., Google Speech-to-Text API). The input is the voice data, and the output is the converted text data. Specifically, voice data is given as input, and the voice recognition API outputs text data.

[0705] Step 3:

[0706] The server applies a natural language processing (NLP) algorithm (e.g., SpaCy, NLTK) to the converted text data to extract key points. The input is text data obtained by speech recognition, and the extracted results containing important information (key points) are output. Specifically, text data is given as input, and the NLP algorithm outputs key points such as "features of the new product" and "how to use it."

[0707] Step 4:

[0708] The server automatically generates work instructions based on the key point extraction results. The input is the extracted key point information, and a document containing the work instructions (e.g., handling procedures for a new product) is generated using a template. Specifically, the key point information is embedded in the template, and the generated work instructions are output.

[0709] Step 5:

[0710] The server uploads the generated work instructions to a portal site and recommends related materials based on the user's profile and work history. The input is the generated work instructions and user profile, and the portal site's recommendation information is output. Specifically, the work instructions are stored on the portal site, and appropriate materials are displayed according to the user.

[0711] Step 6:

[0712] The generated work instructions are sent in real time using a smartphone push notification API (e.g., Firebase Cloud Messaging). The input is the work instructions and information about the recipient smartphone, and the output is a notification message sent to the smartphone. Specifically, the work instructions are sent as a push notification message and are instantly displayed on the worker's smartphone.

[0713] Step 7:

[0714] The user (worker) checks the work instructions notified on their smartphone and carries out the work. The input is the work instructions displayed on the smartphone, and the output is the work procedure to be carried out based on them. Specifically, the user taps the notification to open the work instructions and carries out the work according to the instructed procedure.

[0715] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0716] This invention combines an emotion engine with a system that promotes the standardization and efficiency of users' work. This system is composed of the following means:

[0717] 1. Install and configure the agent:

[0718] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. When a user gives a presentation about a new product, the agent collects the content.

[0719] 2. Data collection and analysis:

[0720] The server analyzes the collected voice data and operation data. The voice data is converted into text data using a transcription tool, and key points are extracted using a natural language processing (NLP) algorithm. An emotion engine then recognizes the user's emotions from the voice and operation data. For example, not only is important information such as "new product features" and "market strategy" identified, but the emotions (e.g., excitement, stress) expressed when the user describes that information are also recognized.

[0721] 3. Automatic generation of materials:

[0722] The server automatically generates a business manual, business flow, case studies, glossary, and business handover document based on the extracted key information and emotion data. For example, a business manual may be generated that particularly emphasizes the features of a new product that the user excitedly explained.

[0723] 4. Portal site management and recommendations:

[0724] The generated materials are stored on a company-wide portal site. The server recommends the most suitable materials based on each user's profile, work history, search history, and even emotional data. For example, when a new employee accesses the portal site, they are presented with an interesting work manual recommended based on their emotional data.

[0725] Program processing

[0726] Device:

[0727] The agent collects user operations and voice data in real time and sends it to the server. It also acquires the user's emotional data using an emotion engine and sends this information to the server. The user simply needs to carry out their normal work or meetings.

[0728] server:

[0729] 1. Transcription of audio data:

[0730] The server converts the collected voice data into text data, using a transcription tool to accurately convert the voice data into text format.

[0731] 2. Key Points Extraction Using Natural Language Processing (NLP):

[0732] The server applies NLP algorithms to the text data to extract key points and keywords, such as "new product features" and "target market."

[0733] 3. Emotional Data Analysis:

[0734] The server analyzes the collected emotion data using the emotion engine to detect the user's emotional state (e.g., stress level and motivation) from the tone of voice and operation logs.

[0735] 4. Automatic generation of materials:

[0736] The server combines key information and emotion data to automatically generate business manuals, workflows, case studies, glossaries, and business handover documents. The generated materials reflect the user's emotions.

[0737] 5. Portal site management and recommendations:

[0738] The server uploads the generated documents to a company-wide portal site and organizes them by category. It also recommends the most suitable documents based on user profiles and emotional data. It also notifies users of the access information for the documents.

[0739] User:

[0740] Users receive notifications and access recommended materials from the portal site, which allows them to efficiently learn and deepen their understanding of their work.

[0741] Specific examples

[0742] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server then converts the voice data into text and uses NLP to extract key points. The emotion engine also analyzes whether the user is excited. Based on the extracted key points and emotional data, a business manual is automatically generated with sections that need to be emphasized highlighted. When new employees access the portal site, this business manual is recommended, and information is provided in a fun and easy-to-learn format.

[0743] In this way, this system supports the standardization and efficiency of business operations, and by utilizing user emotion data, it is possible to provide more appropriate and effective content.

[0744] The processing flow will be explained below.

[0745] Step 1:

[0746] Installing and Configuring the Agent

[0747] server:

[0748] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[0749] Device:

[0750] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[0751] User:

[0752] The user confirms that the agent software is installed and configured.

[0753] Step 2:

[0754] Starting Data Collection

[0755] server:

[0756] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[0757] Device:

[0758] The agent collects data on PC operations and voice data during meetings based on specified conditions and sends it to the server in real time. The emotion engine also analyzes the user's voice tone and operation patterns.

[0759] User:

[0760] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[0761] Step 3:

[0762] Transcription of audio data

[0763] server:

[0764] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[0765] Step 4:

[0766] Extracting key points using natural language processing (NLP)

[0767] server:

[0768] The server then applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[0769] Step 5:

[0770] Emotional Data Analysis

[0771] server:

[0772] The server analyzes the voice tones and operation patterns collected by the emotion engine to determine the user's emotional state. For example, it detects whether the user is excited when explaining a new product. The emotional data is stored together with other business data.

[0773] Step 6:

[0774] Automatic generation of materials

[0775] server:

[0776] The server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates, using the key point extraction results and emotion data. Materials are created in a way that emphasizes parts that strongly reflect the user's emotions (for example, the features of a new product that was explained in an excited state).

[0777] Step 7:

[0778] Preservation and management of generated materials

[0779] server:

[0780] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[0781] Step 8:

[0782] Uploading to the portal site and making recommendations

[0783] server:

[0784] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on user profiles and emotional data to provide the most suitable documents to specific users.

[0785] Device:

[0786] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[0787] User:

[0788] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[0789] By executing each step in this way, the present invention realizes automatic recording, analysis, document generation, and efficiency improvement of business operations, and further utilizes emotion data to provide more appropriate and effective content.

[0790] Example 2

[0791] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0792] Conventional business management systems lack the functionality to efficiently collect users' daily work and meeting details, extract key points, and automatically generate materials based on them. Furthermore, materials are not created taking into account the emotional state of the user, making it difficult to achieve standardized and efficient business processes. Furthermore, they lack the functionality to optimally recommend generated materials, which makes it difficult to provide appropriate information to specific users.

[0793] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for installing an agent in a user's information device and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means for converting collected voice data into text data using a transcription tool; means for recognizing and analyzing the user's emotional state from the voice data and operation data using an emotion engine; and means for automatically generating materials by combining key point information and emotion data. This enables standardization and efficiency of work and the provision of information that takes emotion data into consideration.

[0794] An "agent" is software that is installed on a user's information device and collects the contents of daily tasks and meetings.

[0795] "Information devices" is a general term for electronic devices such as computers, tablets, and smartphones that users use for work and meetings.

[0796] An "emotion engine" is software or algorithms that recognize and analyze a user's emotional state from voice data and operation data.

[0797] "Transcription tool" is a general term for software and services that convert audio data into text data.

[0798] "Natural language processing algorithm" is a general term for algorithms used to extract important points and keywords from text data.

[0799] A "company-wide portal site" is a web-based platform that users can access for sharing and managing information within a company.

[0800] The "recommendation function" is a function that suggests the most suitable materials and information based on the user's profile and behavioral data.

[0801] A "business manual" is a document that contains procedures and guidelines for carrying out a specific business or process.

[0802] A "business flow" is a diagram or table that visually represents the steps and flow of a specific business or process.

[0803] A "case study collection" is a document or data that collects and organizes past cases and case studies.

[0804] A "glossary" is a list of technical terms and definitions related to a particular field or task.

[0805] A "business handover document" is a document that lists the necessary information and procedures to be used when handing over business operations.

[0806] The present invention combines an emotion engine with a system that promotes the standardization and efficiency of user work. Specific embodiments for implementing this system are described below.

[0807] This system is mainly composed of three entities: a server, a terminal, and a user.

[0808] Device:

[0809] 1. Install and configure the agent:

[0810] The server installs agent software on the user's information device. This agent collects in real time the operations performed by the user on a PC or tablet, as well as audio data during a meeting. For example, when a user gives a presentation about a new product, the agent collects detailed information about the presentation.

[0811] 2. Data collection and transmission:

[0812] The terminal agent collects user operation data and conference audio data and sends them to the server. The emotion engine also collects the user's emotion data (e.g., excitement or stress) and sends this data to the server. The user simply carries out their normal work, and data collection is carried out in the background.

[0813] server:

[0814] 1. Transcription of audio data:

[0815] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically analyzed and the dialogue is recorded in text format. For example, a log stating "Voice data text conversion completed" is generated.

[0816] 2. Key Points Extraction Using Natural Language Processing (NLP):

[0817] The server applies an NLP algorithm (e.g., spaCy) to the text data to extract key points and keywords. For example, keywords such as "new product features" and "market strategy" are identified. At this step, the "extracted key points data" are passed on to the next step of processing.

[0818] 3. Emotional Data Analysis:

[0819] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the collected emotion data. It detects the user's emotional state (e.g., excitement, stress, etc.) from the tone of the voice and operation logs. For example, if a user becomes excited during a presentation, this state is recorded as "user is excited."

[0820] 4. Automatic generation of materials:

[0821] The server combines key information and emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that highlights the features of a new product that a user was particularly excited about. Materials are generated using a template engine (e.g., Jinja2).

[0822] 5. Portal site management and recommendations:

[0823] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, they receive a notification that a "business manual highlighting the features of a new product" is displayed.

[0824] User:

[0825] Users can receive notifications and access recommended materials from the portal site, allowing them to efficiently learn about new job content and deepen their understanding.

[0826] Specific examples

[0827] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server converts the collected voice data into text and extracts the main points using an NLP algorithm. The emotion engine also analyzes whether the user is excited.

[0828] For example, if the system detects that the user is excited when explaining the features of a new product, the system will automatically generate an operational manual that emphasizes those features.When a new employee accesses the portal site, the system will present them with an operational manual that is fun and easy to learn, recommended based on emotional data.

[0829] Prompt Sentence Examples

[0830] Prompt statement:

[0831] Collect voice and operation data from users giving a presentation on a new product, extract key points using an NLP algorithm, and analyze the user's level of excitement using an emotion engine. Then, automatically generate an operations manual that highlights the parts that caused the user to feel particularly excited.

[0832] This prompt sentence is used to instruct the generative AI model, ensuring efficient operation of the system.

[0833] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0834] Step 1:

[0835] Installing and Configuring the Agent

[0836] The server installs agent software on the user's device. This agent collects in real time the operations and voice data performed on the device, such as a PC or tablet. For example, when a user gives a presentation about a new product, the agent is ready to collect the content of the presentation. The input is the agent software, and the output is a notification that the installation is complete. Specifically, the agent is installed and initial settings are performed.

[0837] Step 2:

[0838] Data collection and transmission

[0839] The agent on the terminal collects user operation data and voice data during the conference in real time and sends this data to the server. For example, when a user starts a presentation, the content of the presentation is collected. Emotional data (excitement and stress) is also collected at the same time. The input is voice data and operation data, and the output is the data sent to the server. Specifically, the agent records the voice during the conference and keeps an operation log.

[0840] Step 3:

[0841] Transcription of audio data

[0842] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically converted into text format. The input is the voice data sent to the server, and the output is the converted text data. Specifically, the transcription tool analyzes the voice data and outputs the dialogue as text.

[0843] Step 4:

[0844] Extracting key points using natural language processing (NLP)

[0845] The server analyzes the text data converted by the transcription tool using an NLP algorithm (e.g., spaCy) to extract important key points and keywords. For example, key points such as "new product features" and "market strategy" are identified. The converted text data is the input, and the extracted key point data is the output. Specifically, the NLP algorithm automatically analyzes and extracts keywords and key points within the text.

[0846] Step 5:

[0847] Emotional Data Analysis

[0848] The server analyzes the collected emotion data using an emotion engine (e.g., IBM Watson Tone Analyzer) and recognizes the user's emotional state (e.g., excitement, stress, etc.). The input is emotion data including voice tone and operation logs, and the output is analyzed emotional state information. Specifically, the emotion engine analyzes the voice tone and detects the user's emotional state.

[0849] Step 6:

[0850] Automatic generation of materials

[0851] The server combines key information extracted by NLP with emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that emphasizes the features of a new product that a user described with particular enthusiasm. The input is the extracted key information and analyzed emotion data, and the output is automatically generated materials. Specifically, various materials are generated using a template engine (for example, Jinja2).

[0852] Step 7:

[0853] Management and recommendations on the portal site

[0854] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, a business manual with the characteristics that the employee described with excitement is recommended. The input is the generated materials and the user's profile information, and the output is the recommended material information. Specifically, the server uploads the materials and executes the recommendation function.

[0855] (Application example 2)

[0856] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0857] Lack of work efficiency and standardization in traditional logistics centers is a problem. In particular, it takes time to train new workers and understand their work, and it is difficult to respond immediately to their emotional state. This invention is designed to solve these problems and improve work efficiency and the training process.

[0858] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0859] In this invention, the server includes means for installing an agent in a user's information processing device and collecting the contents of daily work and meetings, means for analyzing the collected data and extracting key points, means for automatically generating work guides, work processes, case studies, glossaries, and work handover documents based on the extracted key points, means for storing the generated materials on an information sharing site and providing users with a recommendation function, means for collecting and analyzing worker emotion data in real time, and means for recommending optimal work guides and training materials based on the collected emotion data, thereby making it possible to improve work efficiency and speed up the training process.

[0860] An "agent" is software that is installed on a user's information processing device and collects the contents of daily tasks and meetings.

[0861] An "information processing device" is an electronic device for collecting, analyzing, and communicating data.

[0862] "Daily work and meeting contents" is information about the work and meeting contents that the user performs on a daily basis.

[0863] "Analyzing data" refers to processing collected information to find useful patterns and takeaways.

[0864] "Gist extraction" refers to extracting important information or key content from the analyzed data.

[0865] A "work guide" is a procedure manual for performing specific tasks efficiently and accurately.

[0866] A "business process" is a series of work steps to achieve a specific goal.

[0867] A "case study collection" is a document that compiles past examples and case studies.

[0868] A "glossary" is a document that lists definitions of technical terms and terms.

[0869] A "business handover document" is a document that contains the information necessary when a business person in charge changes.

[0870] An "information sharing site" is a web platform that allows generated materials and information to be stored and shared.

[0871] The "recommended function" is a function that provides the most suitable materials and information to the user.

[0872] "Emotion data" is data that represents the emotional state of the user.

[0873] "Real-time collection" refers to obtaining data immediately.

[0874] "Analysis" refers to examining collected data in detail to find meaning and patterns.

[0875] "Recommending optimal work guides and educational materials according to the user's emotional state" refers to providing the most appropriate work instructions and educational materials in consideration of the user's emotional state.

[0876] This invention is a work support system using smart glasses that aims to improve work efficiency and standardize work in logistics centers.

[0877] Hardware and Software Configuration

[0878] Device:

[0879] The smart glasses have a built-in microphone to capture the worker's voice, allowing them to collect voice input in real time, and a built-in display that can display information on the screen.

[0880] server:

[0881] The server uses the following software to analyze the collected voice data:

[0882] 1. Speech Recognition: A speech recognition library for converting collected voice data into text.

[0883] 2. Transformers (Hugging Face's NLP library): Analyzes text data after speech recognition using natural language processing (NLP) to extract key points.

[0884] 3. Emotion Recognition: An emotion recognition library for recognizing the user's emotional state from collected voice data.

[0885] Specific processing flow:

[0886] 1. Collecting voice data: The microphone built into the smart glasses captures the voice of the worker and sends the data to the server.

[0887] 2. Transcription of voice data: The server converts the collected voice data into text using the Speech Recognition library.

[0888] 3. Gist Extraction: Use the Transformers NLP library on the text data to extract key points.

[0889] 4. Emotion Data Analysis: Use the Emotion Recognition library to analyze the worker's emotional state from the tone and content of their voice.

[0890] 5. Automatic generation of work guides and educational materials: Automatically generate appropriate work guides and educational materials based on key information and emotion data.

[0891] 6. Information display: The generated materials and recommended information are displayed in real time on the smart glasses display.

[0892] User:

[0893] By wearing the smart glasses, workers can receive feedback and guidance information tailored to their own working conditions, which is expected to improve work efficiency and enhance proficiency.

[0894] Specific examples

[0895] For example, if a worker picking a new product says, "I don't know how to pick this product," the voice is captured by the microphone in the smart glasses. The voice data is sent to the server, converted into text, and analyzed, and the optimal picking method and handling instructions for that product are displayed in real time on the worker's smart glasses.

[0896] Example prompts to input to the generative AI model:

[0897] "Please tell me how to analyze voice data and recognize emotions to provide appropriate feedback to workers who are feeling stressed while working, and provide work guidance in real time."

[0898] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0899] Step 1:

[0900] The microphone in the smart glasses captures the worker's voice.

[0901] Input: Worker's voice

[0902] How it works: A microphone built into the smart glasses captures the worker's voice and transmits the voice data to a server in real time.

[0903] Output: Audio data

[0904] Step 2:

[0905] The server converts the received voice data into text using the Speech Recognition library.

[0906] Input: Audio data

[0907] Operation: The server inputs the voice data into the Speech Recognition library and converts it into text data.

[0908] Output: Text data

[0909] Step 3:

[0910] The server uses the Transformers NLP library to parse the text data and extract key points.

[0911] Input: Text data

[0912] How it works: The server inputs text data into an NLP library and applies natural language processing algorithms to extract important key information and concepts.

[0913] Output: Key information

[0914] Step 4:

[0915] The server uses the Emotion Recognition library to recognize emotional states from text data.

[0916] Input: Text data

[0917] How it works: The server inputs the collected text data into the Emotion Recognition library, which analyzes the user's emotional state from the tone and content of the voice.

[0918] Output: Emotion data

[0919] Step 5:

[0920] The server automatically generates work guides and educational materials based on the key information and emotion data.

[0921] Input: Key points, emotion data

[0922] Operation: The server inputs key information and emotion data into the DocumentGenerator module, and automatically generates optimal work guides and educational materials based on that information.

[0923] Output: Work guides, educational materials

[0924] Step 6:

[0925] The generated materials and recommended information are displayed on the smart glasses display.

[0926] Input: Work guides, educational materials

[0927] How it works: The server sends the generated materials and recommended information to the smart glasses, which then display it on their screens. Workers can then perform their work while viewing the information displayed in real time.

[0928] Output: Information displayed on the smart glasses display

[0929] keyword:

[0930] Generative AI model, prompt sentence

[0931] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0932] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0933] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[0934] [Third embodiment]

[0935] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0936] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0937] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0938] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0939] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0940] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0941] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0942] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0943] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0944] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0945] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0946] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[0947] The present invention is a system for achieving standardization and efficiency in business operations. This system is mainly composed of the following means:

[0948] 1. Agent installation and data collection:

[0949] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. For example, the agent collects the content of a user's presentation about a new product.

[0950] 2. Data analysis and key points:

[0951] The server runs algorithms to analyze the collected data. The collected voice data is first converted into text data, and then natural language processing (NLP) algorithms are applied to extract key points. For example, key information such as "new product features" and "market strategy" can be identified from the presentation content.

[0952] 3. Automatic generation of materials:

[0953] Based on the key point extraction results, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents by embedding data into predefined templates. For example, a business manual describing the features of a new product can be automatically generated.

[0954] 4. Portal site management and recommendations:

[0955] The generated materials are stored on a company-wide portal site. The server recommends appropriate materials based on the user's profile, past work history, and search history. For example, when a new employee accesses the portal, they are recommended a new product manual or case study.

[0956] Program processing

[0957] Device:

[0958] The agent sends user operations and voice data to the server in real time. Users simply carry out their usual work or meetings without any special operations.

[0959] server:

[0960] The server first converts the audio data into text, then uses NLP to extract key points. Based on the analyzed key points, various documents are automatically generated using templates and uploaded to the portal site.

[0961] User:

[0962] Users can access the portal site and check the recommended materials to deepen their understanding of their work. For example, when assigned to a new job, they can instantly access the manuals and workflows they need.

[0963] Specific examples

[0964] When a user gives a presentation on a new product, the agent collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[0965] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload for users.

[0966] The processing flow will be explained below.

[0967] Step 1:

[0968] Installing and Configuring the Agent

[0969] server:

[0970] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[0971] Device:

[0972] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[0973] User:

[0974] The user confirms that the agent software is installed and configured.

[0975] Step 2:

[0976] Starting Data Collection

[0977] server:

[0978] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[0979] Device:

[0980] Based on specified conditions, the agent collects data on PC operations and audio data during meetings and sends it to the server in real time.

[0981] User:

[0982] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[0983] Step 3:

[0984] Transcription of audio data

[0985] server:

[0986] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[0987] Step 4:

[0988] Extracting key points using natural language processing (NLP)

[0989] server:

[0990] The server applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[0991] Step 5:

[0992] Analysis of operation log data

[0993] server:

[0994] The server analyzes the collected operation log data to identify frequently occurring operation patterns and business flows. This data will be used as the basis for later generation of documents.

[0995] Step 6:

[0996] Automatic generation of materials

[0997] server:

[0998] Based on the results of key point extraction and operation log analysis, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates. For example, information about the features of a new product can be automatically added to the business manual.

[0999] Step 7:

[1000] Preservation and management of generated materials

[1001] server:

[1002] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[1003] Step 8:

[1004] Uploading to the portal site and making recommendations

[1005] server:

[1006] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on the user profile and work history to provide the most suitable documents to specific users.

[1007] Device:

[1008] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[1009] User:

[1010] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[1011] By sequentially executing these steps, the present invention realizes automatic recording, analysis, automation, and efficiency of business operations.

[1012] Example 1

[1013] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1014] In conventional business processes, a great deal of effort was required to standardize and streamline operations. Manually recording meeting content and work procedures and then creating work manuals and workflows based on those records was particularly tedious work. New employees and transferees also faced the problem of being unable to quickly obtain the information they needed. A system that could automate and streamline a series of tasks was needed.

[1015] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1016] In this invention, the server includes means for installing agent software on a terminal and collecting the contents of daily work and meetings, means for periodically transmitting the collected operation logs and voice data to the server, means for converting the transmitted voice data into text data, means for extracting key points by applying a natural language processing algorithm, means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points, and means for storing the generated materials on a company-wide portal site and providing a recommendation function to users. This makes it possible to automate a series of processes from collecting work data to generating materials and making recommendations.

[1017] "Agent software" is software that is installed on the user's terminal and collects PC operations and conference audio in real time.

[1018] An "operation log" refers to the history of operations performed by a user on a PC, such as keyboard input, mouse clicks, and slide switching.

[1019] "Audio data" refers to data that records what a user says during a meeting or presentation.

[1020] A "server" is a computer system that receives data sent from a terminal, analyzes, converts, and processes the data to generate materials, and stores the materials on a portal site.

[1021] "Text data" refers to character information that has been converted by analyzing voice data.

[1022] "Natural language processing algorithm" is a general term for programs and methods for extracting key points from text data.

[1023] "Key points" refer to important information or keywords that are analyzed and extracted from the collected data.

[1024] A "business manual" refers to a document that describes business procedures and methods.

[1025] "Business flow" refers to a diagram of a series of business processes and procedures.

[1026] A "case study" refers to a document that compiles past business success stories and examples of implementation.

[1027] A "glossary" refers to a document that explains business-related terminology.

[1028] A "business handover document" refers to a document that contains the information necessary to ensure a smooth handover of business operations.

[1029] "Portal site" refers to a website where generated materials are stored and can be accessed and viewed by users.

[1030] The "recommendation function" refers to a function that suggests appropriate materials based on the user's work history and search behavior.

[1031] This invention is a system for standardizing and streamlining user operations. This system mainly includes the following components: agent software, a server, and a portal site. Each component will be described in detail below, along with its operation.

[1032] Agent software installation and data collection

[1033] Device:

[1034] First, agent software is installed on the user's device. This agent software collects in real time the user's PC operations (keyboard input, mouse clicks, switching slides, etc.) and the voices spoken during the meeting. For example, when a user gives a presentation about a new product, the agent software captures the operation log and voice data.

[1035] Data transmission and text conversion

[1036] Device:

[1037] The collected data is sent to the server at regular intervals (for example, every minute) or when an event occurs, allowing for real-time processing of the data.

[1038] server:

[1039] The server takes the received audio data and uses a speech recognition service such as the Google Cloud Speech-to-Text API to convert it into text data, which is then used in the next processing step.

[1040] Extracting key points using natural language processing

[1041] server:

[1042] The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract key information (key points). For example, keywords and phrases such as "new product features" and "market strategy" can be extracted from the text of a new product presentation.

[1043] Automatic generation of materials

[1044] server:

[1045] Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates. For example, a business manual describing the features and market strategy of a new product can be generated.

[1046] Management and recommendations on the portal site

[1047] server:

[1048] The generated materials are stored on a company-wide portal site. The server recommends the most appropriate materials based on the user's work history and search behavior. For example, when a new employee accesses the portal site, a new product manual and case studies are displayed on the top page.

[1049] User:

[1050] Users can deepen their understanding of the business by accessing the portal site and checking the recommended materials, which will enable them to respond quickly to new business tasks.

[1051] Specific examples

[1052] When a user gives a presentation about a new product, the agent software collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice data into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[1053] Example prompts for generative AI models

[1054] 1. "Please summarize the main points of this presentation. The key themes are the features of the new product and the market strategy."

[1055] 2. "Please generate an operations manual from the following text data. Use Microsoft Word as the template."

[1056] 3. "Please extract keywords from the audio data of this conference and upload related resources to the portal site."

[1057] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload of users.

[1058] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1059] Step 1:

[1060] Agent software installation and data collection

[1061] Terminal: Agent software is installed on the user's terminal. The agent software collects in real time the operations performed by the user on the PC (keyboard input, mouse clicks, switching slides, etc.) and the voice uttered during the meeting. The collected data is saved as an operation log and voice data.

[1062] Input: User's PC operation, conference audio

[1063] Output: Operation log, audio data

[1064] Specific operation: Once the agent software is installed and the user begins to operate the device, the operation log and recorded audio are captured in real time and temporarily stored on the device.

[1065] Step 2:

[1066] Data transmission and text conversion

[1067] Terminal: The collected operation log and voice data are sent to the server periodically (for example, every minute).

[1068] Input: Operation log, audio data

[1069] Output: Operation log and audio data sent to the server

[1070] Specific operation: The terminal uploads all the data collected by the agent software to the server at regular intervals or at specific events (e.g., the end of a meeting).

[1071] Server: The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[1072] Input: Audio data

[1073] Output: Text data

[1074] Specific operation: The server sequentially analyzes the received audio files, calls the speech recognition API to convert the audio into text, and passes the result to the next processing step.

[1075] Step 3:

[1076] Extracting key points using natural language processing

[1077] Server: The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract important information and key points.

[1078] Input: Text data

[1079] Output: Key information

[1080] Specific operations: Analyze text data using NLP tools to extract important keywords and phrases such as "new product features" and "market strategy."

[1081] Step 4:

[1082] Automatic generation of materials

[1083] Server: Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates.

[1084] Input: Key information

[1085] Output: Business manuals, business flow, case studies, glossary, business handover documents

[1086] Specific operation: The server embeds key information into a pre-prepared template and generates a document file in a format such as "New Product Manual.docx."

[1087] Step 5:

[1088] Management and recommendations on the portal site

[1089] Server: The generated documents are stored on the company's portal site. The server recommends appropriate documents based on the user's work history and search behavior.

[1090] Input: Generated materials, user work history, search behavior

[1091] Output: Recommended materials

[1092] Specific operation: Documents are uploaded to the portal site, and when a user accesses it, an algorithm is run based on the user's previous work history and search behavior, and documents appropriate for the user are displayed on the top page.

[1093] Step 6:

[1094] Check the materials

[1095] User: The user accesses the portal site and checks the recommended materials, which allows them to deepen their understanding of the business.

[1096] Input: Recommended materials

[1097] Output: Understanding business operations by referring to materials

[1098] Specific operation: A user logs in to the portal site, clicks on the link to recommended materials displayed on the top page, and views the manuals and flow charts.

[1099] (Application example 1)

[1100] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1101] At logistics centers, product handling and picking route procedures change frequently, making it time-consuming to create instructions and manuals to accommodate these changes. In addition, real-time, reliable information provision is required so that workers can immediately understand new instructions and procedures and work efficiently. Conventional methods lack the means to efficiently resolve these issues, making it difficult to streamline and standardize operations.

[1102] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1103] In this invention, the server includes: means for installing an agent on a user's computer and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating work manuals, work flows, case studies, glossaries, and work handover documents based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means including a smartphone agent application for collecting voice data and operation data in real time and sending it to the server; means for analyzing the collected data on the server using voice recognition and natural language processing algorithms and extracting important information; means for automatically generating efficient work instructions based on the extracted key points and notifying the smartphone in real time; and means for providing a portal site that recommends materials based on the user's work history and profile. This enables fast and accurate information provision and work standardization.

[1104] An "agent" is a program that is installed on a user's computer or smartphone and collects information about daily work and meetings.

[1105] "Audio data" refers to data that records the voice generated during a user's work or a meeting.

[1106] "Operation data" refers to data that records the operations performed by a user on a computer or smartphone.

[1107] A "server" is a device or system that has the function of analyzing collected data, extracting key points, and automatically generating and managing materials.

[1108] A "natural language processing algorithm" is an analytical method for extracting key points from text data such as voice data.

[1109] A "business manual" is a guide that describes the procedures and methods for a specific business operation.

[1110] A "business flow" is a diagram or text that shows the flow or process of a business.

[1111] A "case study collection" is a collection of specific past business cases that have been collected and compiled.

[1112] A "glossary" is a document that collects technical terms and their definitions related to a particular field or task.

[1113] A "business handover document" is a document that describes the handover details when a business person in charge is changed.

[1114] A "portal site" is a website that stores generated materials and provides users with a recommendation function.

[1115] The "agent application for smartphones" is software that is installed on a smartphone and collects voice data and operation data in real time and transmits them to a server.

[1116] "Real-time notification" is a function that instantly sends generated work instructions to a smartphone.

[1117] "Key point extraction" is the process of extracting important information or points from collected data.

[1118] The "recommendation function" is a function that recommends the most appropriate materials and information based on the user's work history and profile.

[1119] This invention provides a system that includes a smartphone application and back-end system used in logistics centers and other locations. Smartphones with installed agents collect the daily work details of workers and send them to a server, thereby generating and notifying work instructions in real time. Below, we will explain each component of the system and its processing.

[1120] Terminal part (smartphone)

[1121] Hardware / software used: Smartphone, Android OS, iOS, agent app

[1122] Voice data collection: The voice of the worker is collected in real time using the microphone on the smartphone. For example, when a worker explains how to handle a new product, the voice data is collected.

[1123] Operation data collection: Includes a screen capture function for recording screen operations. For example, when operating a screen to confirm product picking instructions, that operation data is collected.

[1124] Server part

[1125] Hardware / software used: Cloud servers (AWS, Google Cloud, etc.), speech recognition APIs (Google Speech-to-Text API, etc.), NLP libraries (SpaCy, NLTK)

[1126] Speech-to-text conversion: Converts collected voice data into text data. For example, a voice explaining a new product is converted into text data.

[1127] Key point extraction: Using NLP algorithms to extract important information from text data, such as the features and handling precautions of a new product.

[1128] Automatic generation of work instructions: Based on the extracted key points, work instructions are automatically generated using templates. For example, a handling procedure manual for a new product is generated.

[1129] Uploading materials: The generated materials are uploaded to a portal site and recommendations are made based on the user profile. For example, when a worker accesses the portal site, a handling manual for a new product is recommended.

[1130] Real-time notification section

[1131] Hardware / software used: Smartphone, push notification API (Firebase Cloud Messaging, etc.)

[1132] Notifications: Generated work instructions are sent via push notification in real time. For example, the handling procedure for a new product is immediately sent to the worker's smartphone.

[1133] Examples and prompts

[1134] Examples:

[1135] Scenario: A presentation is being given to explain how to handle a new product.

[1136] Data collection: The agent app collects the audio of the presentation.

[1137] Data analysis: The audio is converted into text and the key point, "How to handle the new product," is extracted.

[1138] Instruction generation: Generate handling instructions for new products based on the extracted key points.

[1139] Notification: The procedure manual is pushed to the picking worker's smartphone.

[1140] Example prompt for a generative AI model:

[1141] Analyze the audio recording about how to handle new products and extract the following information:

[1142] 1. New product handling procedures

[1143] 2. Materials and Equipment Needed

[1144] 3. Points to note

[1145] Based on the extracted information, please automatically generate a work instruction in the following format:

[1146] 1. Product name

[1147] 2. Handling Procedures

[1148] 3. Required Materials

[1149] 4. Points to note

[1150] Please notify the smartphone app of the generated work instructions in the following format:

[1151] 1. Title: New Product Handling Instructions

[1152] 2. Content: Extracted work instructions

[1153] This system, configured in this way, will enable the efficiency and standardization of operations at logistics centers, and by providing appropriate instructions to workers in real time, it is expected to reduce errors and improve overall work efficiency.

[1154] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1155] Step 1:

[1156] The agent app on the smartphone collects the worker's voice data and operation data in real time. The voice data is recorded through the smartphone's microphone, and the operation data is recorded using the screen capture function. The voice and operation data are collected as input and sent to the server for later analysis.

[1157] Step 2:

[1158] The collected voice data and operation data are sent to the server. The server receives this data and first converts the voice data into text using a voice recognition API (e.g., Google Speech-to-Text API). The input is the voice data, and the output is the converted text data. Specifically, voice data is given as input, and the voice recognition API outputs text data.

[1159] Step 3:

[1160] The server applies a natural language processing (NLP) algorithm (e.g., SpaCy, NLTK) to the converted text data to extract key points. The input is text data obtained by speech recognition, and the extracted results containing important information (key points) are output. Specifically, text data is given as input, and the NLP algorithm outputs key points such as "features of the new product" and "how to use it."

[1161] Step 4:

[1162] The server automatically generates work instructions based on the key point extraction results. The input is the extracted key point information, and a document containing the work instructions (e.g., handling procedures for a new product) is generated using a template. Specifically, the key point information is embedded in the template, and the generated work instructions are output.

[1163] Step 5:

[1164] The server uploads the generated work instructions to a portal site and recommends related materials based on the user's profile and work history. The input is the generated work instructions and user profile, and the portal site's recommendation information is output. Specifically, the work instructions are stored on the portal site, and appropriate materials are displayed according to the user.

[1165] Step 6:

[1166] The generated work instructions are sent in real time using a smartphone push notification API (e.g., Firebase Cloud Messaging). The input is the work instructions and information about the recipient smartphone, and the output is a notification message sent to the smartphone. Specifically, the work instructions are sent as a push notification message and are instantly displayed on the worker's smartphone.

[1167] Step 7:

[1168] The user (worker) checks the work instructions notified on their smartphone and carries out the work. The input is the work instructions displayed on the smartphone, and the output is the work procedure to be carried out based on them. Specifically, the user taps the notification to open the work instructions and carries out the work according to the instructed procedure.

[1169] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1170] This invention combines an emotion engine with a system that promotes the standardization and efficiency of users' work. This system is composed of the following means:

[1171] 1. Install and configure the agent:

[1172] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. When a user gives a presentation about a new product, the agent collects the content.

[1173] 2. Data collection and analysis:

[1174] The server analyzes the collected voice data and operation data. The voice data is converted into text data using a transcription tool, and key points are extracted using a natural language processing (NLP) algorithm. An emotion engine then recognizes the user's emotions from the voice and operation data. For example, not only is important information such as "new product features" and "market strategy" identified, but the emotions (e.g., excitement, stress) expressed when the user describes that information are also recognized.

[1175] 3. Automatic generation of materials:

[1176] The server automatically generates a business manual, business flow, case studies, glossary, and business handover document based on the extracted key information and emotion data. For example, a business manual may be generated that particularly emphasizes the features of a new product that the user excitedly explained.

[1177] 4. Portal site management and recommendations:

[1178] The generated materials are stored on a company-wide portal site. The server recommends the most suitable materials based on each user's profile, work history, search history, and even emotional data. For example, when a new employee accesses the portal site, they are presented with an interesting work manual recommended based on their emotional data.

[1179] Program processing

[1180] Device:

[1181] The agent collects user operations and voice data in real time and sends it to the server. It also acquires the user's emotional data using an emotion engine and sends this information to the server. The user simply needs to carry out their normal work or meetings.

[1182] server:

[1183] 1. Transcription of audio data:

[1184] The server converts the collected voice data into text data, using a transcription tool to accurately convert the voice data into text format.

[1185] 2. Key Points Extraction Using Natural Language Processing (NLP):

[1186] The server applies NLP algorithms to the text data to extract key points and keywords, such as "new product features" and "target market."

[1187] 3. Emotional Data Analysis:

[1188] The server analyzes the collected emotion data using the emotion engine to detect the user's emotional state (e.g., stress level and motivation) from the tone of voice and operation logs.

[1189] 4. Automatic generation of materials:

[1190] The server combines key information and emotion data to automatically generate business manuals, workflows, case studies, glossaries, and business handover documents. The generated materials reflect the user's emotions.

[1191] 5. Portal site management and recommendations:

[1192] The server uploads the generated documents to a company-wide portal site and organizes them by category. It also recommends the most suitable documents based on user profiles and emotional data. It also notifies users of the access information for the documents.

[1193] User:

[1194] Users receive notifications and access recommended materials from the portal site, which allows them to efficiently learn and deepen their understanding of their work.

[1195] Specific examples

[1196] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server then converts the voice data into text and uses NLP to extract key points. The emotion engine also analyzes whether the user is excited. Based on the extracted key points and emotional data, a business manual is automatically generated with sections that need to be emphasized highlighted. When new employees access the portal site, this business manual is recommended, and information is provided in a fun and easy-to-learn format.

[1197] In this way, this system supports the standardization and efficiency of business operations, and by utilizing user emotion data, it is possible to provide more appropriate and effective content.

[1198] The processing flow will be explained below.

[1199] Step 1:

[1200] Installing and Configuring the Agent

[1201] server:

[1202] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[1203] Device:

[1204] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[1205] User:

[1206] The user confirms that the agent software is installed and configured.

[1207] Step 2:

[1208] Starting Data Collection

[1209] server:

[1210] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[1211] Device:

[1212] The agent collects data on PC operations and voice data during meetings based on specified conditions and sends it to the server in real time. The emotion engine also analyzes the user's voice tone and operation patterns.

[1213] User:

[1214] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[1215] Step 3:

[1216] Transcription of audio data

[1217] server:

[1218] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[1219] Step 4:

[1220] Extracting key points using natural language processing (NLP)

[1221] server:

[1222] The server then applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[1223] Step 5:

[1224] Emotional Data Analysis

[1225] server:

[1226] The server analyzes the voice tones and operation patterns collected by the emotion engine to determine the user's emotional state. For example, it detects whether the user is excited when explaining a new product. The emotional data is stored together with other business data.

[1227] Step 6:

[1228] Automatic generation of materials

[1229] server:

[1230] The server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates, using the key point extraction results and emotion data. Materials are created in a way that emphasizes parts that strongly reflect the user's emotions (for example, the features of a new product that was explained in an excited state).

[1231] Step 7:

[1232] Preservation and management of generated materials

[1233] server:

[1234] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[1235] Step 8:

[1236] Uploading to the portal site and making recommendations

[1237] server:

[1238] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on user profiles and emotional data to provide the most suitable documents to specific users.

[1239] Device:

[1240] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[1241] User:

[1242] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[1243] By executing each step in this way, the present invention realizes automatic recording, analysis, document generation, and efficiency improvement of business operations, and further utilizes emotion data to provide more appropriate and effective content.

[1244] Example 2

[1245] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1246] Conventional business management systems lack the functionality to efficiently collect users' daily work and meeting details, extract key points, and automatically generate materials based on them. Furthermore, materials are not created taking into account the emotional state of the user, making it difficult to achieve standardized and efficient business processes. Furthermore, they lack the functionality to optimally recommend generated materials, which makes it difficult to provide appropriate information to specific users.

[1247] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for installing an agent in a user's information device and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means for converting collected voice data into text data using a transcription tool; means for recognizing and analyzing the user's emotional state from the voice data and operation data using an emotion engine; and means for automatically generating materials by combining key point information and emotion data. This enables standardization and efficiency of work and the provision of information that takes emotion data into consideration.

[1248] An "agent" is software that is installed on a user's information device and collects the contents of daily tasks and meetings.

[1249] "Information devices" is a general term for electronic devices such as computers, tablets, and smartphones that users use for work and meetings.

[1250] An "emotion engine" is software or algorithms that recognize and analyze a user's emotional state from voice data and operation data.

[1251] "Transcription tool" is a general term for software and services that convert audio data into text data.

[1252] "Natural language processing algorithm" is a general term for algorithms used to extract important points and keywords from text data.

[1253] A "company-wide portal site" is a web-based platform that users can access for sharing and managing information within a company.

[1254] The "recommendation function" is a function that suggests the most suitable materials and information based on the user's profile and behavioral data.

[1255] A "business manual" is a document that contains procedures and guidelines for carrying out a specific business or process.

[1256] A "business flow" is a diagram or table that visually represents the steps and flow of a specific business or process.

[1257] A "case study collection" is a document or data that collects and organizes past cases and case studies.

[1258] A "glossary" is a list of technical terms and definitions related to a particular field or task.

[1259] A "business handover document" is a document that lists the necessary information and procedures to be used when handing over business operations.

[1260] The present invention combines an emotion engine with a system that promotes the standardization and efficiency of user work. Specific embodiments for implementing this system are described below.

[1261] This system is mainly composed of three entities: a server, a terminal, and a user.

[1262] Device:

[1263] 1. Install and configure the agent:

[1264] The server installs agent software on the user's information device. This agent collects in real time the operations performed by the user on a PC or tablet, as well as audio data during a meeting. For example, when a user gives a presentation about a new product, the agent collects detailed information about the presentation.

[1265] 2. Data collection and transmission:

[1266] The terminal agent collects user operation data and conference audio data and sends them to the server. The emotion engine also collects the user's emotion data (e.g., excitement or stress) and sends this data to the server. The user simply carries out their normal work, and data collection is carried out in the background.

[1267] server:

[1268] 1. Transcription of audio data:

[1269] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically analyzed and the dialogue is recorded in text format. For example, a log stating "Voice data text conversion completed" is generated.

[1270] 2. Key Points Extraction Using Natural Language Processing (NLP):

[1271] The server applies an NLP algorithm (e.g., spaCy) to the text data to extract key points and keywords. For example, keywords such as "new product features" and "market strategy" are identified. At this step, the "extracted key points data" are passed on to the next step of processing.

[1272] 3. Emotional Data Analysis:

[1273] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the collected emotion data. It detects the user's emotional state (e.g., excitement, stress, etc.) from the tone of the voice and operation logs. For example, if a user becomes excited during a presentation, this state is recorded as "user is excited."

[1274] 4. Automatic generation of materials:

[1275] The server combines key information and emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that highlights the features of a new product that a user was particularly excited about. Materials are generated using a template engine (e.g., Jinja2).

[1276] 5. Portal site management and recommendations:

[1277] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, they receive a notification that a "business manual highlighting the features of a new product" is displayed.

[1278] User:

[1279] Users can receive notifications and access recommended materials from the portal site, allowing them to efficiently learn about new job content and deepen their understanding.

[1280] Specific examples

[1281] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server converts the collected voice data into text and extracts the main points using an NLP algorithm. The emotion engine also analyzes whether the user is excited.

[1282] For example, if the system detects that the user is excited when explaining the features of a new product, the system will automatically generate an operational manual that emphasizes those features.When a new employee accesses the portal site, the system will present them with an operational manual that is fun and easy to learn, recommended based on emotional data.

[1283] Prompt Sentence Examples

[1284] Prompt statement:

[1285] Collect voice and operation data from users giving a presentation on a new product, extract key points using an NLP algorithm, and analyze the user's level of excitement using an emotion engine. Then, automatically generate an operations manual that highlights the parts that caused the user to feel particularly excited.

[1286] This prompt sentence is used to instruct the generative AI model, ensuring efficient operation of the system.

[1287] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1288] Step 1:

[1289] Installing and Configuring the Agent

[1290] The server installs agent software on the user's device. This agent collects in real time the operations and voice data performed on the device, such as a PC or tablet. For example, when a user gives a presentation about a new product, the agent is ready to collect the content of the presentation. The input is the agent software, and the output is a notification that the installation is complete. Specifically, the agent is installed and initial settings are performed.

[1291] Step 2:

[1292] Data collection and transmission

[1293] The agent on the terminal collects user operation data and voice data during the conference in real time and sends this data to the server. For example, when a user starts a presentation, the content of the presentation is collected. Emotional data (excitement and stress) is also collected at the same time. The input is voice data and operation data, and the output is the data sent to the server. Specifically, the agent records the voice during the conference and keeps an operation log.

[1294] Step 3:

[1295] Transcription of audio data

[1296] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically converted into text format. The input is the voice data sent to the server, and the output is the converted text data. Specifically, the transcription tool analyzes the voice data and outputs the dialogue as text.

[1297] Step 4:

[1298] Extracting key points using natural language processing (NLP)

[1299] The server analyzes the text data converted by the transcription tool using an NLP algorithm (e.g., spaCy) to extract important key points and keywords. For example, key points such as "new product features" and "market strategy" are identified. The converted text data is the input, and the extracted key point data is the output. Specifically, the NLP algorithm automatically analyzes and extracts keywords and key points within the text.

[1300] Step 5:

[1301] Emotional Data Analysis

[1302] The server analyzes the collected emotion data using an emotion engine (e.g., IBM Watson Tone Analyzer) and recognizes the user's emotional state (e.g., excitement, stress, etc.). The input is emotion data including voice tone and operation logs, and the output is analyzed emotional state information. Specifically, the emotion engine analyzes the voice tone and detects the user's emotional state.

[1303] Step 6:

[1304] Automatic generation of materials

[1305] The server combines key information extracted by NLP with emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that emphasizes the features of a new product that a user described with particular enthusiasm. The input is the extracted key information and analyzed emotion data, and the output is automatically generated materials. Specifically, various materials are generated using a template engine (for example, Jinja2).

[1306] Step 7:

[1307] Management and recommendations on the portal site

[1308] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, a business manual with the characteristics that the employee described with excitement is recommended. The input is the generated materials and the user's profile information, and the output is the recommended material information. Specifically, the server uploads the materials and executes the recommendation function.

[1309] (Application example 2)

[1310] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1311] Lack of work efficiency and standardization in traditional logistics centers is a problem. In particular, it takes time to train new workers and understand their work, and it is difficult to respond immediately to their emotional state. This invention is designed to solve these problems and improve work efficiency and the training process.

[1312] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1313] In this invention, the server includes means for installing an agent in a user's information processing device and collecting the contents of daily work and meetings, means for analyzing the collected data and extracting key points, means for automatically generating work guides, work processes, case studies, glossaries, and work handover documents based on the extracted key points, means for storing the generated materials on an information sharing site and providing users with a recommendation function, means for collecting and analyzing worker emotion data in real time, and means for recommending optimal work guides and training materials based on the collected emotion data, thereby making it possible to improve work efficiency and speed up the training process.

[1314] An "agent" is software that is installed on a user's information processing device and collects the contents of daily tasks and meetings.

[1315] An "information processing device" is an electronic device for collecting, analyzing, and communicating data.

[1316] "Daily work and meeting contents" is information about the work and meeting contents that the user performs on a daily basis.

[1317] "Analyzing data" refers to processing collected information to find useful patterns and takeaways.

[1318] "Gist extraction" refers to extracting important information or key content from the analyzed data.

[1319] A "work guide" is a procedure manual for performing specific tasks efficiently and accurately.

[1320] A "business process" is a series of work steps to achieve a specific goal.

[1321] A "case study collection" is a document that compiles past examples and case studies.

[1322] A "glossary" is a document that lists definitions of technical terms and terms.

[1323] A "business handover document" is a document that contains the information necessary when a business person in charge changes.

[1324] An "information sharing site" is a web platform that allows generated materials and information to be stored and shared.

[1325] The "recommended function" is a function that provides the most suitable materials and information to the user.

[1326] "Emotion data" is data that represents the emotional state of the user.

[1327] "Real-time collection" refers to obtaining data immediately.

[1328] "Analysis" refers to examining collected data in detail to find meaning and patterns.

[1329] "Recommending optimal work guides and educational materials according to the user's emotional state" refers to providing the most appropriate work instructions and educational materials in consideration of the user's emotional state.

[1330] This invention is a work support system using smart glasses that aims to improve work efficiency and standardize work in logistics centers.

[1331] Hardware and Software Configuration

[1332] Device:

[1333] The smart glasses have a built-in microphone to capture the worker's voice, allowing them to collect voice input in real time, and a built-in display that can display information on the screen.

[1334] server:

[1335] The server uses the following software to analyze the collected voice data:

[1336] 1. Speech Recognition: A speech recognition library for converting collected voice data into text.

[1337] 2. Transformers (Hugging Face's NLP library): Analyzes text data after speech recognition using natural language processing (NLP) to extract key points.

[1338] 3. Emotion Recognition: An emotion recognition library for recognizing the user's emotional state from collected voice data.

[1339] Specific processing flow:

[1340] 1. Collecting voice data: The microphone built into the smart glasses captures the voice of the worker and sends the data to the server.

[1341] 2. Transcription of voice data: The server converts the collected voice data into text using the Speech Recognition library.

[1342] 3. Gist Extraction: Use the Transformers NLP library on the text data to extract key points.

[1343] 4. Emotion Data Analysis: Use the Emotion Recognition library to analyze the worker's emotional state from the tone and content of their voice.

[1344] 5. Automatic generation of work guides and educational materials: Automatically generate appropriate work guides and educational materials based on key information and emotion data.

[1345] 6. Information display: The generated materials and recommended information are displayed in real time on the smart glasses display.

[1346] User:

[1347] By wearing the smart glasses, workers can receive feedback and guidance information tailored to their own working conditions, which is expected to improve work efficiency and enhance proficiency.

[1348] Specific examples

[1349] For example, if a worker picking a new product says, "I don't know how to pick this product," the voice is captured by the microphone in the smart glasses. The voice data is sent to the server, converted into text, and analyzed, and the optimal picking method and handling instructions for that product are displayed in real time on the worker's smart glasses.

[1350] Example prompts to input to the generative AI model:

[1351] "Please tell me how to analyze voice data and recognize emotions to provide appropriate feedback to workers who are feeling stressed while working, and provide work guidance in real time."

[1352] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1353] Step 1:

[1354] The microphone in the smart glasses captures the worker's voice.

[1355] Input: Worker's voice

[1356] How it works: A microphone built into the smart glasses captures the worker's voice and transmits the voice data to a server in real time.

[1357] Output: Audio data

[1358] Step 2:

[1359] The server converts the received voice data into text using the Speech Recognition library.

[1360] Input: Audio data

[1361] Operation: The server inputs the voice data into the Speech Recognition library and converts it into text data.

[1362] Output: Text data

[1363] Step 3:

[1364] The server uses the Transformers NLP library to parse the text data and extract key points.

[1365] Input: Text data

[1366] How it works: The server inputs text data into an NLP library and applies natural language processing algorithms to extract important key information and concepts.

[1367] Output: Key information

[1368] Step 4:

[1369] The server uses the Emotion Recognition library to recognize emotional states from text data.

[1370] Input: Text data

[1371] How it works: The server inputs the collected text data into the Emotion Recognition library, which analyzes the user's emotional state from the tone and content of the voice.

[1372] Output: Emotion data

[1373] Step 5:

[1374] The server automatically generates work guides and educational materials based on the key information and emotion data.

[1375] Input: Key points, emotion data

[1376] Operation: The server inputs key information and emotion data into the DocumentGenerator module, and automatically generates optimal work guides and educational materials based on that information.

[1377] Output: Work guides, educational materials

[1378] Step 6:

[1379] The generated materials and recommended information are displayed on the smart glasses display.

[1380] Input: Work guides, educational materials

[1381] How it works: The server sends the generated materials and recommended information to the smart glasses, which then display it on their screens. Workers can then perform their work while viewing the information displayed in real time.

[1382] Output: Information displayed on the smart glasses display

[1383] keyword:

[1384] Generative AI model, prompt sentence

[1385] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1386] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1387] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1388] [Fourth embodiment]

[1389] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1390] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1391] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1392] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1393] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1394] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1395] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1396] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1397] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1398] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1399] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1400] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1401] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1402] The present invention is a system for achieving standardization and efficiency in business operations. This system is mainly composed of the following means:

[1403] 1. Agent installation and data collection:

[1404] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. For example, the agent collects the content of a user's presentation about a new product.

[1405] 2. Data analysis and key points:

[1406] The server runs algorithms to analyze the collected data. The collected voice data is first converted into text data, and then natural language processing (NLP) algorithms are applied to extract key points. For example, key information such as "new product features" and "market strategy" can be identified from the presentation content.

[1407] 3. Automatic generation of materials:

[1408] Based on the key point extraction results, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents by embedding data into predefined templates. For example, a business manual describing the features of a new product can be automatically generated.

[1409] 4. Portal site management and recommendations:

[1410] The generated materials are stored on a company-wide portal site. The server recommends appropriate materials based on the user's profile, past work history, and search history. For example, when a new employee accesses the portal, they are recommended a new product manual or case study.

[1411] Program processing

[1412] Device:

[1413] The agent sends user operations and voice data to the server in real time. Users simply carry out their usual work or meetings without any special operations.

[1414] server:

[1415] The server first converts the audio data into text, then uses NLP to extract key points. Based on the analyzed key points, various documents are automatically generated using templates and uploaded to the portal site.

[1416] User:

[1417] Users can access the portal site and check the recommended materials to deepen their understanding of their work. For example, when assigned to a new job, they can instantly access the manuals and workflows they need.

[1418] Specific examples

[1419] When a user gives a presentation on a new product, the agent collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[1420] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload for users.

[1421] The processing flow will be explained below.

[1422] Step 1:

[1423] Installing and Configuring the Agent

[1424] server:

[1425] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[1426] Device:

[1427] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[1428] User:

[1429] The user confirms that the agent software is installed and configured.

[1430] Step 2:

[1431] Starting Data Collection

[1432] server:

[1433] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[1434] Device:

[1435] Based on specified conditions, the agent collects data on PC operations and audio data during meetings and sends it to the server in real time.

[1436] User:

[1437] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[1438] Step 3:

[1439] Transcription of audio data

[1440] server:

[1441] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[1442] Step 4:

[1443] Extracting key points using natural language processing (NLP)

[1444] server:

[1445] The server applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[1446] Step 5:

[1447] Analysis of operation log data

[1448] server:

[1449] The server analyzes the collected operation log data to identify frequently occurring operation patterns and business flows. This data will be used as the basis for later generation of documents.

[1450] Step 6:

[1451] Automatic generation of materials

[1452] server:

[1453] Based on the results of key point extraction and operation log analysis, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates. For example, information about the features of a new product can be automatically added to the business manual.

[1454] Step 7:

[1455] Preservation and management of generated materials

[1456] server:

[1457] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[1458] Step 8:

[1459] Uploading to the portal site and making recommendations

[1460] server:

[1461] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on the user profile and work history to provide the most suitable documents to specific users.

[1462] Device:

[1463] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[1464] User:

[1465] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[1466] By sequentially executing these steps, the present invention realizes automatic recording, analysis, automation, and efficiency of business operations.

[1467] Example 1

[1468] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1469] In conventional business processes, a great deal of effort was required to standardize and streamline operations. Manually recording meeting content and work procedures and then creating work manuals and workflows based on those records was particularly tedious work. New employees and transferees also faced the problem of being unable to quickly obtain the information they needed. A system that could automate and streamline a series of tasks was needed.

[1470] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1471] In this invention, the server includes means for installing agent software on a terminal and collecting the contents of daily work and meetings, means for periodically transmitting the collected operation logs and voice data to the server, means for converting the transmitted voice data into text data, means for extracting key points by applying a natural language processing algorithm, means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points, and means for storing the generated materials on a company-wide portal site and providing a recommendation function to users. This makes it possible to automate a series of processes from collecting work data to generating materials and making recommendations.

[1472] "Agent software" is software that is installed on the user's terminal and collects PC operations and conference audio in real time.

[1473] An "operation log" refers to the history of operations performed by a user on a PC, such as keyboard input, mouse clicks, and slide switching.

[1474] "Audio data" refers to data that records what a user says during a meeting or presentation.

[1475] A "server" is a computer system that receives data sent from a terminal, analyzes, converts, and processes the data to generate materials, and stores the materials on a portal site.

[1476] "Text data" refers to character information that has been converted by analyzing voice data.

[1477] "Natural language processing algorithm" is a general term for programs and methods for extracting key points from text data.

[1478] "Key points" refer to important information or keywords that are analyzed and extracted from the collected data.

[1479] A "business manual" refers to a document that describes business procedures and methods.

[1480] "Business flow" refers to a diagram of a series of business processes and procedures.

[1481] A "case study" refers to a document that compiles past business success stories and examples of implementation.

[1482] A "glossary" refers to a document that explains business-related terminology.

[1483] A "business handover document" refers to a document that contains the information necessary to ensure a smooth handover of business operations.

[1484] "Portal site" refers to a website where generated materials are stored and can be accessed and viewed by users.

[1485] The "recommendation function" refers to a function that suggests appropriate materials based on the user's work history and search behavior.

[1486] This invention is a system for standardizing and streamlining user operations. This system mainly includes the following components: agent software, a server, and a portal site. Each component will be described in detail below, along with its operation.

[1487] Agent software installation and data collection

[1488] Device:

[1489] First, agent software is installed on the user's device. This agent software collects in real time the user's PC operations (keyboard input, mouse clicks, switching slides, etc.) and the voices spoken during the meeting. For example, when a user gives a presentation about a new product, the agent software captures the operation log and voice data.

[1490] Data transmission and text conversion

[1491] Device:

[1492] The collected data is sent to the server at regular intervals (for example, every minute) or when an event occurs, allowing for real-time processing of the data.

[1493] server:

[1494] The server takes the received audio data and uses a speech recognition service such as the Google Cloud Speech-to-Text API to convert it into text data, which is then used in the next processing step.

[1495] Extracting key points using natural language processing

[1496] server:

[1497] The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract key information (key points). For example, keywords and phrases such as "new product features" and "market strategy" can be extracted from the text of a new product presentation.

[1498] Automatic generation of materials

[1499] server:

[1500] Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates. For example, a business manual describing the features and market strategy of a new product can be generated.

[1501] Management and recommendations on the portal site

[1502] server:

[1503] The generated materials are stored on a company-wide portal site. The server recommends the most appropriate materials based on the user's work history and search behavior. For example, when a new employee accesses the portal site, a new product manual and case studies are displayed on the top page.

[1504] User:

[1505] Users can deepen their understanding of the business by accessing the portal site and checking the recommended materials, which will enable them to respond quickly to new business tasks.

[1506] Specific examples

[1507] When a user gives a presentation about a new product, the agent software collects the voice content and PC operations (displaying materials, switching slides, etc.). The server then converts the voice data into text and uses NLP to extract key points such as the "features of the new product" and "target market." An operations manual is automatically generated based on the extracted key points and uploaded to the portal site. When new employees or those who have been transferred use the portal site, the new product's operations manual is recommended, allowing them to immediately understand the work.

[1508] Example prompts for generative AI models

[1509] 1. "Please summarize the main points of this presentation. The key themes are the features of the new product and the market strategy."

[1510] 2. "Please generate an operations manual from the following text data. Use Microsoft Word as the template."

[1511] 3. "Please extract keywords from the audio data of this conference and upload related resources to the portal site."

[1512] This system automates a series of processes, from collecting business data to generating documents and making recommendations, significantly reducing the workload of users.

[1513] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1514] Step 1:

[1515] Agent software installation and data collection

[1516] Terminal: Agent software is installed on the user's terminal. The agent software collects in real time the operations performed by the user on the PC (keyboard input, mouse clicks, switching slides, etc.) and the voice uttered during the meeting. The collected data is saved as an operation log and voice data.

[1517] Input: User's PC operation, conference audio

[1518] Output: Operation log, audio data

[1519] Specific operation: Once the agent software is installed and the user begins to operate the device, the operation log and recorded audio are captured in real time and temporarily stored on the device.

[1520] Step 2:

[1521] Data transmission and text conversion

[1522] Terminal: The collected operation log and voice data are sent to the server periodically (for example, every minute).

[1523] Input: Operation log, audio data

[1524] Output: Operation log and audio data sent to the server

[1525] Specific operation: The terminal uploads all the data collected by the agent software to the server at regular intervals or at specific events (e.g., the end of a meeting).

[1526] Server: The server converts the received voice data into text data using the Google Cloud Speech-to-Text API.

[1527] Input: Audio data

[1528] Output: Text data

[1529] Specific operation: The server sequentially analyzes the received audio files, calls the speech recognition API to convert the audio into text, and passes the result to the next processing step.

[1530] Step 3:

[1531] Extracting key points using natural language processing

[1532] Server: The server applies natural language processing (NLP) algorithms such as spaCy and BERT to the text data to extract important information and key points.

[1533] Input: Text data

[1534] Output: Key information

[1535] Specific operations: Analyze text data using NLP tools to extract important keywords and phrases such as "new product features" and "market strategy."

[1536] Step 4:

[1537] Automatic generation of materials

[1538] Server: Based on the extracted key information, the server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents using Microsoft Word templates.

[1539] Input: Key information

[1540] Output: Business manuals, business flow, case studies, glossary, business handover documents

[1541] Specific operation: The server embeds key information into a pre-prepared template and generates a document file in a format such as "New Product Manual.docx."

[1542] Step 5:

[1543] Management and recommendations on the portal site

[1544] Server: The generated documents are stored on the company's portal site. The server recommends appropriate documents based on the user's work history and search behavior.

[1545] Input: Generated materials, user work history, search behavior

[1546] Output: Recommended materials

[1547] Specific operation: Documents are uploaded to the portal site, and when a user accesses it, an algorithm is run based on the user's previous work history and search behavior, and documents appropriate for the user are displayed on the top page.

[1548] Step 6:

[1549] Check the materials

[1550] User: The user accesses the portal site and checks the recommended materials, which allows them to deepen their understanding of the business.

[1551] Input: Recommended materials

[1552] Output: Understanding business operations by referring to materials

[1553] Specific operation: A user logs in to the portal site, clicks on the link to recommended materials displayed on the top page, and views the manuals and flow charts.

[1554] (Application example 1)

[1555] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1556] At logistics centers, product handling and picking route procedures change frequently, making it time-consuming to create instructions and manuals to accommodate these changes. In addition, real-time, reliable information provision is required so that workers can immediately understand new instructions and procedures and work efficiently. Conventional methods lack the means to efficiently resolve these issues, making it difficult to streamline and standardize operations.

[1557] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1558] In this invention, the server includes: means for installing an agent on a user's computer and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating work manuals, work flows, case studies, glossaries, and work handover documents based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means including a smartphone agent application for collecting voice data and operation data in real time and sending it to the server; means for analyzing the collected data on the server using voice recognition and natural language processing algorithms and extracting important information; means for automatically generating efficient work instructions based on the extracted key points and notifying the smartphone in real time; and means for providing a portal site that recommends materials based on the user's work history and profile. This enables fast and accurate information provision and work standardization.

[1559] An "agent" is a program that is installed on a user's computer or smartphone and collects information about daily work and meetings.

[1560] "Audio data" refers to data that records the voice generated during a user's work or a meeting.

[1561] "Operation data" refers to data that records the operations performed by a user on a computer or smartphone.

[1562] A "server" is a device or system that has the function of analyzing collected data, extracting key points, and automatically generating and managing materials.

[1563] A "natural language processing algorithm" is an analytical method for extracting key points from text data such as voice data.

[1564] A "business manual" is a guide that describes the procedures and methods for a specific business operation.

[1565] A "business flow" is a diagram or text that shows the flow or process of a business.

[1566] A "case study collection" is a collection of specific past business cases that have been collected and compiled.

[1567] A "glossary" is a document that collects technical terms and their definitions related to a particular field or task.

[1568] A "business handover document" is a document that describes the handover details when a business person in charge is changed.

[1569] A "portal site" is a website that stores generated materials and provides users with a recommendation function.

[1570] The "agent application for smartphones" is software that is installed on a smartphone and collects voice data and operation data in real time and transmits them to a server.

[1571] "Real-time notification" is a function that instantly sends generated work instructions to a smartphone.

[1572] "Key point extraction" is the process of extracting important information or points from collected data.

[1573] The "recommendation function" is a function that recommends the most appropriate materials and information based on the user's work history and profile.

[1574] This invention provides a system that includes a smartphone application and back-end system used in logistics centers and other locations. Smartphones with installed agents collect the daily work details of workers and send them to a server, thereby generating and notifying work instructions in real time. Below, we will explain each component of the system and its processing.

[1575] Terminal part (smartphone)

[1576] Hardware / software used: Smartphone, Android OS, iOS, agent app

[1577] Voice data collection: The voice of the worker is collected in real time using the microphone on the smartphone. For example, when a worker explains how to handle a new product, the voice data is collected.

[1578] Operation data collection: Includes a screen capture function for recording screen operations. For example, when operating a screen to confirm product picking instructions, that operation data is collected.

[1579] Server part

[1580] Hardware / software used: Cloud servers (AWS, Google Cloud, etc.), speech recognition APIs (Google Speech-to-Text API, etc.), NLP libraries (SpaCy, NLTK)

[1581] Speech-to-text conversion: Converts collected voice data into text data. For example, a voice explaining a new product is converted into text data.

[1582] Key point extraction: Using NLP algorithms to extract important information from text data, such as the features and handling precautions of a new product.

[1583] Automatic generation of work instructions: Based on the extracted key points, work instructions are automatically generated using templates. For example, a handling procedure manual for a new product is generated.

[1584] Uploading materials: The generated materials are uploaded to a portal site and recommendations are made based on the user profile. For example, when a worker accesses the portal site, a handling manual for a new product is recommended.

[1585] Real-time notification section

[1586] Hardware / software used: Smartphone, push notification API (Firebase Cloud Messaging, etc.)

[1587] Notifications: Generated work instructions are sent via push notification in real time. For example, the handling procedure for a new product is immediately sent to the worker's smartphone.

[1588] Examples and prompts

[1589] Examples:

[1590] Scenario: A presentation is being given to explain how to handle a new product.

[1591] Data collection: The agent app collects the audio of the presentation.

[1592] Data analysis: The audio is converted into text and the key point, "How to handle the new product," is extracted.

[1593] Instruction generation: Generate handling instructions for new products based on the extracted key points.

[1594] Notification: The procedure manual is pushed to the picking worker's smartphone.

[1595] Example prompt for a generative AI model:

[1596] Analyze the audio recording about how to handle new products and extract the following information:

[1597] 1. New product handling procedures

[1598] 2. Materials and Equipment Needed

[1599] 3. Points to note

[1600] Based on the extracted information, please automatically generate a work instruction in the following format:

[1601] 1. Product name

[1602] 2. Handling Procedures

[1603] 3. Required Materials

[1604] 4. Points to note

[1605] Please notify the smartphone app of the generated work instructions in the following format:

[1606] 1. Title: New Product Handling Instructions

[1607] 2. Content: Extracted work instructions

[1608] This system, configured in this way, will enable the efficiency and standardization of operations at logistics centers, and by providing appropriate instructions to workers in real time, it is expected to reduce errors and improve overall work efficiency.

[1609] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1610] Step 1:

[1611] The agent app on the smartphone collects the worker's voice data and operation data in real time. The voice data is recorded through the smartphone's microphone, and the operation data is recorded using the screen capture function. The voice and operation data are collected as input and sent to the server for later analysis.

[1612] Step 2:

[1613] The collected voice data and operation data are sent to the server. The server receives this data and first converts the voice data into text using a voice recognition API (e.g., Google Speech-to-Text API). The input is the voice data, and the output is the converted text data. Specifically, voice data is given as input, and the voice recognition API outputs text data.

[1614] Step 3:

[1615] The server applies a natural language processing (NLP) algorithm (e.g., SpaCy, NLTK) to the converted text data to extract key points. The input is text data obtained by speech recognition, and the extracted results containing important information (key points) are output. Specifically, text data is given as input, and the NLP algorithm outputs key points such as "features of the new product" and "how to use it."

[1616] Step 4:

[1617] The server automatically generates work instructions based on the key point extraction results. The input is the extracted key point information, and a document containing the work instructions (e.g., handling procedures for a new product) is generated using a template. Specifically, the key point information is embedded in the template, and the generated work instructions are output.

[1618] Step 5:

[1619] The server uploads the generated work instructions to a portal site and recommends related materials based on the user's profile and work history. The input is the generated work instructions and user profile, and the portal site's recommendation information is output. Specifically, the work instructions are stored on the portal site, and appropriate materials are displayed according to the user.

[1620] Step 6:

[1621] The generated work instructions are sent in real time using a smartphone push notification API (e.g., Firebase Cloud Messaging). The input is the work instructions and information about the recipient smartphone, and the output is a notification message sent to the smartphone. Specifically, the work instructions are sent as a push notification message and are instantly displayed on the worker's smartphone.

[1622] Step 7:

[1623] The user (worker) checks the work instructions notified on their smartphone and carries out the work. The input is the work instructions displayed on the smartphone, and the output is the work procedure to be carried out based on them. Specifically, the user taps the notification to open the work instructions and carries out the work according to the instructed procedure.

[1624] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1625] This invention combines an emotion engine with a system that promotes the standardization and efficiency of users' work. This system is composed of the following means:

[1626] 1. Install and configure the agent:

[1627] The server installs agent software on the user's device. This agent collects PC operations and audio during meetings in real time. When a user gives a presentation about a new product, the agent collects the content.

[1628] 2. Data collection and analysis:

[1629] The server analyzes the collected voice data and operation data. The voice data is converted into text data using a transcription tool, and key points are extracted using a natural language processing (NLP) algorithm. An emotion engine then recognizes the user's emotions from the voice and operation data. For example, not only is important information such as "new product features" and "market strategy" identified, but the emotions (e.g., excitement, stress) expressed when the user describes that information are also recognized.

[1630] 3. Automatic generation of materials:

[1631] The server automatically generates a business manual, business flow, case studies, glossary, and business handover document based on the extracted key information and emotion data. For example, a business manual may be generated that particularly emphasizes the features of a new product that the user excitedly explained.

[1632] 4. Portal site management and recommendations:

[1633] The generated materials are stored on a company-wide portal site. The server recommends the most suitable materials based on each user's profile, work history, search history, and even emotional data. For example, when a new employee accesses the portal site, they are presented with an interesting work manual recommended based on their emotional data.

[1634] Program processing

[1635] Device:

[1636] The agent collects user operations and voice data in real time and sends it to the server. It also acquires the user's emotional data using an emotion engine and sends this information to the server. The user simply needs to carry out their normal work or meetings.

[1637] server:

[1638] 1. Transcription of audio data:

[1639] The server converts the collected voice data into text data, using a transcription tool to accurately convert the voice data into text format.

[1640] 2. Key Points Extraction Using Natural Language Processing (NLP):

[1641] The server applies NLP algorithms to the text data to extract key points and keywords, such as "new product features" and "target market."

[1642] 3. Emotional Data Analysis:

[1643] The server analyzes the collected emotion data using the emotion engine to detect the user's emotional state (e.g., stress level and motivation) from the tone of voice and operation logs.

[1644] 4. Automatic generation of materials:

[1645] The server combines key information and emotion data to automatically generate business manuals, workflows, case studies, glossaries, and business handover documents. The generated materials reflect the user's emotions.

[1646] 5. Portal site management and recommendations:

[1647] The server uploads the generated documents to a company-wide portal site and organizes them by category. It also recommends the most suitable documents based on user profiles and emotional data. It also notifies users of the access information for the documents.

[1648] User:

[1649] Users receive notifications and access recommended materials from the portal site, which allows them to efficiently learn and deepen their understanding of their work.

[1650] Specific examples

[1651] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server then converts the voice data into text and uses NLP to extract key points. The emotion engine also analyzes whether the user is excited. Based on the extracted key points and emotional data, a business manual is automatically generated with sections that need to be emphasized highlighted. When new employees access the portal site, this business manual is recommended, and information is provided in a fun and easy-to-learn format.

[1652] In this way, this system supports the standardization and efficiency of business operations, and by utilizing user emotion data, it is possible to provide more appropriate and effective content.

[1653] The processing flow will be explained below.

[1654] Step 1:

[1655] Installing and Configuring the Agent

[1656] server:

[1657] The server issues instructions to the user's device to install the agent software. The server then sends a configuration file to the device, completing the agent software setup.

[1658] Device:

[1659] The terminal receives the agent software and completes the installation. The agent is then launched and sends a connection request to the server.

[1660] User:

[1661] The user confirms that the agent software is installed and configured.

[1662] Step 2:

[1663] Starting Data Collection

[1664] server:

[1665] The server issues a command to the agent to start collecting data, and sets trigger conditions for which operations and sounds will be collected.

[1666] Device:

[1667] The agent collects data on PC operations and voice data during meetings based on specified conditions and sends it to the server in real time. The emotion engine also analyzes the user's voice tone and operation patterns.

[1668] User:

[1669] Users simply go about their normal business and meetings, and the agent automatically collects the data.

[1670] Step 3:

[1671] Transcription of audio data

[1672] server:

[1673] The server sends the collected voice data to a transcription tool, converts the voice data into text data, and temporarily stores the converted text data.

[1674] Step 4:

[1675] Extracting key points using natural language processing (NLP)

[1676] server:

[1677] The server then applies natural language processing (NLP) algorithms to the transcribed text data to extract key points and keywords, such as "new product features" and "market strategy."

[1678] Step 5:

[1679] Emotional Data Analysis

[1680] server:

[1681] The server analyzes the voice tones and operation patterns collected by the emotion engine to determine the user's emotional state. For example, it detects whether the user is excited when explaining a new product. The emotional data is stored together with other business data.

[1682] Step 6:

[1683] Automatic generation of materials

[1684] server:

[1685] The server automatically generates business manuals, business flows, case studies, glossaries, and business handover documents based on templates, using the key point extraction results and emotion data. Materials are created in a way that emphasizes parts that strongly reflect the user's emotions (for example, the features of a new product that was explained in an excited state).

[1686] Step 7:

[1687] Preservation and management of generated materials

[1688] server:

[1689] The server stores the generated material in a specific directory and puts it under version control, which makes it easy to update or modify it in the future.

[1690] Step 8:

[1691] Uploading to the portal site and making recommendations

[1692] server:

[1693] The server uploads the generated documents to a company-wide portal site. The documents are organized by category and set up for easy user access. Furthermore, a recommendation algorithm is applied based on user profiles and emotional data to provide the most suitable documents to specific users.

[1694] Device:

[1695] The terminal sends a notification to the user about the recommended material, including the type of material, a summary, and an access link.

[1696] User:

[1697] Users receive notifications and access recommended materials from the portal site to deepen their understanding of their work.

[1698] By executing each step in this way, the present invention realizes automatic recording, analysis, document generation, and efficiency improvement of business operations, and further utilizes emotion data to provide more appropriate and effective content.

[1699] Example 2

[1700] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1701] Conventional business management systems lack the functionality to efficiently collect users' daily work and meeting details, extract key points, and automatically generate materials based on them. Furthermore, materials are not created taking into account the emotional state of the user, making it difficult to achieve standardized and efficient business processes. Furthermore, they lack the functionality to optimally recommend generated materials, which makes it difficult to provide appropriate information to specific users.

[1702] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes: means for installing an agent in a user's information device and collecting the contents of daily work and meetings; means for analyzing the collected data and extracting key points; means for automatically generating a work manual, work flow, case studies, glossary, and work handover document based on the extracted key points; means for storing the generated materials on a company-wide portal site and providing a recommendation function to users; means for converting collected voice data into text data using a transcription tool; means for recognizing and analyzing the user's emotional state from the voice data and operation data using an emotion engine; and means for automatically generating materials by combining key point information and emotion data. This enables standardization and efficiency of work and the provision of information that takes emotion data into consideration.

[1703] An "agent" is software that is installed on a user's information device and collects the contents of daily tasks and meetings.

[1704] "Information devices" is a general term for electronic devices such as computers, tablets, and smartphones that users use for work and meetings.

[1705] An "emotion engine" is software or algorithms that recognize and analyze a user's emotional state from voice data and operation data.

[1706] "Transcription tool" is a general term for software and services that convert audio data into text data.

[1707] "Natural language processing algorithm" is a general term for algorithms used to extract important points and keywords from text data.

[1708] A "company-wide portal site" is a web-based platform that users can access for sharing and managing information within a company.

[1709] The "recommendation function" is a function that suggests the most suitable materials and information based on the user's profile and behavioral data.

[1710] A "business manual" is a document that contains procedures and guidelines for carrying out a specific business or process.

[1711] A "business flow" is a diagram or table that visually represents the steps and flow of a specific business or process.

[1712] A "case study collection" is a document or data that collects and organizes past cases and case studies.

[1713] A "glossary" is a list of technical terms and definitions related to a particular field or task.

[1714] A "business handover document" is a document that lists the necessary information and procedures to be used when handing over business operations.

[1715] The present invention combines an emotion engine with a system that promotes the standardization and efficiency of user work. Specific embodiments for implementing this system are described below.

[1716] This system is mainly composed of three entities: a server, a terminal, and a user.

[1717] Device:

[1718] 1. Install and configure the agent:

[1719] The server installs agent software on the user's information device. This agent collects in real time the operations performed by the user on a PC or tablet, as well as audio data during a meeting. For example, when a user gives a presentation about a new product, the agent collects detailed information about the presentation.

[1720] 2. Data collection and transmission:

[1721] The terminal agent collects user operation data and conference audio data and sends them to the server. The emotion engine also collects the user's emotion data (e.g., excitement or stress) and sends this data to the server. The user simply carries out their normal work, and data collection is carried out in the background.

[1722] server:

[1723] 1. Transcription of audio data:

[1724] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically analyzed and the dialogue is recorded in text format. For example, a log stating "Voice data text conversion completed" is generated.

[1725] 2. Key Points Extraction Using Natural Language Processing (NLP):

[1726] The server applies an NLP algorithm (e.g., spaCy) to the text data to extract key points and keywords. For example, keywords such as "new product features" and "market strategy" are identified. At this step, the "extracted key points data" are passed on to the next step of processing.

[1727] 3. Emotional Data Analysis:

[1728] The server uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the collected emotion data. It detects the user's emotional state (e.g., excitement, stress, etc.) from the tone of the voice and operation logs. For example, if a user becomes excited during a presentation, this state is recorded as "user is excited."

[1729] 4. Automatic generation of materials:

[1730] The server combines key information and emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that highlights the features of a new product that a user was particularly excited about. Materials are generated using a template engine (e.g., Jinja2).

[1731] 5. Portal site management and recommendations:

[1732] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, they receive a notification that a "business manual highlighting the features of a new product" is displayed.

[1733] User:

[1734] Users can receive notifications and access recommended materials from the portal site, allowing them to efficiently learn about new job content and deepen their understanding.

[1735] Specific examples

[1736] When a user gives a presentation about a new product, the agent collects the content of the presentation (voice data and operation data) and the user's emotional data. The server converts the collected voice data into text and extracts the main points using an NLP algorithm. The emotion engine also analyzes whether the user is excited.

[1737] For example, if the system detects that the user is excited when explaining the features of a new product, the system will automatically generate an operational manual that emphasizes those features.When a new employee accesses the portal site, the system will present them with an operational manual that is fun and easy to learn, recommended based on emotional data.

[1738] Prompt Sentence Examples

[1739] Prompt statement:

[1740] Collect voice and operation data from users giving a presentation on a new product, extract key points using an NLP algorithm, and analyze the user's level of excitement using an emotion engine. Then, automatically generate an operations manual that highlights the parts that caused the user to feel particularly excited.

[1741] This prompt sentence is used to instruct the generative AI model, ensuring efficient operation of the system.

[1742] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1743] Step 1:

[1744] Installing and Configuring the Agent

[1745] The server installs agent software on the user's device. This agent collects in real time the operations and voice data performed on the device, such as a PC or tablet. For example, when a user gives a presentation about a new product, the agent is ready to collect the content of the presentation. The input is the agent software, and the output is a notification that the installation is complete. Specifically, the agent is installed and initial settings are performed.

[1746] Step 2:

[1747] Data collection and transmission

[1748] The agent on the terminal collects user operation data and voice data during the conference in real time and sends this data to the server. For example, when a user starts a presentation, the content of the presentation is collected. Emotional data (excitement and stress) is also collected at the same time. The input is voice data and operation data, and the output is the data sent to the server. Specifically, the agent records the voice during the conference and keeps an operation log.

[1749] Step 3:

[1750] Transcription of audio data

[1751] The server converts the collected voice data into text data using a transcription tool (e.g., Google Cloud Speech-to-Text). During this process, the voice data is automatically converted into text format. The input is the voice data sent to the server, and the output is the converted text data. Specifically, the transcription tool analyzes the voice data and outputs the dialogue as text.

[1752] Step 4:

[1753] Extracting key points using natural language processing (NLP)

[1754] The server analyzes the text data converted by the transcription tool using an NLP algorithm (e.g., spaCy) to extract important key points and keywords. For example, key points such as "new product features" and "market strategy" are identified. The converted text data is the input, and the extracted key point data is the output. Specifically, the NLP algorithm automatically analyzes and extracts keywords and key points within the text.

[1755] Step 5:

[1756] Emotional Data Analysis

[1757] The server analyzes the collected emotion data using an emotion engine (e.g., IBM Watson Tone Analyzer) and recognizes the user's emotional state (e.g., excitement, stress, etc.). The input is emotion data including voice tone and operation logs, and the output is analyzed emotional state information. Specifically, the emotion engine analyzes the voice tone and detects the user's emotional state.

[1758] Step 6:

[1759] Automatic generation of materials

[1760] The server combines key information extracted by NLP with emotion data to automatically generate business manuals, flow diagrams, case studies, glossaries, and business handover documents. For example, a business manual may be created that emphasizes the features of a new product that a user described with particular enthusiasm. The input is the extracted key information and analyzed emotion data, and the output is automatically generated materials. Specifically, various materials are generated using a template engine (for example, Jinja2).

[1761] Step 7:

[1762] Management and recommendations on the portal site

[1763] The server uploads the generated materials to a company-wide portal site (e.g., SharePoint) and organizes them by category. It also recommends the most appropriate materials based on user profiles and emotional data. For example, when a new employee accesses the portal site, a business manual with the characteristics that the employee described with excitement is recommended. The input is the generated materials and the user's profile information, and the output is the recommended material information. Specifically, the server uploads the materials and executes the recommendation function.

[1764] (Application example 2)

[1765] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1766] Lack of work efficiency and standardization in traditional logistics centers is a problem. In particular, it takes time to train new workers and understand their work, and it is difficult to respond immediately to their emotional state. This invention is designed to solve these problems and improve work efficiency and the training process.

[1767] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1768] In this invention, the server includes means for installing an agent in a user's information processing device and collecting the contents of daily work and meetings, means for analyzing the collected data and extracting key points, means for automatically generating work guides, work processes, case studies, glossaries, and work handover documents based on the extracted key points, means for storing the generated materials on an information sharing site and providing users with a recommendation function, means for collecting and analyzing worker emotion data in real time, and means for recommending optimal work guides and training materials based on the collected emotion data, thereby making it possible to improve work efficiency and speed up the training process.

[1769] An "agent" is software that is installed on a user's information processing device and collects the contents of daily tasks and meetings.

[1770] An "information processing device" is an electronic device for collecting, analyzing, and communicating data.

[1771] "Daily work and meeting contents" is information about the work and meeting contents that the user performs on a daily basis.

[1772] "Analyzing data" refers to processing collected information to find useful patterns and takeaways.

[1773] "Gist extraction" refers to extracting important information or key content from the analyzed data.

[1774] A "work guide" is a procedure manual for performing specific tasks efficiently and accurately.

[1775] A "business process" is a series of work steps to achieve a specific goal.

[1776] A "case study collection" is a document that compiles past examples and case studies.

[1777] A "glossary" is a document that lists definitions of technical terms and terms.

[1778] A "business handover document" is a document that contains the information necessary when a business person in charge changes.

[1779] An "information sharing site" is a web platform that allows generated materials and information to be stored and shared.

[1780] The "recommended function" is a function that provides the most suitable materials and information to the user.

[1781] "Emotion data" is data that represents the emotional state of the user.

[1782] "Real-time collection" refers to obtaining data immediately.

[1783] "Analysis" refers to examining collected data in detail to find meaning and patterns.

[1784] "Recommending optimal work guides and educational materials according to the user's emotional state" refers to providing the most appropriate work instructions and educational materials in consideration of the user's emotional state.

[1785] This invention is a work support system using smart glasses that aims to improve work efficiency and standardize work in logistics centers.

[1786] Hardware and Software Configuration

[1787] Device:

[1788] The smart glasses have a built-in microphone to capture the worker's voice, allowing them to collect voice input in real time, and a built-in display that can display information on the screen.

[1789] server:

[1790] The server uses the following software to analyze the collected voice data:

[1791] 1. Speech Recognition: A speech recognition library for converting collected voice data into text.

[1792] 2. Transformers (Hugging Face's NLP library): Analyzes text data after speech recognition using natural language processing (NLP) to extract key points.

[1793] 3. Emotion Recognition: An emotion recognition library for recognizing the user's emotional state from collected voice data.

[1794] Specific processing flow:

[1795] 1. Collecting voice data: The microphone built into the smart glasses captures the voice of the worker and sends the data to the server.

[1796] 2. Transcription of voice data: The server converts the collected voice data into text using the Speech Recognition library.

[1797] 3. Gist Extraction: Use the Transformers NLP library on the text data to extract key points.

[1798] 4. Emotion Data Analysis: Use the Emotion Recognition library to analyze the worker's emotional state from the tone and content of their voice.

[1799] 5. Automatic generation of work guides and educational materials: Automatically generate appropriate work guides and educational materials based on key information and emotion data.

[1800] 6. Information display: The generated materials and recommended information are displayed in real time on the smart glasses display.

[1801] User:

[1802] By wearing the smart glasses, workers can receive feedback and guidance information tailored to their own working conditions, which is expected to improve work efficiency and enhance proficiency.

[1803] Specific examples

[1804] For example, if a worker picking a new product says, "I don't know how to pick this product," the voice is captured by the microphone in the smart glasses. The voice data is sent to the server, converted into text, and analyzed, and the optimal picking method and handling instructions for that product are displayed in real time on the worker's smart glasses.

[1805] Example prompts to input to the generative AI model:

[1806] "Please tell me how to analyze voice data and recognize emotions to provide appropriate feedback to workers who are feeling stressed while working, and provide work guidance in real time."

[1807] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1808] Step 1:

[1809] The microphone in the smart glasses captures the worker's voice.

[1810] Input: Worker's voice

[1811] How it works: A microphone built into the smart glasses captures the worker's voice and transmits the voice data to a server in real time.

[1812] Output: Audio data

[1813] Step 2:

[1814] The server converts the received voice data into text using the Speech Recognition library.

[1815] Input: Audio data

[1816] Operation: The server inputs the voice data into the Speech Recognition library and converts it into text data.

[1817] Output: Text data

[1818] Step 3:

[1819] The server uses the Transformers NLP library to parse the text data and extract key points.

[1820] Input: Text data

[1821] How it works: The server inputs text data into an NLP library and applies natural language processing algorithms to extract important key information and concepts.

[1822] Output: Key information

[1823] Step 4:

[1824] The server uses the Emotion Recognition library to recognize emotional states from text data.

[1825] Input: Text data

[1826] How it works: The server inputs the collected text data into the Emotion Recognition library, which analyzes the user's emotional state from the tone and content of the voice.

[1827] Output: Emotion data

[1828] Step 5:

[1829] The server automatically generates work guides and educational materials based on the key information and emotion data.

[1830] Input: Key points, emotion data

[1831] Operation: The server inputs key information and emotion data into the DocumentGenerator module, and automatically generates optimal work guides and educational materials based on that information.

[1832] Output: Work guides, educational materials

[1833] Step 6:

[1834] The generated materials and recommended information are displayed on the smart glasses display.

[1835] Input: Work guides, educational materials

[1836] How it works: The server sends the generated materials and recommended information to the smart glasses, which then display it on their screens. Workers can then perform their work while viewing the information displayed in real time.

[1837] Output: Information displayed on the smart glasses display

[1838] keyword:

[1839] Generative AI model, prompt sentence

[1840] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[1841] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1842] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[1843] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1844] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[1845] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[1846] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[1847] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[1848] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[1849] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[1850] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[1851] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[1852] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[1853] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1854] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[1855] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[1856] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[1857] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[1858] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[1859] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[1860] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[1861] The following is further disclosed regarding the above embodiment.

[1862] (Claim 1)

[1863] A means for installing an agent on a user's computer and collecting the contents of daily tasks and meetings;

[1864] A means of analyzing the collected data and extracting key points;

[1865] A means to automatically generate business manuals, business flows, case studies, glossaries, and business handover documents based on the extracted key points;

[1866] A means of storing the generated materials on a company-wide portal site and providing users with a recommendation function;

[1867] A system including:

[1868] (Claim 2)

[1869] 10. The system of claim 1, further comprising means for recommending optimal materials based on a user's work history and search behavior.

[1870] (Claim 3)

[1871] 10. The system of claim 1, further comprising means for converting the collected voice data into text data and applying a natural language processing algorithm to extract key points.

[1872] "Example 1"

[1873] (Claim 1)

[1874] A means for installing agent software on a user's terminal and collecting the contents of daily work and meetings;

[1875] a means for periodically transmitting the collected operation log and voice data to a server;

[1876] means for converting the transmitted voice data into text data;

[1877] means for applying natural language processing algorithms to extract key points;

[1878] A means to automatically generate business manuals, business flows, case studies, glossaries, and business handover documents based on the extracted key points;

[1879] A means of storing the generated materials on a company-wide portal site and providing users with a recommendation function;

[1880] A system including:

[1881] (Claim 2)

[1882] 10. The system of claim 1, further comprising means for recommending optimal materials based on a user's work history and search behavior.

[1883] (Claim 3)

[1884] 10. The system of claim 1, further comprising means for automatically generating materials using templates based on gist information extracted by a natural language processing algorithm.

[1885] "Application Example 1"

[1886] (Claim 1)

[1887] A means for installing an agent on a user's computer and collecting the contents of daily tasks and meetings;

[1888] A means of analyzing the collected data and extracting key points;

[1889] A means to automatically generate business manuals, business flows, case studies, glossaries, and business handover documents based on the extracted key points;

[1890] A means of storing the generated materials on a company-wide portal site and providing users with a recommendation function;

[1891] a means including an agent application for a smartphone that collects voice data and operation data in real time and transmits the collected data to a server;

[1892] A means for analyzing the collected data on a server using speech recognition and natural language processing algorithms to extract important information;

[1893] A method to automatically generate efficient work instructions based on the extracted key points and notify them to smartphones in real time.

[1894] A means for providing a portal site that recommends materials based on a user's work history and profile;

[1895] A system including:

[1896] (Claim 2)

[1897] 10. The system of claim 1, further comprising means for recommending optimal materials based on a user's work history and search behavior.

[1898] (Claim 3)

[1899] 10. The system of claim 1, further comprising means for converting the collected voice data into text data and applying a natural language processing algorithm to extract key points.

[1900] "Example 2: Combining Emotion Engines"

[1901] (Claim 1)

[1902] A means for installing an agent on a user's information device and collecting the contents of daily business and meetings;

[1903] A means of analyzing the collected data and extracting key points;

[1904] A means to automatically generate business manuals, business flows, case studies, glossaries, and business handover documents based on the extracted key points;

[1905] A means of storing the generated materials on a company-wide portal site and providing users with a recommendation function;

[1906] A means of converting the collected voice data into text data using a transcription tool;

[1907] a means for recognizing and analyzing the emotional state of a user from voice data and operation data using an emotion engine;

[1908] A means for automatically generating materials by combining key information and emotion data;

[1909] A system including:

[1910] (Claim 2)

[1911] 10. The system of claim 1, further comprising means for recommending optimal materials based on a user's work history and search behavior.

[1912] (Claim 3)

[1913] 10. The system of claim 1, further comprising means for converting the collected voice data into text data and applying a natural language processing algorithm to extract key points.

[1914] "Application example 2 when combining emotion engines"

[1915] (Claim 1)

[1916] a means for installing an agent in a user's information processing device and collecting the contents of daily tasks and meetings;

[1917] A means of analyzing the collected data and extracting key points;

[1918] A means to automatically generate business guides, business processes, case studies, glossaries, and business handover documents based on the extracted key points;

[1919] A means for storing the generated materials on an information sharing site and providing a recommendation function to users;

[1920] A means for collecting and analyzing worker emotion data in real time;

[1921] A method for recommending optimal work guides and educational materials based on the collected emotional data according to the user's emotional state;

[1922] A system including:

[1923] (Claim 2)

[1924] 10. The system of claim 1, further comprising means for recommending optimal materials based on a user's work history and information search behavior.

[1925] (Claim 3)

[1926] 10. The system of claim 1, further comprising: means for converting the collected voice data into text data and applying a natural language processing algorithm to extract key points; and means for applying an emotion engine to analyze the user's emotion data. [Explanation of symbols]

[1927] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for installing an agent on a user's computer and collecting the contents of daily tasks and meetings; A means of analyzing the collected data and extracting key points; A means to automatically generate business manuals, business flows, case studies, glossaries, and business handover documents based on the extracted key points; A means of storing the generated materials on a company-wide portal site and providing users with a recommendation function; A system including:

2. 10. The system of claim 1, further comprising means for recommending optimal materials based on a user's work history and search behavior.

3. 10. The system of claim 1, further comprising means for converting the collected speech data into text data and applying a natural language processing algorithm to extract key points.

Images