system

Abstract

Provide a system. 【Solution means】Means for converting voice data into text, means for extracting important information from the text and generating a meeting record, means for automatically extracting tasks from the meeting record and adding them to the management system, means for checking the language and format of documents in real time, means for analyzing the content of emails and documents and generating a summary, means for detecting technical terms from the analyzed content and providing explanations, means for checking the schedules of participants and proposing an optimal meeting date and time, means for notifying participants of the optimal meeting date and time, A system including the above.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In business activities, problems include that a lot of time and labor are spent on creating meeting minutes, task management, document creation, summarizing the content of emails and documents, explaining technical terms, and meeting coordination. These tasks are often performed manually, resulting in a high probability of errors and a decrease in work efficiency. Also, information is dispersed by using multiple tools and platforms, making unified management difficult.

Means for Solving the Problems

[0005] The present invention significantly improves work efficiency through a system that includes means for converting audio data into text, extracting important information from the text and generating meeting minutes, automatically extracting tasks from the meeting minutes and adding them to a management system, checking the wording and formatting of documents in real time, analyzing the content of emails and documents and generating summaries, detecting technical terms from the analyzed content and providing explanations, checking participants' schedules and proposing the optimal meeting date and time, and notifying participants of the optimal meeting date and time. Furthermore, this system includes means for providing real-time feedback on the results of wording and formatting checks to the user, means for applying advanced natural language processing algorithms to provide the results of wording and formatting checks, means for displaying summaries and explanations of technical terms to the user, means for receiving feedback from participants and making adjustments if necessary, and means for notifying all participants of the finalized meeting date and time, thereby providing centralized management and improving work efficiency.

[0006] "Audio data" refers to data in which audio is recorded and stored in digital format.

[0007] "Text" refers to string data in a format that a computer can read.

[0008] A "speech recognition API" is an application program interface for converting speech data into text.

[0009] An "automatic summarization algorithm" is an algorithm that extracts important information from a text and generates a short summary.

[0010] A "task extraction algorithm" is an algorithm that automatically recognizes and extracts specific action items from text or conversations.

[0011] A "management system" is a system that centrally manages information such as tasks and projects, and operates them efficiently.

[0012] "Real-time checking" refers to a process that immediately inspects and provides feedback on data as soon as it is entered.

[0013] A "natural language processing algorithm" is an algorithm that allows computers to understand and process human language.

[0014] A "summary" is a piece of information that extracts the most important points from the original information and presents them in a concise format.

[0015] "Technical terms" refer to specific terms used in a particular field or industry.

[0016] "Explanation" refers to explaining the meaning or background of a particular matter or term.

[0017] "Schedule adjustment" is the process of determining the optimal date and time, taking into account the participants' schedules.

[0018] "Notification" refers to conveying information about a particular matter.

[0019] "Feedback" refers to providing opinions or comments on specific actions or results. [Brief explanation of the drawing]

[0020] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] It is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] It is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] It is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] It shows an emotion map to which a plurality of emotions are mapped. [Figure 10] It shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Embodiments for Carrying Out the Invention

[0021] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0022] First, the language used in the following description will be explained.

[0023] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), and APU (Accelerated Processing Unit).

[0024] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0025] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0026] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0027] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0028] [First Embodiment]

[0029] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0030] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0031] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0032] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0033] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0034] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0035] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0036] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0037] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0038] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0039] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0040] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0041] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[0042] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[0043] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. It also uses a task extraction algorithm to automatically extract tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in Charge A created Document Y" is summarized and managed as specific tasks.

[0044] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[0045] When sending emails or documents, users upload the content they send to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0046] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the scheduling results, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0047] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[0048] The following describes the processing flow.

[0049] Meeting minutes and task management support during meetings.

[0050] Step 1:

[0051] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[0052] Step 2:

[0053] Terminal: Sends recorded audio data to the server.

[0054] Step 3:

[0055] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[0056] Step 4:

[0057] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[0058] Step 5:

[0059] Server: Use a task extraction algorithm to extract specific tasks from summarized meeting minutes and add them to the management system.

[0060] Step 6:

[0061] Server: Sends the generated meeting minutes and task list to the terminal.

[0062] Step 7:

[0063] Terminal: Displays received meeting minutes and task lists to the user.

[0064] Support for checking the wording and formatting when creating documents.

[0065] Step 1:

[0066] User: Create documents using a dedicated application.

[0067] Step 2:

[0068] Terminal: Performs real-time grammar checks and formatting consistency, and provides feedback to the user.

[0069] Step 3:

[0070] Terminal: Sends a draft of the created document to the server.

[0071] Step 4:

[0072] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[0073] Step 5:

[0074] Server: Sends check results and suggestions to the terminal.

[0075] Step 6:

[0076] Terminal: Displays the received check results and suggestions to the user.

[0077] Support for summarizing sent emails and documents, and explaining technical terms.

[0078] Step 1:

[0079] User: Upload emails and documents to the server.

[0080] Step 2:

[0081] Server: Analyzes the content of emails and documents using an automated summarization algorithm and a technical term detection algorithm.

[0082] Step 3:

[0083] Server: Generates a summary from the analysis results.

[0084] Step 4:

[0085] Server: Detects technical terms from analysis results and generates explanations.

[0086] Step 5:

[0087] Server: Sends the generated summary and explanations of technical terms to the terminal.

[0088] Step 6:

[0089] Terminal: Displays the received summary and explanations of technical terms to the user.

[0090] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0091] Step 1:

[0092] User: Enter the meeting title, participant list, and preferred date and time options in the meeting request form.

[0093] Step 2:

[0094] Terminal: Sends the entered information to the server.

[0095] Step 3:

[0096] Server: Uses the participant's calendar API to check each participant's availability.

[0097] Step 4:

[0098] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[0099] Step 5:

[0100] Server: Receive feedback from participants and make adjustments as needed.

[0101] Step 6:

[0102] Server: Notifies all participants of the finalized meeting date and time.

[0103] Step 7:

[0104] Terminal: Displays the finalized meeting date and time to the user.

[0105] The above outlines the specific processing steps for implementing the present invention. This system will improve the efficiency of business activities.

[0106] (Example 1)

[0107] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0108] In today's business environment, tasks such as conducting meetings, taking minutes, managing tasks, creating documents, and scheduling are complex and multifaceted challenges. In particular, there is a need for a way to efficiently automate tasks by converting vast amounts of audio data into text, extracting important information, and streamlining the process. Furthermore, it is necessary to improve work efficiency by analyzing and summarizing the content of user-generated documents and electronic messages, and providing explanations of technical terms. Additionally, a system that checks participants' schedules, automatically suggests optimal meeting times, and effectively coordinates them is crucial. There is a need for a system that can centrally address all of these challenges.

[0109] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0110] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of electronic messages and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This enables centralized management and efficiency of tasks, from the automatic processing of vast amounts of audio data to scheduling.

[0111] "Audio data" refers to conversations and voice information recorded in digital format.

[0112] "Text" refers to written information converted from audio data.

[0113] "Important information" refers to particularly noteworthy points extracted from the content of a meeting or similar event.

[0114] "Meeting minutes" are documents that record the content of meetings and other gatherings in text format.

[0115] "Tasks" refer to specific tasks and activities extracted from meeting minutes and registered in the management system.

[0116] A "management system" is a software system used to manage business operations in an organized manner.

[0117] "Wording" refers to the words and phrases used in a document.

[0118] "Format" refers to the way a document is presented in terms of its format and appearance.

[0119] "Real-time" refers to a situation where processing is performed instantly without delay.

[0120] "Electronic messages" refer to content communicated in digital format, such as email or chat messages.

[0121] A "summary" is a brief overview of a document, statement, or other similar content.

[0122] "Technical terms" are specific words or phrases used in a particular field.

[0123] "Explanation" refers to a commentary that clarifies the meaning and usage of technical terms.

[0124] "Participants" are people who attend meetings or similar events.

[0125] A "schedule" is a time-based plan based on the participants' schedules.

[0126] "Meeting date and time" refers to the date and time the meeting will be held.

[0127] A "notification" is a message or alert used to inform others of information.

[0128] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[0129] At the start of the system, the user first activates a dedicated recording device to collect audio data. This audio data is sent to the server via the terminal. The server uses a speech recognition API, such as Amazon Transcribe, to convert the audio data into text data. For example, audio data such as "At today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[0130] Next, the server uses an automated summarization algorithm (e.g., a BERT-based model) to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to automatically extract tasks from the meeting minutes and add them to the management system (e.g., JIRA). As a result, important information such as "Progress of Project X" and "Person A created Document Y" is summarized and managed as specific tasks.

[0131] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[0132] When sending emails or documents, the user uploads the content to the server. The server analyzes the content using an automatic summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0133] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it from their device to the server. The server uses participants' calendar APIs (e.g., Google® Calendar API) to check their schedules and suggests the best meeting date and time. The scheduling results are notified to the device, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0134] Specific examples of prompt statements are as follows:

[0135] "Please convert the following audio data to text and generate a meeting minutes summary: 'Today's meeting agenda is the progress of Project X. It was decided that person A will create document Y as the next step.' Also, please register the proposed task in the management system."

[0136] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[0137] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0138] Step 1:

[0139] The user activates a dedicated recording device at the start of the meeting to collect audio data. The collected audio data is sent to the server via the terminal. Specifically, the user presses the record button, saves the recording data after the meeting ends, and uploads it to the server. In this process, the input is audio data, and the output is an audio file stored on the server.

[0140] Step 2:

[0141] The server uses a speech recognition API to convert the received audio data into text data. Specifically, the server sends the audio data to a speech recognition service such as Amazon Transcribe and receives the text data in return. In this process, the input is an audio file on the server, and the output is the text data of the audio.

[0142] Step 3:

[0143] The server uses an automated summarization algorithm to extract key points from the generated text data and produce a summary of the meeting minutes. Furthermore, it uses a task extraction algorithm to extract tasks from the meeting minutes and add them to the management system. Specifically, the server inputs text data into the summarization algorithm to extract key points. In this process, the input is text data generated by speech recognition, and the output is the summarized meeting minutes and tasks.

[0144] Step 4:

[0145] Users create documents using a dedicated application. This application checks grammar and formatting in real time and provides feedback to the user. Specifically, the user edits the document, and the application checks the entered text in real time. In this process, the input is the text data entered by the user, and the output is the feedback information.

[0146] Step 5:

[0147] When a user sends an email or document, they upload the content to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. Specifically, the server inputs the uploaded content into the analysis algorithm and generates the analysis results. In this process, the input is the uploaded content, and the output is the summary and explanations of technical terms.

[0148] Step 6:

[0149] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it from their device to the server. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. Specifically, the server uses the calendar API to check participants' schedules and notifies the user of the suggested date and time. In this process, the input is the data from the meeting request form, and the output is the suggested meeting date and time.

[0150] Step 7:

[0151] If final adjustments are needed, the user receives feedback from participants and makes further adjustments. The finalized meeting date and time are notified to all participants by the server. Specifically, the user inputs feedback, the server makes adjustments, and notifies the final decision. In this process, the input is feedback information, and the output is a notification of the finalized meeting date and time.

[0152] (Application Example 1)

[0153] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0154] Traditional meeting management systems often require manual processes for transcribing audio data, creating meeting minutes, extracting tasks, and scheduling meetings, which can be particularly inefficient in production meetings within factories. Furthermore, the lack of real-time minute generation and task management can lead to delays in post-meeting follow-up. This results in a decrease in overall operational efficiency. While advanced technologies such as automating audio data and scheduling are necessary, currently, no integrated system exists that combines these functions.

[0155] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0156] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of emails and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for checking the schedules of personnel participating in the meeting and adjusting the meeting time appropriately. This makes it possible to improve the efficiency of meetings.

[0157] "Audio data" refers to data that records audio in digital format.

[0158] "Text" refers to written information converted from audio data.

[0159] "Important information" refers to the key points or essential content that deserve particular attention during a meeting or discussion.

[0160] "Meeting minutes" refers to a document that records the content of a meeting or discussion.

[0161] A "task" refers to an individual task or activity that must be performed to achieve a specific objective.

[0162] A "management system" is software or a system used to comprehensively manage tasks, schedules, documents, and other similar items.

[0163] "Wording and style" refers to the wording, layout, and formatting used in a document.

[0164] A "summary" is a document or piece of information that concisely summarizes long texts or complex information.

[0165] "Technical terms" refer to special words or terms used in a particular field or area of ​​expertise.

[0166] "Explanation" refers to explaining specialized or complex topics in an easy-to-understand manner, or to the document containing such an explanation.

[0167] "Participants" refer to the people who attend a meeting or discussion.

[0168] A "schedule" is a plan that outlines the planned and scheduled times for specific activities or tasks.

[0169] The "optimal meeting date and time" refers to the most suitable date and time, taking into account the schedules of all meeting participants.

[0170] "Real-time" refers to processing and responding in accordance with the actual moment in which an event is unfolding.

[0171] An "advanced natural language processing algorithm" is an algorithm that uses advanced techniques to perform highly accurate semantic analysis and generation of text.

[0172] "Feedback" refers to the act of returning a response or evaluation of an activity or its results.

[0173] A system for implementing this invention can be realized with the following configuration and procedure.

[0174] 1. Collection of audio data and conversion to text:

[0175] The user first uses a dedicated device to collect audio data during meetings and discussions. A robot equipped with a microphone records the meeting audio, and this data is sent to a server via the terminal. The server uses a speech recognition API (for example, Google's speech recognition API) to convert the audio data into text.

[0176] 2. Extraction of important information and generation of meeting minutes:

[0177] The server extracts key information from the converted text using an automated summarization algorithm (such as Hugging Face's NLP model) and generates meeting minutes. These minutes include important points from the meeting and tasks to be taken as the next steps.

[0178] 3. Task extraction and addition to the management system:

[0179] An algorithm is applied to automatically extract tasks from meeting minutes, and the extracted tasks are added to the management system. This enables efficient task management.

[0180] 4. Real-time checking of wording and formatting:

[0181] When users create meeting materials or reports, they use a dedicated application. This application checks the wording and formatting in real time and provides feedback to the user. The check results are sent back to the server, where advanced natural language processing algorithms are applied.

[0182] 5. Content analysis and explanation of technical terms:

[0183] When emails or documents are uploaded to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. Simultaneously, an algorithm that detects technical terms is activated and provides explanations to the user.

[0184] 6. Scheduling and notifications:

[0185] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the best meeting date and time. Once the best date and time are determined, all participants are notified. Scheduling meetings is particularly important, as this improves productivity.

[0186] As a concrete example, the following prompt statement is used:

[0187] Audio recording: "At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step."

[0188] Summarized text: "Production line adjustments: Person in charge submits revised proposal."

[0189] Example of a prompt:

[0190] "Please summarize this meeting: 'Today's meeting discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[0191] "Extract the task from this text: 'At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[0192] In this way, the system implementing the present invention efficiently supports a wide range of tasks in business activities.

[0193] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0194] Step 1:

[0195] The user collects audio data using a dedicated device during meetings and discussions. A robot equipped with a microphone records the meeting audio and transmits the audio data to a server via a terminal. In this step, the audio data (input) is transmitted to the server (output).

[0196] Step 2:

[0197] The server uses a speech recognition API (for example, Google's speech recognition API) to convert the received audio data into text. This text conversion process transforms the audio data (input) into text data (output). This conversion includes parsing the audio data and generating a string.

[0198] Step 3:

[0199] The server analyzes the text data using an automated summarization algorithm (e.g., Hugging Face's NLP model) to extract important information from the text. In this step, the text data (input) is converted into summarized text (output). The process then extracts keywords and important phrases from the text.

[0200] Step 4:

[0201] The server generates meeting minutes and applies an algorithm to extract tasks from those minutes. Here, the summarized text (input) is generated as meeting minutes (output), and tasks (output) are further extracted. The extracted tasks are added to the management system.

[0202] Step 5:

[0203] A dedicated application used by users to create documents checks the wording and formatting in real time and provides feedback. The application analyzes text data (input) and provides suggested wording revisions and formatting check results (output). The feedback content is generated and notified to the user.

[0204] Step 6:

[0205] When a user uploads emails or documents to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. A technical term detection algorithm is also applied to generate explanations of technical terms. In this process, the uploaded content (input) is transformed into a summary and explanations of technical terms (output).

[0206] Step 7:

[0207] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the most suitable meeting date and time. Here, the meeting request information (input) is suggested as the optimal meeting date and time (output).

[0208] Step 8:

[0209] The server notifies all participants of the optimal meeting date and time. Email and calendar app APIs are used for notification. This ensures that the meeting date and time suggested by the server (input) is notified to all participants (output).

[0210] Step 9:

[0211] After the meeting, the server adds tasks extracted from the generated meeting minutes to the management system and manages their schedules. Here, the task information (input) from the meeting minutes is added to the task list (output) in the management system, and the schedule is updated.

[0212] Detailed processing at each step automates the process, allowing it to proceed more efficiently.

[0213] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0214] This invention is a system that combines the generation of meeting minutes from audio data, task management, checking the wording and formatting of documents, summarizing emails and documents and providing explanations of technical terms, and adjusting meeting schedules with an emotion engine that recognizes the user's emotions. Specific embodiments for carrying out this invention are described below.

[0215] Meeting minutes and task management support during meetings.

[0216] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[0217] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[0218] Combination with the emotion engine

[0219] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[0220] Support for checking the wording and formatting when creating documents.

[0221] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[0222] Combination with the emotion engine

[0223] The emotion engine recognizes the user's emotional state and provides feedback that takes this into account if stress or anxiety is detected. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[0224] Support for summarizing sent emails and documents, and explaining technical terms.

[0225] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0226] Combination with the emotion engine

[0227] The emotion engine evaluates the user's emotional state when analyzing emails and documents, and adjusts the content and expression of summaries and explanations based on that evaluation. For example, if it determines that "the user is tired," it can provide a clear and concise summary.

[0228] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0229] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0230] Combination with the emotion engine

[0231] The emotion engine analyzes participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[0232] As described above, the present invention is a system that combines an emotion engine to significantly improve work efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[0233] The following describes the processing flow.

[0234] Meeting minutes and task management support during meetings.

[0235] Step 1:

[0236] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[0237] Step 2:

[0238] Terminal: Sends recorded audio data to the server.

[0239] Step 3:

[0240] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[0241] Step 4:

[0242] Server: Uses an emotion engine to analyze the user's emotional state and adds relevant information to the text data.

[0243] Step 5:

[0244] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[0245] Step 6:

[0246] Server: Use a task extraction algorithm to extract specific tasks from meeting minutes and add them to the management system.

[0247] Step 7:

[0248] Server: Sends the generated meeting minutes and task list to the terminal.

[0249] Step 8:

[0250] Terminal: Displays received meeting minutes and task lists to the user.

[0251] Support for checking the wording and formatting when creating documents.

[0252] Step 1:

[0253] User: Create documents using a dedicated application.

[0254] Step 2:

[0255] Terminal: Performs grammar checks and formatting consistency in real time, and provides feedback of the check results to the user.

[0256] Step 3:

[0257] Terminal: Uses an emotion engine to analyze the user's emotional state and reflect it in the check feedback.

[0258] Step 4:

[0259] Terminal: Sends a draft of the created document to the server.

[0260] Step 5:

[0261] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[0262] Step 6:

[0263] Server: Sends check results and suggestions to the terminal.

[0264] Step 7:

[0265] Terminal: Displays the received check results and suggestions to the user.

[0266] Support for summarizing sent emails and documents, and explaining technical terms.

[0267] Step 1:

[0268] User: Upload emails and documents to the server.

[0269] Step 2:

[0270] Server: Analyze the content of emails and documents using an automatic summarization algorithm and a technical term detection algorithm.

[0271] Step 3:

[0272] Server: Generate a summary from the analysis results.

[0273] Step 4:

[0274] Server: Use an emotion engine to adjust the content and expression of the summary and explanation according to the user's emotional state.

[0275] Step 5:

[0276] Server: Detect technical terms from the analysis results and generate explanations.

[0277] Step 6:

[0278] Server: Send the generated summary and the explanations of technical terms to the terminal.

[0279] Step 7:

[0280] Terminal: Display the received summary and the explanations of technical terms to the user.

[0281] Automatic adjustment support for the meeting request recipient and the requester

[0282] Step 1:

[0283] User: Enter the meeting title, participant list, and priority date candidates in the meeting request form.

[0284] Step 2:

[0285] Terminal: Send the entered information to the server.

[0286] Step 3: [[ID=**68**]]

[0287] Server: Uses the participant's calendar API to check each participant's availability.

[0288] Step 4:

[0289] Server: Uses an emotion engine to analyze participants' emotional states based on historical data and current conditions.

[0290] Step 5:

[0291] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[0292] Step 6:

[0293] Server: Receive feedback from participants and make adjustments as needed.

[0294] Step 7:

[0295] Server: Notifies all participants of the finalized meeting date and time.

[0296] Step 8:

[0297] Terminal: Displays the finalized meeting date and time to the user.

[0298] The above outlines the specific processing steps of the present invention, which incorporates an emotion engine. This system enables the entire business process to proceed efficiently and provides meticulous support that takes into account the user's emotions.

[0299] (Example 2)

[0300] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0301] Traditional business support systems, such as those for meeting minute generation, task management, document creation, email summarization, and meeting scheduling, are often provided individually, resulting in problems with the time and effort required for integrated operation and management. Furthermore, the difficulty in providing feedback and support that takes into account the user's emotional state posed a risk of decreased work efficiency and user satisfaction.

[0302] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting audio data, means for transmitting audio data to the server, means for converting audio data to text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for monitoring the user's emotional state, means for reflecting the user's emotional information in the meeting minutes, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for adjusting the meeting date and time considering the user's emotional state. As a result, the user can process tasks integrally and efficiently with a single system, and furthermore, receiving feedback and support that takes into account their emotional state improves work efficiency and increases user satisfaction.

[0303] "Audio data" refers to data collected using recording devices, such as audio from meetings or conferences.

[0304] A "server" is a computer system that processes, records, analyzes, and notifies audio data.

[0305] A "speech recognition API" is an application programming interface for converting speech data into text data.

[0306] "Text data" refers to character information converted using the speech recognition API.

[0307] "Minutes" refers to a document recording the content of a meeting or consultation.

[0308] "Task management system" refers to a system for managing the extracted tasks and tracking progress.

[0309] "Emotion engine" refers to software for analyzing emotions from a user's voice, expression, etc.

[0310] "Grammar check" refers to a process of checking whether the language of a document follows the rules of the language.

[0311] "Format check" refers to a process of checking whether a document follows a certain format or layout.

[0312] "Automatic summarization algorithm" refers to a program for extracting important parts from a long text and summarizing them concisely.

[0313] "Technical term detection algorithm" refers to a program for identifying technical terms in a document and providing their definitions and explanations.

[0314] "Calendar API" refers to an application programming interface for checking the schedules of each participant.

[0315] "Feedback" refers to the evaluation and improvement suggestions provided by the system to the user.

[0316] Mode for carrying out the invention

[0317] This invention is a system that combines the generation of meeting minutes from audio data, task management, text formatting checks for document creation, summarization of emails and documents, explanation of technical terms, and scheduling of meetings with an emotion engine that recognizes user emotions. Specific embodiments for carrying out this invention are described below.

[0318] Meeting minutes and task management support during meetings.

[0319] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[0320] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[0321] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[0322] Support for checking the wording and formatting when creating documents.

[0323] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[0324] The server uses an emotion engine to recognize the user's emotional state and provides feedback that takes stress or anxiety into account. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[0325] Support for summarizing sent emails and documents, and explaining technical terms.

[0326] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0327] The server uses an emotion engine to assess the user's emotional state when analyzing emails and documents, and adjusts the content and wording of summaries and explanations based on that assessment. For example, if it determines that the user is tired, it can provide a clear and concise summary.

[0328] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0329] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0330] The server uses an emotion engine to analyze participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[0331] Examples of specific cases and prompt statements

[0332] The following prompt statements are possible as concrete examples of how this system is operated:

[0333] 1. "Record meetings and automatically generate meeting minutes and tasks. Analyze user sentiment as well."

[0334] 2. "Please check the materials in real time and suggest appropriate wording and formatting. We will also take user sentiment into consideration."

[0335] 3. "Upload emails and documents, and generate summaries of the content and explanations of technical terms. User sentiment will also be reflected as appropriate."

[0336] 4. "Please coordinate the meeting and propose the most suitable date and time. Please also consider the feelings of the participants."

[0337] As described above, the system of the present invention provides an integrated solution that combines an emotion engine to significantly improve operational efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[0338] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0339] Meeting minutes and task management support during meetings.

[0340] Step 1:

[0341] The user activates the recording device and collects audio data.

[0342] Input: Audio data of a meeting initiated by the user.

[0343] Specific operation: The user manually activates a dedicated recording device and uses a high-sensitivity microphone to collect audio from the meeting.

[0344] Output: Recorded audio data.

[0345] Step 2:

[0346] The device sends voice data to the server.

[0347] Input: Audio data collected by a recording device.

[0348] Specific operation: The device uploads the recorded audio data to the server using Wi-Fi or mobile data communication.

[0349] Output: Audio data uploaded to the server.

[0350] Step 3:

[0351] The server uses a speech recognition API to convert the audio data into text.

[0352] Input: Audio data uploaded to the server.

[0353] Specific operation: The server uses speech recognition APIs such as the Google Cloud Speech-to-Text API to convert speech data into text data.

[0354] Output: Converted text data.

[0355] Step 4:

[0356] The server extracts key points from the text and generates a summary of the meeting minutes.

[0357] Input: Converted text data.

[0358] Specific operation: The server uses natural language processing algorithms to extract important information from text data and generate a summary.

[0359] Output: A summary of the generated meeting minutes.

[0360] Step 5:

[0361] The server generates tasks using a task extraction algorithm and adds them to the management system.

[0362] Input: A summary of the generated meeting minutes.

[0363] Specific operation: The server applies a task extraction algorithm to extract specific tasks from the summary and add them to the management system.

[0364] Output: Tasks added to the management system.

[0365] Step 6:

[0366] The server uses an emotion engine to monitor the user's emotional state and reflect it in the meeting minutes.

[0367] Input: Audio recordings, facial expressions, and text input during meetings.

[0368] Specific operation: The emotion engine analyzes this data and recognizes the user's emotional state.

[0369] Output: Meeting minutes containing emotional information.

[0370] Support for checking the wording and formatting when creating documents.

[0371] Step 1:

[0372] Users create documents using a dedicated application.

[0373] Input: Drafts or documents of materials.

[0374] Specific operation: The user creates a document using a document creation application, and grammar and formatting checks are performed in real time.

[0375] Output: Draft version of the document.

[0376] Step 2:

[0377] The terminal sends a draft of the document to the server.

[0378] Input: Draft version of the document.

[0379] Specific operation: The terminal temporarily stores the draft of the document on the server and then securely transmits it.

[0380] Output: Draft version of the document sent to the server.

[0381] Step 3:

[0382] The server checks the wording and formatting and suggests revisions.

[0383] Input: Draft document sent to the server.

[0384] Specific operation: The server applies advanced natural language processing algorithms to check the wording and formatting. For example, it might suggest changing "This document is very easy to understand" to "This document is extremely easy to understand."

[0385] Output: Check results of the document including the proposed revisions.

[0386] Step 4:

[0387] The server sends the correction results to the terminal and provides feedback to the user.

[0388] Input: Check results of the document containing the proposed revisions.

[0389] Specific operation: The check results are reflected in real time on the interface of the dedicated application.

[0390] Output: Correction results provided to the user as feedback.

[0391] Step 5:

[0392] The server uses an emotion engine to adjust correction suggestions based on the user's emotions.

[0393] Input: User emotional state data (e.g., whether the user is tense, stressed, etc.).

[0394] Specific operation: The emotion engine analyzes emotional data and adjusts the suggested corrections based on the analysis results. For example, if the user is feeling stressed, it will suggest corrections using gentler language.

[0395] Output: Revised correction suggestions.

[0396] Support for summarizing sent emails and documents, and explaining technical terms.

[0397] Step 1:

[0398] Users upload emails and documents to the server.

[0399] Input: Emails or documents to send.

[0400] Specific operation: Users use a dedicated upload form to upload emails and documents to the server.

[0401] Output: Emails and documents uploaded to the server.

[0402] Step 2:

[0403] The server performs the analysis using an automatic summarization algorithm and a technical term detection algorithm.

[0404] Input: Emails and documents uploaded to the server.

[0405] Specific operation: The server uses an automatic summarization algorithm to summarize the content and a terminology detection algorithm to extract technical terms.

[0406] Output: Summary and list of technical terms.

[0407] Step 3:

[0408] The server generates a summary and explanations of technical terms.

[0409] Input: Summary and list of technical terms.

[0410] Specific operation: The server generates explanations of technical terms and formats them together with summaries of emails and documents.

[0411] Output: Summary and explanation of technical terms.

[0412] Step 4:

[0413] The server sends the generated summary and explanation to the terminal and provides feedback to the user.

[0414] Input: Summary and explanation of technical terms.

[0415] Specific operation: The summary and explanation results are sent to the terminal and displayed to the user.

[0416] Output: Summary and explanation of technical terms displayed to the user.

[0417] Step 5:

[0418] The server uses an emotion engine to adjust the content and expression of summaries and explanations.

[0419] Input: User's emotional state data.

[0420] Specific operation: The emotion engine assesses the user's emotional state and adjusts the content and expression of the summary and explanation based on that assessment. For example, if the user is tired, it will provide a clear and concise summary.

[0421] Output: Adjusted summary and explanation of technical terms.

[0422] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0423] Step 1:

[0424] The user enters the meeting details into a meeting request form and sends it from their device to the server.

[0425] Input: Details such as the meeting title, participant list, and preferred date and time options.

[0426] Specific operation: The user enters the required information into the meeting request form and sends it from their device to the server.

[0427] Output: Meeting details sent to the server.

[0428] Step 2:

[0429] The server uses the participants' calendar API to check their schedules.

[0430] Input: Participant list.

[0431] Specific operation: The server uses the Calendar API to check each participant's schedule.

[0432] Output: A list of available time slots for all participants.

[0433] Step 3:

[0434] The server will suggest the optimal meeting date and time.

[0435] Input: A list of available time slots for all participants.

[0436] Specific operation: The server calculates and proposes the optimal meeting date and time when all participants can attend.

[0437] Output: Proposed meeting date and time.

[0438] Step 4:

[0439] The server sends the adjustment results to the terminal and notifies the user.

[0440] Input: Proposed meeting date and time.

[0441] Specific operation: The adjustment results are sent to the terminal and notified to the user.

[0442] Output: The adjustment results notified to the user.

[0443] Step 5:

[0444] The server uses an emotion engine to analyze the emotional state of participants and suggest the optimal meeting date and time.

[0445] Input: Participant emotional state data.

[0446] Specific operation: The emotion engine analyzes the participants' busyness and stress levels and suggests the optimal meeting date and time, taking these factors into consideration.

[0447] Output: Results of adjusting meeting dates and times, taking emotional states into consideration.

[0448] The above describes the specific processing steps and operations related to the program of this system.

[0449] (Application Example 2)

[0450] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0451] In today's business environment, there is a growing demand for increased efficiency in various meetings and conferences. In particular, meetings conducted within autonomous mobile devices face challenges due to the lack of automation in tasks such as generating meeting minutes from audio data, task management, formatting checks for documents, and summarizing emails and documents while providing explanations of technical terms. Furthermore, the lack of means to provide appropriate feedback tailored to the emotional state of users impacts the quality and efficiency of work.

[0452] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for collecting audio data in real time within an autonomous mobile device, means for analyzing the user's emotions from the collected audio and text using an emotion recognition engine, means for providing appropriate feedback and summaries based on the user's emotional state, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This makes it possible to improve the efficiency and quality of meetings and conferences.

[0453] "Audio data" refers to data that records audio in digital format.

[0454] "Means of converting to text" refers to technology or equipment for analyzing audio data and converting it into text format.

[0455] "Means for extracting important information" refers to technologies or devices that automatically select key points or important information from text.

[0456] "Means for generating meeting minutes" refers to the technology or device that compiles extracted important information into a format suitable for meeting minutes.

[0457] "Means for automatically extracting tasks and adding them to a management system" refers to a technology or device that identifies tasks from meeting minutes and automatically registers them in a management system.

[0458] "Autonomous mobile devices" refer to all mobile devices that can move automatically without human intervention.

[0459] "Means for collecting audio data in real time" refers to technology or equipment for recording audio in real time within an autonomous mobile device.

[0460] An "emotion recognition engine" is a technology or software used to analyze a user's emotions from voice or text data.

[0461] "Means of providing feedback" refers to technologies or devices that communicate analysis results and support details to users.

[0462] "Methods for checking wording and formatting in real time" refers to technologies or devices that allow for real-time verification of the accuracy and formatting of text in documents and materials, and the provision of revision suggestions.

[0463] "Means for analyzing the content of emails and documents and generating summaries" refers to technologies or devices that read the text of emails and documents and concisely summarize their main points.

[0464] "Means for detecting and providing explanations of technical terms" refers to technology or devices that find technical terms within a document and automatically generate explanations related to them.

[0465] "A means of checking participants' schedules and proposing the optimal meeting date and time" refers to a technology or device that obtains the schedule information of meeting participants and calculates the optimal meeting date and time.

[0466] "Means of notifying participants of the optimal meeting date and time" refers to technology or equipment for notifying meeting participants of the decided meeting date and time.

[0467] The system for realizing this invention consists of a combination of various software and hardware for collecting, converting, analyzing, and providing feedback on audio data. The following describes specific embodiments for implementing this invention.

[0468] First, when a user initiates a meeting or conference within the autonomous mobile device, a dedicated recording device is activated to collect audio data. This recording device includes a microphone and a speech recognition module, and collects audio data in real time. The collected audio data is then transmitted to a server via the terminal.

[0469] The server converts the collected audio data into text using a speech recognition API (e.g., Google Speech API). For example, audio data such as "In today's meeting, we discussed the project's progress. The person in charge decided to create documentation as the next step" would be identified as text.

[0470] Next, the server extracts key points from the text data using an automated summarization algorithm (e.g., Hugging Face's Transformers library) and generates a summary in the form of meeting minutes. This summary might look something like, "Project progress, responsible person created the document."

[0471] After the meeting minutes are generated, the server uses a task extraction algorithm to identify important tasks and automatically add them to the management system. For example, "The person in charge creates the document" is registered as a specific task in the management system.

[0472] An emotion recognition engine (e.g., Hugging Face's emotion recognition model) is also integrated into the server, analyzing the user's emotional state from collected voice and text data. This emotional information is reflected in meeting minutes and feedback, and is recorded, for example, as "the user is feeling stressed."

[0473] When creating documents, users create them on a dedicated application. This application checks the wording and formatting in real time and sends it to a server as needed to apply advanced natural language processing algorithms. For example, in response to the sentence "This document is very easy to understand," a suggested revision would be made to "This document is extremely easy to understand."

[0474] Furthermore, by uploading emails and documents sent by users to the server, automatic summarization and explanations of technical terms are performed. Based on the analysis results, the server provides easy-to-understand summaries and explanations of technical terms. For example, "document" might be explained as "document for a specific project."

[0475] When scheduling a meeting, the user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the most suitable meeting date and time. The decided meeting date and time are then notified to all participants.

[0476] Specific example

[0477] Meeting agenda:

[0478] "At today's meeting, we discussed the project's progress. The person in charge will now prepare the necessary documents as the next step."

[0479] Generated summary:

[0480] "Project progress: The person in charge creates the document."

[0481] Emotion recognition results:

[0482] "Users are feeling stressed."

[0483] Example of a prompt

[0484] Meeting Minutes Generation: "Summarize the key points based on the following text: 'Today's meeting discussed project progress. The person in charge decided to prepare the document as the next step.'"

[0485] Sentiment Analysis: "Analyze the user's emotional state from the following text: 'In today's meeting, we discussed the project's progress. The person in charge decided to create a document as the next step.'"

[0486] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0487] Step 1:

[0488] Audio data is collected within the autonomous mobile device.

[0489] A dedicated recording device is activated to collect audio data from meetings and conferences in real time. The collected audio data is transmitted to the terminal via a microphone.

[0490] Input: Audio data of the user's conversation

[0491] Output: Audio data sent to the terminal

[0492] Step 2:

[0493] Convert audio data to text.

[0494] The device sends the collected audio data to the server, which then uses a speech recognition API (e.g., Google Speech API) to convert the audio data into text.

[0495] Input: Audio data sent from the device

[0496] Output: Text data

[0497] Step 3:

[0498] Extract important information from text data and generate a summary.

[0499] The server uses an automated summarization algorithm (e.g., Hugging Face's Transformers library) to extract key points from text data and generate a summary.

[0500] Input: Text data

[0501] Output: Summary text

[0502] Step 4:

[0503] Based on the summarized text, generate meeting minutes and extract tasks.

[0504] The server generates meeting minutes based on the generated summary text, identifies key tasks using a task extraction algorithm, and automatically adds them to the management system.

[0505] Input: Summary text

[0506] Output: Meeting minutes, task list

[0507] Step 5:

[0508] The system uses an emotion recognition engine to analyze the user's emotions.

[0509] The server uses an emotion recognition engine (e.g., Hugging Face's emotion recognition model) to analyze the user's emotional state from the collected audio and text data and record that state.

[0510] Input: Audio data, text data

[0511] Output: Emotional state

[0512] Step 6:

[0513] We provide users with analysis results and feedback.

[0514] The terminal receives meeting minutes, task lists, and emotional state analysis results sent from the server, and displays them on the user's screen.

[0515] Input: Meeting minutes, task list, emotional state

[0516] Output: Feedback information displayed on the user's screen

[0517] Step 7:

[0518] Check the wording and formatting of the document in real time.

[0519] When creating documents using a dedicated application, the wording and formatting are checked in real time, and if necessary, the documents are sent to a server where advanced natural language processing algorithms are applied to suggest revisions.

[0520] Input: Document being created

[0521] Output: Proposed revisions

[0522] Step 8:

[0523] It analyzes the content of emails and documents, and provides summaries and explanations of technical terms.

[0524] When a user uploads an email or document to the server, the server analyzes the content using an automated summarization and terminology detection algorithm, generating a summary and explanation.

[0525] Input: Email, documents

[0526] Output: Summary and explanation of technical terms

[0527] Step 9:

[0528] We will adjust the meeting schedule and propose the most suitable date and time.

[0529] The user fills out a meeting request form and sends it from their device to the server. The server uses the participants' calendar API to check their schedules, suggests the best meeting time, and notifies all participants.

[0530] Input: Meeting request form, participant calendar information

[0531] Output: Notification of optimal meeting date and time

[0532] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0533] Data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0534] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0535] [Second Embodiment]

[0536] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0537] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0538] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0539] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0540] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0541] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0542] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0543] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0544] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0545] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0546] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0547] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0548] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[0549] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[0550] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. It also uses a task extraction algorithm to automatically extract tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in Charge A created Document Y" is summarized and managed as specific tasks.

[0551] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[0552] When sending emails or documents, users upload the content they send to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0553] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the scheduling results, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0554] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[0555] The following describes the processing flow.

[0556] Meeting minutes and task management support during meetings.

[0557] Step 1:

[0558] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[0559] Step 2:

[0560] Terminal: Sends recorded audio data to the server.

[0561] Step 3:

[0562] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[0563] Step 4:

[0564] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[0565] Step 5:

[0566] Server: Use a task extraction algorithm to extract specific tasks from summarized meeting minutes and add them to the management system.

[0567] Step 6:

[0568] Server: Sends the generated meeting minutes and task list to the terminal.

[0569] Step 7:

[0570] Terminal: Displays received meeting minutes and task lists to the user.

[0571] Support for checking the wording and formatting when creating documents.

[0572] Step 1:

[0573] User: Create documents using a dedicated application.

[0574] Step 2:

[0575] Terminal: Performs real-time grammar checks and formatting consistency, and provides feedback to the user.

[0576] Step 3:

[0577] Terminal: Sends a draft of the created document to the server.

[0578] Step 4:

[0579] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[0580] Step 5:

[0581] Server: Sends check results and suggestions to the terminal.

[0582] Step 6:

[0583] Terminal: Displays the received check results and suggestions to the user.

[0584] Support for summarizing sent emails and documents, and explaining technical terms.

[0585] Step 1:

[0586] User: Upload emails and documents to the server.

[0587] Step 2:

[0588] Server: Analyzes the content of emails and documents using an automated summarization algorithm and a technical term detection algorithm.

[0589] Step 3:

[0590] Server: Generates a summary from the analysis results.

[0591] Step 4:

[0592] Server: Detects technical terms from analysis results and generates explanations.

[0593] Step 5:

[0594] Server: Sends the generated summary and explanations of technical terms to the terminal.

[0595] Step 6:

[0596] Terminal: Displays the received summary and explanations of technical terms to the user.

[0597] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0598] Step 1:

[0599] User: Enter the meeting title, participant list, and preferred date and time options in the meeting request form.

[0600] Step 2:

[0601] Terminal: Sends the entered information to the server.

[0602] Step 3:

[0603] Server: Uses the participant's calendar API to check each participant's availability.

[0604] Step 4:

[0605] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[0606] Step 5:

[0607] Server: Receive feedback from participants and make adjustments as needed.

[0608] Step 6:

[0609] Server: Notifies all participants of the finalized meeting date and time.

[0610] Step 7:

[0611] Terminal: Displays the finalized meeting date and time to the user.

[0612] The above outlines the specific processing steps for implementing the present invention. This system will improve the efficiency of business activities.

[0613] (Example 1)

[0614] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0615] In today's business environment, tasks such as conducting meetings, taking minutes, managing tasks, creating documents, and scheduling are complex and multifaceted challenges. In particular, there is a need for a way to efficiently automate tasks by converting vast amounts of audio data into text, extracting important information, and streamlining the process. Furthermore, it is necessary to improve work efficiency by analyzing and summarizing the content of user-generated documents and electronic messages, and providing explanations of technical terms. Additionally, a system that checks participants' schedules, automatically suggests optimal meeting times, and effectively coordinates them is crucial. There is a need for a system that can centrally address all of these challenges.

[0616] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0617] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of electronic messages and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This enables centralized management and efficiency of tasks, from the automatic processing of vast amounts of audio data to scheduling.

[0618] "Audio data" refers to conversations and voice information recorded in digital format.

[0619] "Text" refers to written information converted from audio data.

[0620] "Important information" refers to particularly noteworthy points extracted from the content of a meeting or similar event.

[0621] "Meeting minutes" are documents that record the content of meetings and other gatherings in text format.

[0622] "Tasks" refer to specific tasks and activities extracted from meeting minutes and registered in the management system.

[0623] A "management system" is a software system used to manage business operations in an organized manner.

[0624] "Wording" refers to the words and phrases used in a document.

[0625] "Format" refers to the way a document is presented in terms of its format and appearance.

[0626] "Real-time" refers to a situation where processing is performed instantly without delay.

[0627] "Electronic messages" refer to content communicated in digital format, such as email or chat messages.

[0628] A "summary" is a brief overview of a document, statement, or other similar content.

[0629] "Technical terms" are specific words or phrases used in a particular field.

[0630] "Explanation" refers to a commentary that clarifies the meaning and usage of technical terms.

[0631] "Participants" are people who attend meetings or similar events.

[0632] A "schedule" is a time-based plan based on the participants' schedules.

[0633] "Meeting date and time" refers to the date and time the meeting will be held.

[0634] A "notification" is a message or alert used to inform others of information.

[0635] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[0636] At the start of the system, the user first activates a dedicated recording device to collect audio data. This audio data is sent to the server via the terminal. The server uses a speech recognition API, such as Amazon Transcribe, to convert the audio data into text data. For example, audio data such as "At today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[0637] Next, the server uses an automated summarization algorithm (e.g., a BERT-based model) to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to automatically extract tasks from the meeting minutes and add them to the management system (e.g., JIRA). As a result, important information such as "Progress of Project X" and "Person A created Document Y" is summarized and managed as specific tasks.

[0638] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[0639] When sending emails or documents, the user uploads the content to the server. The server analyzes the content using an automatic summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0640] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it from their device to the server. The server uses participants' calendar APIs (e.g., Google Calendar API) to check their schedules and suggests the best meeting date and time. The scheduling results are notified to the device, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0641] Specific examples of prompt statements are as follows:

[0642] "Please convert the following audio data to text and generate a meeting minutes summary: 'Today's meeting agenda is the progress of Project X. It was decided that person A will create document Y as the next step.' Also, please register the proposed task in the management system."

[0643] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[0644] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0645] Step 1:

[0646] The user activates a dedicated recording device at the start of the meeting to collect audio data. The collected audio data is sent to the server via the terminal. Specifically, the user presses the record button, saves the recording data after the meeting ends, and uploads it to the server. In this process, the input is audio data, and the output is an audio file stored on the server.

[0647] Step 2:

[0648] The server uses a speech recognition API to convert the received audio data into text data. Specifically, the server sends the audio data to a speech recognition service such as Amazon Transcribe and receives the text data in return. In this process, the input is an audio file on the server, and the output is the text data of the audio.

[0649] Step 3:

[0650] The server uses an automated summarization algorithm to extract key points from the generated text data and produce a summary of the meeting minutes. Furthermore, it uses a task extraction algorithm to extract tasks from the meeting minutes and add them to the management system. Specifically, the server inputs text data into the summarization algorithm to extract key points. In this process, the input is text data generated by speech recognition, and the output is the summarized meeting minutes and tasks.

[0651] Step 4:

[0652] Users create documents using a dedicated application. This application checks grammar and formatting in real time and provides feedback to the user. Specifically, the user edits the document, and the application checks the entered text in real time. In this process, the input is the text data entered by the user, and the output is the feedback information.

[0653] Step 5:

[0654] When a user sends an email or document, they upload the content to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. Specifically, the server inputs the uploaded content into the analysis algorithm and generates the analysis results. In this process, the input is the uploaded content, and the output is the summary and explanations of technical terms.

[0655] Step 6:

[0656] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it from their device to the server. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. Specifically, the server uses the calendar API to check participants' schedules and notifies the user of the suggested date and time. In this process, the input is the data from the meeting request form, and the output is the suggested meeting date and time.

[0657] Step 7:

[0658] If final adjustments are needed, the user receives feedback from participants and makes further adjustments. The finalized meeting date and time are notified to all participants by the server. Specifically, the user inputs feedback, the server makes adjustments, and notifies the final decision. In this process, the input is feedback information, and the output is a notification of the finalized meeting date and time.

[0659] (Application Example 1)

[0660] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0661] Traditional meeting management systems often require manual processes for transcribing audio data, creating meeting minutes, extracting tasks, and scheduling meetings, which can be particularly inefficient in production meetings within factories. Furthermore, the lack of real-time minute generation and task management can lead to delays in post-meeting follow-up. This results in a decrease in overall operational efficiency. While advanced technologies such as automating audio data and scheduling are necessary, currently, no integrated system exists that combines these functions.

[0662] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0663] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of emails and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for checking the schedules of personnel participating in the meeting and adjusting the meeting time appropriately. This makes it possible to improve the efficiency of meetings.

[0664] "Audio data" refers to data that records audio in digital format.

[0665] "Text" refers to written information converted from audio data.

[0666] "Important information" refers to the key points or essential content that deserve particular attention during a meeting or discussion.

[0667] "Meeting minutes" refers to a document that records the content of a meeting or discussion.

[0668] A "task" refers to an individual task or activity that must be performed to achieve a specific objective.

[0669] A "management system" is software or a system used to comprehensively manage tasks, schedules, documents, and other similar items.

[0670] "Wording and style" refers to the wording, layout, and formatting used in a document.

[0671] A "summary" is a document or piece of information that concisely summarizes long texts or complex information.

[0672] "Technical terms" refer to special words or terms used in a particular field or area of ​​expertise.

[0673] "Explanation" refers to explaining specialized or complex topics in an easy-to-understand manner, or to the document containing such an explanation.

[0674] "Participants" refer to the people who attend a meeting or discussion.

[0675] A "schedule" is a plan that outlines the planned and scheduled times for specific activities or tasks.

[0676] The "optimal meeting date and time" refers to the most suitable date and time, taking into account the schedules of all meeting participants.

[0677] "Real-time" refers to processing and responding in accordance with the actual moment in which an event is unfolding.

[0678] An "advanced natural language processing algorithm" is an algorithm that uses advanced techniques to perform highly accurate semantic analysis and generation of text.

[0679] "Feedback" refers to the act of returning a response or evaluation of an activity or its results.

[0680] A system for implementing this invention can be realized with the following configuration and procedure.

[0681] 1. Collection of audio data and conversion to text:

[0682] The user first uses a dedicated device to collect audio data during meetings and discussions. A robot equipped with a microphone records the meeting audio, and this data is sent to a server via the terminal. The server uses a speech recognition API (for example, Google's speech recognition API) to convert the audio data into text.

[0683] 2. Extraction of important information and generation of meeting minutes:

[0684] The server extracts key information from the converted text using an automated summarization algorithm (such as Hugging Face's NLP model) and generates meeting minutes. These minutes include important points from the meeting and tasks to be taken as the next steps.

[0685] 3. Task extraction and addition to the management system:

[0686] An algorithm is applied to automatically extract tasks from meeting minutes, and the extracted tasks are added to the management system. This enables efficient task management.

[0687] 4. Real-time checking of wording and formatting:

[0688] When users create meeting materials or reports, they use a dedicated application. This application checks the wording and formatting in real time and provides feedback to the user. The check results are sent back to the server, where advanced natural language processing algorithms are applied.

[0689] 5. Content analysis and explanation of technical terms:

[0690] When emails or documents are uploaded to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. Simultaneously, an algorithm that detects technical terms is activated and provides explanations to the user.

[0691] 6. Scheduling and notifications:

[0692] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the best meeting date and time. Once the best date and time are determined, all participants are notified. Scheduling meetings is particularly important, as this improves productivity.

[0693] As a concrete example, the following prompt statement is used:

[0694] Audio recording: "At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step."

[0695] Summarized text: "Production line adjustments: Person in charge submits revised proposal."

[0696] Example of a prompt:

[0697] "Please summarize this meeting: 'Today's meeting discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[0698] "Extract the task from this text: 'At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[0699] In this way, the system implementing the present invention efficiently supports a wide range of tasks in business activities.

[0700] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0701] Step 1:

[0702] The user collects audio data using a dedicated device during meetings and discussions. A robot equipped with a microphone records the meeting audio and transmits the audio data to a server via a terminal. In this step, the audio data (input) is transmitted to the server (output).

[0703] Step 2:

[0704] The server uses a speech recognition API (for example, Google's speech recognition API) to convert the received audio data into text. This text conversion process transforms the audio data (input) into text data (output). This conversion includes parsing the audio data and generating a string.

[0705] Step 3:

[0706] The server analyzes the text data using an automated summarization algorithm (e.g., Hugging Face's NLP model) to extract important information from the text. In this step, the text data (input) is converted into summarized text (output). The process then extracts keywords and important phrases from the text.

[0707] Step 4:

[0708] The server generates meeting minutes and applies an algorithm to extract tasks from those minutes. Here, summarized text (input) is generated as meeting minutes (output), and tasks (output) are further extracted. The extracted tasks are added to the management system.

[0709] Step 5:

[0710] A dedicated application used by users to create documents checks the wording and formatting in real time and provides feedback. The application analyzes text data (input) and provides suggested wording revisions and formatting check results (output). The feedback content is generated and notified to the user.

[0711] Step 6:

[0712] When a user uploads emails or documents to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. A technical term detection algorithm is also applied to generate explanations of technical terms. In this process, the uploaded content (input) is transformed into a summary and explanations of technical terms (output).

[0713] Step 7:

[0714] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the most suitable meeting date and time. Here, the meeting request information (input) is suggested as the optimal meeting date and time (output).

[0715] Step 8:

[0716] The server notifies all participants of the optimal meeting date and time. Email and calendar app APIs are used for notification. This ensures that the meeting date and time suggested by the server (input) is notified to all participants (output).

[0717] Step 9:

[0718] After the meeting, the server adds tasks extracted from the generated meeting minutes to the management system and manages their schedules. Here, the task information (input) from the meeting minutes is added to the task list (output) in the management system, and the schedule is updated.

[0719] Detailed processing at each step automates the process, allowing it to proceed more efficiently.

[0720] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0721] This invention is a system that combines the generation of meeting minutes from audio data, task management, checking the wording and formatting of documents, summarizing emails and documents and providing explanations of technical terms, and adjusting meeting schedules with an emotion engine that recognizes the user's emotions. Specific embodiments for carrying out this invention are described below.

[0722] Meeting minutes and task management support during meetings.

[0723] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[0724] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[0725] Combination with the emotion engine

[0726] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[0727] Support for checking the wording and formatting when creating documents.

[0728] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[0729] Combination with the emotion engine

[0730] The emotion engine recognizes the user's emotional state and provides feedback that takes this into account if stress or anxiety is detected. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[0731] Support for summarizing sent emails and documents, and explaining technical terms.

[0732] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0733] Combination with the emotion engine

[0734] The emotion engine evaluates the user's emotional state when analyzing emails and documents, and adjusts the content and expression of summaries and explanations based on that evaluation. For example, if it determines that "the user is tired," it can provide a clear and concise summary.

[0735] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0736] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0737] Combination with the emotion engine

[0738] The emotion engine analyzes participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[0739] As described above, the present invention is a system that combines an emotion engine to significantly improve work efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[0740] The following describes the processing flow.

[0741] Meeting minutes and task management support during meetings.

[0742] Step 1:

[0743] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[0744] Step 2:

[0745] Terminal: Sends recorded audio data to the server.

[0746] Step 3:

[0747] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[0748] Step 4:

[0749] Server: Uses an emotion engine to analyze the user's emotional state and adds relevant information to the text data.

[0750] Step 5:

[0751] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[0752] Step 6:

[0753] Server: Use a task extraction algorithm to extract specific tasks from meeting minutes and add them to the management system.

[0754] Step 7:

[0755] Server: Sends the generated meeting minutes and task list to the terminal.

[0756] Step 8:

[0757] Terminal: Displays received meeting minutes and task lists to the user.

[0758] Support for checking the wording and formatting when creating documents.

[0759] Step 1:

[0760] User: Create documents using a dedicated application.

[0761] Step 2:

[0762] Terminal: Performs grammar checks and formatting consistency in real time, and provides feedback of the check results to the user.

[0763] Step 3:

[0764] Terminal: Uses an emotion engine to analyze the user's emotional state and reflect it in the check feedback.

[0765] Step 4:

[0766] Terminal: Sends a draft of the created document to the server.

[0767] Step 5:

[0768] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[0769] Step 6:

[0770] Server: Sends check results and suggestions to the terminal.

[0771] Step 7:

[0772] Terminal: Displays the received check results and suggestions to the user.

[0773] Support for summarizing sent emails and documents, and explaining technical terms.

[0774] Step 1:

[0775] User: Upload emails and documents to the server.

[0776] Step 2:

[0777] Server: Analyzes the content of emails and documents using an automated summarization algorithm and a technical term detection algorithm.

[0778] Step 3:

[0779] Server: Generates a summary from the analysis results.

[0780] Step 4:

[0781] Server: Uses an emotion engine to adjust the content and expression of summaries and explanations according to the user's emotional state.

[0782] Step 5:

[0783] Server: Detects technical terms from analysis results and generates explanations.

[0784] Step 6:

[0785] Server: Sends the generated summary and explanations of technical terms to the terminal.

[0786] Step 7:

[0787] Terminal: Displays the received summary and explanations of technical terms to the user.

[0788] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0789] Step 1:

[0790] User: Enter the meeting title, participant list, and preferred date and time options in the meeting request form.

[0791] Step 2:

[0792] Terminal: Sends the entered information to the server.

[0793] Step 3:

[0794] Server: Uses the participant's calendar API to check each participant's availability.

[0795] Step 4:

[0796] Server: Uses an emotion engine to analyze participants' emotional states based on historical data and current conditions.

[0797] Step 5:

[0798] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[0799] Step 6:

[0800] Server: Receive feedback from participants and make adjustments as needed.

[0801] Step 7:

[0802] Server: Notifies all participants of the finalized meeting date and time.

[0803] Step 8:

[0804] Terminal: Displays the finalized meeting date and time to the user.

[0805] The above outlines the specific processing steps of the present invention, which incorporates an emotion engine. This system enables the entire business process to proceed efficiently and provides meticulous support that takes into account the user's emotions.

[0806] (Example 2)

[0807] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0808] Traditional business support systems, such as those for meeting minute generation, task management, document creation, email summarization, and meeting scheduling, are often provided individually, resulting in problems with the time and effort required for integrated operation and management. Furthermore, the difficulty in providing feedback and support that takes into account the user's emotional state posed a risk of decreased work efficiency and user satisfaction.

[0809] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting audio data, means for transmitting audio data to the server, means for converting audio data to text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for monitoring the user's emotional state, means for reflecting the user's emotional information in the meeting minutes, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for adjusting the meeting date and time considering the user's emotional state. As a result, the user can process tasks integrally and efficiently with a single system, and furthermore, receiving feedback and support that takes into account their emotional state improves work efficiency and increases user satisfaction.

[0810] "Audio data" refers to data collected using recording devices, such as audio from meetings or conferences.

[0811] A "server" is a computer system that processes, records, analyzes, and notifies audio data.

[0812] A "speech recognition API" is an application programming interface for converting speech data into text data.

[0813] "Text data" refers to character information converted using a speech recognition API.

[0814] "Meeting minutes" are documents that record the content of meetings or discussions.

[0815] A "task management system" is a system for managing extracted tasks and tracking their progress.

[0816] An "emotion engine" is software that analyzes a user's emotions from their voice, facial expressions, and other data.

[0817] "Grammar check" is the process of verifying whether the wording in a document conforms to linguistic rules.

[0818] "Format checking" is the process of verifying whether a document adheres to a certain format and layout.

[0819] An "automatic summarization algorithm" is a program that extracts important parts from long texts and summarizes them concisely.

[0820] A "technical term detection algorithm" is a program that identifies technical terms within a document and provides their definitions and explanations.

[0821] The "Calendar API" is an application programming interface for checking each participant's schedule.

[0822] "Feedback" refers to the evaluations and improvement suggestions that a system provides to its users.

[0823] Modes for carrying out the invention

[0824] This invention is a system that combines the generation of meeting minutes from audio data, task management, text formatting checks for document creation, summarization of emails and documents, explanation of technical terms, and scheduling of meetings with an emotion engine that recognizes user emotions. Specific embodiments for carrying out this invention are described below.

[0825] Meeting minutes and task management support during meetings.

[0826] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[0827] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[0828] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[0829] Support for checking the wording and formatting when creating documents.

[0830] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[0831] The server uses an emotion engine to recognize the user's emotional state and provides feedback that takes stress or anxiety into account. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[0832] Support for summarizing sent emails and documents, and explaining technical terms.

[0833] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[0834] The server uses an emotion engine to assess the user's emotional state when analyzing emails and documents, and adjusts the content and wording of summaries and explanations based on that assessment. For example, if it determines that the user is tired, it can provide a clear and concise summary.

[0835] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0836] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[0837] The server uses an emotion engine to analyze participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[0838] Examples of specific cases and prompt statements

[0839] The following prompt statements are possible as concrete examples of how this system is operated:

[0840] 1. "Record meetings and automatically generate meeting minutes and tasks. Analyze user sentiment as well."

[0841] 2. "Please check the materials in real time and suggest appropriate wording and formatting. We will also take user sentiment into consideration."

[0842] 3. "Upload emails and documents, and generate summaries of the content and explanations of technical terms. User sentiment will also be reflected as appropriate."

[0843] 4. "Please coordinate the meeting and propose the most suitable date and time. Please also consider the feelings of the participants."

[0844] As described above, the system of the present invention provides an integrated solution that combines an emotion engine to significantly improve operational efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[0845] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0846] Meeting minutes and task management support during meetings.

[0847] Step 1:

[0848] The user activates the recording device and collects audio data.

[0849] Input: Audio data of a meeting initiated by the user.

[0850] Specific operation: The user manually activates a dedicated recording device and uses a high-sensitivity microphone to collect audio from the meeting.

[0851] Output: Recorded audio data.

[0852] Step 2:

[0853] The device sends voice data to the server.

[0854] Input: Audio data collected by a recording device.

[0855] Specific operation: The device uploads the recorded audio data to the server using Wi-Fi or mobile data communication.

[0856] Output: Audio data uploaded to the server.

[0857] Step 3:

[0858] The server uses a speech recognition API to convert the audio data into text.

[0859] Input: Audio data uploaded to the server.

[0860] Specific operation: The server uses speech recognition APIs such as the Google Cloud Speech-to-Text API to convert speech data into text data.

[0861] Output: Converted text data.

[0862] Step 4:

[0863] The server extracts key points from the text and generates a summary of the meeting minutes.

[0864] Input: Converted text data.

[0865] Specific operation: The server uses natural language processing algorithms to extract important information from text data and generate a summary.

[0866] Output: A summary of the generated meeting minutes.

[0867] Step 5:

[0868] The server generates tasks using a task extraction algorithm and adds them to the management system.

[0869] Input: A summary of the generated meeting minutes.

[0870] Specific operation: The server applies a task extraction algorithm to extract specific tasks from the summary and add them to the management system.

[0871] Output: Tasks added to the management system.

[0872] Step 6:

[0873] The server uses an emotion engine to monitor the user's emotional state and reflect it in the meeting minutes.

[0874] Input: Audio recordings, facial expressions, and text input during meetings.

[0875] Specific operation: The emotion engine analyzes this data and recognizes the user's emotional state.

[0876] Output: Meeting minutes containing emotional information.

[0877] Support for checking the wording and formatting when creating documents.

[0878] Step 1:

[0879] Users create documents using a dedicated application.

[0880] Input: Drafts or documents of materials.

[0881] Specific operation: The user creates a document using a document creation application, and grammar and formatting checks are performed in real time.

[0882] Output: Draft version of the document.

[0883] Step 2:

[0884] The terminal sends a draft of the document to the server.

[0885] Input: Draft version of the document.

[0886] Specific operation: The terminal temporarily stores the draft of the document on the server and then securely transmits it.

[0887] Output: Draft version of the document sent to the server.

[0888] Step 3:

[0889] The server checks the wording and formatting and suggests revisions.

[0890] Input: Draft document sent to the server.

[0891] Specific operation: The server applies advanced natural language processing algorithms to check the wording and formatting. For example, it might suggest changing "This document is very easy to understand" to "This document is extremely easy to understand."

[0892] Output: Check results of the document including the proposed revisions.

[0893] Step 4:

[0894] The server sends the correction results to the terminal and provides feedback to the user.

[0895] Input: Check results of the document containing the proposed revisions.

[0896] Specific operation: The check results are reflected in real time on the interface of the dedicated application.

[0897] Output: Correction results provided to the user as feedback.

[0898] Step 5:

[0899] The server uses an emotion engine to adjust correction suggestions based on the user's emotions.

[0900] Input: User emotional state data (e.g., whether the user is tense, stressed, etc.).

[0901] Specific operation: The emotion engine analyzes emotional data and adjusts the suggested corrections based on the analysis results. For example, if the user is feeling stressed, it will suggest corrections using gentler language.

[0902] Output: Revised correction suggestions.

[0903] Support for summarizing sent emails and documents, and explaining technical terms.

[0904] Step 1:

[0905] Users upload emails and documents to the server.

[0906] Input: Emails or documents to send.

[0907] Specific operation: Users use a dedicated upload form to upload emails and documents to the server.

[0908] Output: Emails and documents uploaded to the server.

[0909] Step 2:

[0910] The server performs the analysis using an automatic summarization algorithm and a technical term detection algorithm.

[0911] Input: Emails and documents uploaded to the server.

[0912] Specific operation: The server uses an automatic summarization algorithm to summarize the content and a terminology detection algorithm to extract technical terms.

[0913] Output: Summary and list of technical terms.

[0914] Step 3:

[0915] The server generates a summary and explanations of technical terms.

[0916] Input: Summary and list of technical terms.

[0917] Specific operation: The server generates explanations of technical terms and formats them together with summaries of emails and documents.

[0918] Output: Summary and explanation of technical terms.

[0919] Step 4:

[0920] The server sends the generated summary and explanation to the terminal and provides feedback to the user.

[0921] Input: Summary and explanation of technical terms.

[0922] Specific operation: The summary and explanation results are sent to the terminal and displayed to the user.

[0923] Output: Summary and explanation of technical terms displayed to the user.

[0924] Step 5:

[0925] The server uses an emotion engine to adjust the content and expression of summaries and explanations.

[0926] Input: User's emotional state data.

[0927] Specific operation: The emotion engine assesses the user's emotional state and adjusts the content and expression of the summary and explanation based on that assessment. For example, if the user is tired, it will provide a clear and concise summary.

[0928] Output: Adjusted summary and explanation of technical terms.

[0929] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[0930] Step 1:

[0931] The user enters the meeting details into a meeting request form and sends it from their device to the server.

[0932] Input: Details such as the meeting title, participant list, and preferred date and time options.

[0933] Specific operation: The user enters the required information into the meeting request form and sends it from their device to the server.

[0934] Output: Meeting details sent to the server.

[0935] Step 2:

[0936] The server uses the participants' calendar API to check their schedules.

[0937] Input: Participant list.

[0938] Specific operation: The server uses the Calendar API to check each participant's schedule.

[0939] Output: A list of available time slots for all participants.

[0940] Step 3:

[0941] The server will suggest the optimal meeting date and time.

[0942] Input: A list of available time slots for all participants.

[0943] Specific operation: The server calculates and proposes the optimal meeting date and time when all participants can attend.

[0944] Output: Proposed meeting date and time.

[0945] Step 4:

[0946] The server sends the adjustment results to the terminal and notifies the user.

[0947] Input: Proposed meeting date and time.

[0948] Specific operation: The adjustment results are sent to the terminal and notified to the user.

[0949] Output: The adjustment results notified to the user.

[0950] Step 5:

[0951] The server uses an emotion engine to analyze the emotional state of participants and suggest the optimal meeting date and time.

[0952] Input: Participant emotional state data.

[0953] Specific operation: The emotion engine analyzes the participants' busyness and stress levels and suggests the optimal meeting date and time, taking these factors into consideration.

[0954] Output: Results of adjusting the meeting date and time, taking emotional states into consideration.

[0955] The above describes the specific processing steps and operations related to the program of this system.

[0956] (Application Example 2)

[0957] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0958] In today's business environment, there is a growing demand for increased efficiency in various meetings and conferences. In particular, meetings conducted within autonomous mobile devices face challenges due to the lack of automation in tasks such as generating meeting minutes from audio data, task management, formatting checks for documents, and summarizing emails and documents while providing explanations of technical terms. Furthermore, the lack of means to provide appropriate feedback tailored to the emotional state of users impacts the quality and efficiency of work.

[0959] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for collecting audio data in real time within an autonomous mobile device, means for analyzing the user's emotions from the collected audio and text using an emotion recognition engine, means for providing appropriate feedback and summaries based on the user's emotional state, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This makes it possible to improve the efficiency and quality of meetings and conferences.

[0960] "Audio data" refers to data that records audio in digital format.

[0961] "Means of converting to text" refers to technology or equipment for analyzing audio data and converting it into text format.

[0962] "Means for extracting important information" refers to technologies or devices that automatically select key points or important information from text.

[0963] "Means for generating meeting minutes" refers to the technology or device that compiles extracted important information into a format suitable for meeting minutes.

[0964] "Means for automatically extracting tasks and adding them to a management system" refers to a technology or device that identifies tasks from meeting minutes and automatically registers them in a management system.

[0965] "Autonomous mobile devices" refer to all mobile devices that can move automatically without human intervention.

[0966] "Means for collecting audio data in real time" refers to technology or equipment for recording audio in real time within an autonomous mobile device.

[0967] An "emotion recognition engine" is a technology or software used to analyze a user's emotions from voice or text data.

[0968] "Means of providing feedback" refers to technologies or devices that communicate analysis results and support details to users.

[0969] "Methods for checking wording and formatting in real time" refers to technologies or devices that allow for real-time verification of the accuracy and formatting of text in documents and materials, and the provision of revision suggestions.

[0970] "Means for analyzing the content of emails and documents and generating summaries" refers to technology or devices that read the text of emails and documents and concisely summarize their main points.

[0971] "Means for detecting and providing explanations of technical terms" refers to technology or devices that find technical terms within a document and automatically generate explanations related to them.

[0972] "A means of checking participants' schedules and proposing the optimal meeting date and time" refers to a technology or device that obtains the schedule information of meeting participants and calculates the optimal meeting date and time.

[0973] "Means of notifying participants of the optimal meeting date and time" refers to technology or equipment for notifying meeting participants of the decided meeting date and time.

[0974] The system for realizing this invention consists of a combination of various software and hardware for collecting, converting, analyzing, and providing feedback on audio data. The following describes specific embodiments for implementing this invention.

[0975] First, when a user initiates a meeting or conference within the autonomous mobile device, a dedicated recording device is activated to collect audio data. This recording device includes a microphone and a speech recognition module, and collects audio data in real time. The collected audio data is then transmitted to a server via the terminal.

[0976] The server converts the collected audio data into text using a speech recognition API (e.g., Google Speech API). For example, audio data such as "In today's meeting, we discussed the project's progress. The person in charge decided to create documentation as the next step" would be identified as text.

[0977] Next, the server extracts key points from the text data using an automated summarization algorithm (e.g., Hugging Face's Transformers library) and generates a summary in the form of meeting minutes. This summary might look something like, "Project progress, responsible person created the document."

[0978] After the meeting minutes are generated, the server uses a task extraction algorithm to identify important tasks and automatically add them to the management system. For example, "The person in charge creates the document" is registered as a specific task in the management system.

[0979] An emotion recognition engine (e.g., Hugging Face's emotion recognition model) is also integrated into the server, analyzing the user's emotional state from collected voice and text data. This emotional information is reflected in meeting minutes and feedback, and is recorded, for example, as "the user is feeling stressed."

[0980] When creating documents, users create them on a dedicated application. This application checks the wording and formatting in real time and sends it to a server as needed to apply advanced natural language processing algorithms. For example, in response to the sentence "This document is very easy to understand," a suggested revision would be made to "This document is extremely easy to understand."

[0981] Furthermore, by uploading emails and documents sent by users to the server, automatic summarization and explanations of technical terms are performed. Based on the analysis results, the server provides easy-to-understand summaries and explanations of technical terms. For example, "document" might be explained as "document for a specific project."

[0982] When scheduling a meeting, the user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the most suitable meeting date and time. The decided meeting date and time are then notified to all participants.

[0983] Specific example

[0984] Meeting agenda:

[0985] "At today's meeting, we discussed the project's progress. The person in charge will now prepare the necessary documents as the next step."

[0986] Generated summary:

[0987] "Project progress: The person in charge creates the document."

[0988] Results of emotion recognition:

[0989] "Users are feeling stressed."

[0990] Example of a prompt

[0991] Meeting Minutes Generation: "Summarize the key points based on the following text: 'Today's meeting discussed project progress. The person in charge decided to prepare the document as the next step.'"

[0992] Sentiment Analysis: "Analyze the user's emotional state from the following text: 'In today's meeting, we discussed the project's progress. The person in charge decided to create a document as the next step.'"

[0993] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0994] Step 1:

[0995] Audio data is collected within the autonomous mobile device.

[0996] A dedicated recording device is activated to collect audio data from meetings and conferences in real time. The collected audio data is transmitted to the terminal via a microphone.

[0997] Input: Audio data of the user's conversation

[0998] Output: Audio data sent to the terminal

[0999] Step 2:

[1000] Convert audio data to text.

[1001] The device sends the collected audio data to the server, which then uses a speech recognition API (e.g., Google Speech API) to convert the audio data into text.

[1002] Input: Audio data sent from the device

[1003] Output: Text data

[1004] Step 3:

[1005] Extract important information from text data and generate a summary.

[1006] The server uses an automated summarization algorithm (e.g., Hugging Face's Transformers library) to extract key points from text data and generate a summary.

[1007] Input: Text data

[1008] Output: Summary text

[1009] Step 4:

[1010] Based on the summarized text, generate meeting minutes and extract tasks.

[1011] The server generates meeting minutes based on the generated summary text, identifies key tasks using a task extraction algorithm, and automatically adds them to the management system.

[1012] Input: Summary text

[1013] Output: Meeting minutes, task list

[1014] Step 5:

[1015] The system uses an emotion recognition engine to analyze the user's emotions.

[1016] The server uses an emotion recognition engine (e.g., Hugging Face's emotion recognition model) to analyze the user's emotional state from the collected audio and text data and record that state.

[1017] Input: Audio data, text data

[1018] Output: Emotional state

[1019] Step 6:

[1020] We provide users with analysis results and feedback.

[1021] The terminal receives meeting minutes, task lists, and emotional state analysis results sent from the server, and displays them on the user's screen.

[1022] Input: Meeting minutes, task list, emotional state

[1023] Output: Feedback information displayed on the user's screen

[1024] Step 7:

[1025] Check the wording and formatting of the document in real time.

[1026] When creating documents using a dedicated application, the wording and formatting are checked in real time, and if necessary, the documents are sent to a server where advanced natural language processing algorithms are applied to suggest revisions.

[1027] Input: Document being created

[1028] Output: Proposed revisions

[1029] Step 8:

[1030] It analyzes the content of emails and documents, and provides summaries and explanations of technical terms.

[1031] When a user uploads an email or document to the server, the server analyzes the content using automatic summarization and terminology detection algorithms, generating a summary and explanation.

[1032] Input: Email, documents

[1033] Output: Summary and explanation of technical terms

[1034] Step 9:

[1035] We will adjust the meeting schedule and propose the most suitable date and time.

[1036] The user fills out a meeting request form and sends it from their device to the server. The server uses the participants' calendar API to check their schedules, suggests the best meeting time, and notifies all participants.

[1037] Input: Meeting request form, participant calendar information

[1038] Output: Notification of optimal meeting date and time

[1039] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1040] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1041] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[1042] [Third Embodiment]

[1043] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[1044] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1045] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1046] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[1047] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1048] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1049] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1050] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1051] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1052] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1053] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1054] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[1055] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[1056] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[1057] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. It also uses a task extraction algorithm to automatically extract tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in Charge A created Document Y" is summarized and managed as specific tasks.

[1058] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[1059] When sending emails or documents, users upload the content they send to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1060] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the scheduling results, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1061] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[1062] The following describes the processing flow.

[1063] Meeting minutes and task management support during meetings.

[1064] Step 1:

[1065] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[1066] Step 2:

[1067] Terminal: Sends recorded audio data to the server.

[1068] Step 3:

[1069] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[1070] Step 4:

[1071] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[1072] Step 5:

[1073] Server: Use a task extraction algorithm to extract specific tasks from summarized meeting minutes and add them to the management system.

[1074] Step 6:

[1075] Server: Sends the generated meeting minutes and task list to the terminal.

[1076] Step 7:

[1077] Terminal: Displays received meeting minutes and task lists to the user.

[1078] Support for checking the wording and formatting when creating documents.

[1079] Step 1:

[1080] User: Create documents using a dedicated application.

[1081] Step 2:

[1082] Terminal: Performs real-time grammar checks and formatting consistency, and provides feedback to the user.

[1083] Step 3:

[1084] Terminal: Sends a draft of the created document to the server.

[1085] Step 4:

[1086] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[1087] Step 5:

[1088] Server: Sends check results and suggestions to the terminal.

[1089] Step 6:

[1090] Terminal: Displays the received check results and suggestions to the user.

[1091] Support for summarizing sent emails and documents, and explaining technical terms.

[1092] Step 1:

[1093] User: Upload emails and documents to the server.

[1094] Step 2:

[1095] Server: Analyzes the content of emails and documents using an automated summarization algorithm and a technical term detection algorithm.

[1096] Step 3:

[1097] Server: Generates a summary from the analysis results.

[1098] Step 4:

[1099] Server: Detects technical terms from analysis results and generates explanations.

[1100] Step 5:

[1101] Server: Sends the generated summary and explanations of technical terms to the terminal.

[1102] Step 6:

[1103] Terminal: Displays the received summary and explanations of technical terms to the user.

[1104] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1105] Step 1:

[1106] User: Enter the meeting title, participant list, and preferred date and time options in the meeting request form.

[1107] Step 2:

[1108] Terminal: Sends the entered information to the server.

[1109] Step 3:

[1110] Server: Uses the participant's calendar API to check each participant's availability.

[1111] Step 4:

[1112] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[1113] Step 5:

[1114] Server: Receive feedback from participants and make adjustments as needed.

[1115] Step 6:

[1116] Server: Notifies all participants of the finalized meeting date and time.

[1117] Step 7:

[1118] Terminal: Displays the finalized meeting date and time to the user.

[1119] The above outlines the specific processing steps for implementing the present invention. This system will improve the efficiency of business activities.

[1120] (Example 1)

[1121] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1122] In today's business environment, tasks such as conducting meetings, taking minutes, managing tasks, creating documents, and scheduling are complex and multifaceted challenges. In particular, there is a need for a way to efficiently automate tasks by converting vast amounts of audio data into text, extracting important information, and streamlining the process. Furthermore, it is necessary to improve work efficiency by analyzing and summarizing the content of user-generated documents and electronic messages, and providing explanations of technical terms. Additionally, a system that checks participants' schedules, automatically suggests optimal meeting times, and effectively coordinates them is crucial. There is a need for a system that can centrally address all of these challenges.

[1123] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1124] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of electronic messages and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This enables centralized management and efficiency of tasks, from the automatic processing of vast amounts of audio data to scheduling.

[1125] "Audio data" refers to conversations and voice information recorded in digital format.

[1126] "Text" refers to written information converted from audio data.

[1127] "Important information" refers to particularly noteworthy points extracted from the content of a meeting or similar event.

[1128] "Meeting minutes" are documents that record the content of meetings and other gatherings in text format.

[1129] "Tasks" refer to specific tasks and activities extracted from meeting minutes and registered in the management system.

[1130] A "management system" is a software system used to manage business operations in an organized manner.

[1131] "Wording" refers to the words and phrases used in a document.

[1132] "Format" refers to the way a document is presented in terms of its format and appearance.

[1133] "Real-time" refers to a situation where processing is performed instantly without delay.

[1134] "Electronic messages" refer to content communicated in digital format, such as email or chat messages.

[1135] A "summary" is a brief overview of a document, statement, or other similar content.

[1136] "Technical terms" are specific words or phrases used in a particular field.

[1137] "Explanation" refers to a commentary that clarifies the meaning and usage of technical terms.

[1138] "Participants" are people who attend meetings or similar events.

[1139] A "schedule" is a time-based plan based on the participants' schedules.

[1140] "Meeting date and time" refers to the date and time the meeting will be held.

[1141] A "notification" is a message or alert used to inform others of information.

[1142] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[1143] At the start of the system, the user first activates a dedicated recording device to collect audio data. This audio data is sent to the server via the terminal. The server uses a speech recognition API, such as Amazon Transcribe, to convert the audio data into text data. For example, audio data such as "At today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[1144] Next, the server uses an automated summarization algorithm (e.g., a BERT-based model) to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to automatically extract tasks from the meeting minutes and add them to the management system (e.g., JIRA). As a result, important information such as "Progress of Project X" and "Person A created Document Y" is summarized and managed as specific tasks.

[1145] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[1146] When sending emails or documents, the user uploads the content to the server. The server analyzes the content using an automatic summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1147] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it from their device to the server. The server uses participants' calendar APIs (e.g., Google Calendar API) to check their schedules and suggests the best meeting date and time. The scheduling results are notified to the device, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1148] Specific examples of prompt statements are as follows:

[1149] "Please convert the following audio data to text and generate a meeting minutes summary: 'Today's meeting agenda is the progress of Project X. It was decided that person A will create document Y as the next step.' Also, please register the proposed task in the management system."

[1150] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[1151] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1152] Step 1:

[1153] The user activates a dedicated recording device at the start of the meeting to collect audio data. The collected audio data is sent to the server via the terminal. Specifically, the user presses the record button, saves the recording data after the meeting ends, and uploads it to the server. In this process, the input is audio data, and the output is an audio file stored on the server.

[1154] Step 2:

[1155] The server uses a speech recognition API to convert the received audio data into text data. Specifically, the server sends the audio data to a speech recognition service such as Amazon Transcribe and receives the text data in return. In this process, the input is an audio file on the server, and the output is the text data of the audio.

[1156] Step 3:

[1157] The server uses an automated summarization algorithm to extract key points from the generated text data and produce a summary of the meeting minutes. Furthermore, it uses a task extraction algorithm to extract tasks from the meeting minutes and add them to the management system. Specifically, the server inputs text data into the summarization algorithm to extract key points. In this process, the input is text data generated by speech recognition, and the output is the summarized meeting minutes and tasks.

[1158] Step 4:

[1159] Users create documents using a dedicated application. This application checks grammar and formatting in real time and provides feedback to the user. Specifically, the user edits the document, and the application checks the entered text in real time. In this process, the input is the text data entered by the user, and the output is the feedback information.

[1160] Step 5:

[1161] When a user sends an email or document, they upload the content to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. Specifically, the server inputs the uploaded content into the analysis algorithm and generates the analysis results. In this process, the input is the uploaded content, and the output is the summary and explanations of technical terms.

[1162] Step 6:

[1163] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it from their device to the server. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. Specifically, the server uses the calendar API to check participants' schedules and notifies the user of the suggested date and time. In this process, the input is the data from the meeting request form, and the output is the suggested meeting date and time.

[1164] Step 7:

[1165] If final adjustments are needed, the user receives feedback from participants and makes further adjustments. The finalized meeting date and time are notified to all participants by the server. Specifically, the user inputs feedback, the server makes adjustments, and notifies the final decision. In this process, the input is feedback information, and the output is a notification of the finalized meeting date and time.

[1166] (Application Example 1)

[1167] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1168] Traditional meeting management systems often require manual processes for transcribing audio data, creating meeting minutes, extracting tasks, and scheduling meetings, which can be particularly inefficient in production meetings within factories. Furthermore, the lack of real-time minute generation and task management can lead to delays in post-meeting follow-up. This results in a decrease in overall operational efficiency. While advanced technologies such as automating audio data and scheduling are necessary, currently, no integrated system exists that combines these functions.

[1169] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1170] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of emails and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for checking the schedules of personnel participating in the meeting and adjusting the meeting time appropriately. This makes it possible to improve the efficiency of meetings.

[1171] "Audio data" refers to data that records audio in digital format.

[1172] "Text" refers to written information converted from audio data.

[1173] "Important information" refers to the key points or essential content that deserve particular attention during a meeting or discussion.

[1174] "Meeting minutes" refers to a document that records the content of a meeting or discussion.

[1175] A "task" refers to an individual task or activity that must be performed to achieve a specific objective.

[1176] A "management system" is software or a system used to comprehensively manage tasks, schedules, documents, and other similar items.

[1177] "Wording and style" refers to the wording, layout, and formatting used in a document.

[1178] A "summary" is a document or piece of information that concisely summarizes long texts or complex information.

[1179] "Technical terms" refer to special words or terms used in a particular field or area of ​​expertise.

[1180] "Explanation" refers to explaining specialized or complex topics in an easy-to-understand manner, or to the document containing such an explanation.

[1181] "Participants" refer to the people who attend a meeting or discussion.

[1182] A "schedule" is a plan that outlines the planned and scheduled times for specific activities or tasks.

[1183] The "optimal meeting date and time" refers to the most suitable date and time, taking into account the schedules of all meeting participants.

[1184] "Real-time" refers to processing and responding in accordance with the actual moment in which an event is unfolding.

[1185] An "advanced natural language processing algorithm" is an algorithm that uses advanced techniques to perform highly accurate semantic analysis and generation of text.

[1186] "Feedback" refers to the act of returning a response or evaluation of an activity or its results.

[1187] A system for implementing this invention can be realized with the following configuration and procedure.

[1188] 1. Collection of audio data and conversion to text:

[1189] The user first uses a dedicated device to collect audio data during meetings and discussions. A robot equipped with a microphone records the meeting audio, and this data is sent to a server via the terminal. The server uses a speech recognition API (for example, Google's speech recognition API) to convert the audio data into text.

[1190] 2. Extraction of important information and generation of meeting minutes:

[1191] The server extracts key information from the converted text using an automated summarization algorithm (such as Hugging Face's NLP model) and generates meeting minutes. These minutes include important points from the meeting and tasks to be taken as the next steps.

[1192] 3. Task extraction and addition to the management system:

[1193] An algorithm is applied to automatically extract tasks from meeting minutes, and the extracted tasks are added to the management system. This enables efficient task management.

[1194] 4. Real-time checking of wording and formatting:

[1195] When users create meeting materials or reports, they use a dedicated application. This application checks the wording and formatting in real time and provides feedback to the user. The check results are sent back to the server, where advanced natural language processing algorithms are applied.

[1196] 5. Content analysis and explanation of technical terms:

[1197] When emails or documents are uploaded to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. Simultaneously, an algorithm that detects technical terms is activated and provides explanations to the user.

[1198] 6. Scheduling and notifications:

[1199] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the best meeting date and time. Once the best date and time are determined, all participants are notified. Scheduling meetings is particularly important, as this improves productivity.

[1200] As a concrete example, the following prompt statement is used:

[1201] Audio recording: "At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step."

[1202] Summarized text: "Production line adjustments: Person in charge submits revised proposal."

[1203] Example of a prompt:

[1204] "Please summarize this meeting: 'Today's meeting discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[1205] "Extract the task from this text: 'At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[1206] In this way, the system implementing the present invention efficiently supports a wide range of tasks in business activities.

[1207] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1208] Step 1:

[1209] The user collects audio data using a dedicated device during meetings and discussions. A robot equipped with a microphone records the meeting audio and transmits the audio data to a server via a terminal. In this step, the audio data (input) is transmitted to the server (output).

[1210] Step 2:

[1211] The server uses a speech recognition API (for example, Google's speech recognition API) to convert the received audio data into text. This text conversion process transforms the audio data (input) into text data (output). This conversion includes parsing the audio data and generating a string.

[1212] Step 3:

[1213] The server analyzes the text data using an automated summarization algorithm (e.g., Hugging Face's NLP model) to extract important information from the text. In this step, the text data (input) is converted into summarized text (output). The process then extracts keywords and important phrases from the text.

[1214] Step 4:

[1215] The server generates meeting minutes and applies an algorithm to extract tasks from those minutes. Here, summarized text (input) is generated as meeting minutes (output), and tasks (output) are further extracted. The extracted tasks are added to the management system.

[1216] Step 5:

[1217] A dedicated application used by users to create documents checks the wording and formatting in real time and provides feedback. The application analyzes text data (input) and provides suggested wording revisions and formatting check results (output). The feedback content is generated and notified to the user.

[1218] Step 6:

[1219] When a user uploads emails or documents to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. A technical term detection algorithm is also applied to generate explanations of technical terms. In this process, the uploaded content (input) is transformed into a summary and explanations of technical terms (output).

[1220] Step 7:

[1221] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the most suitable meeting date and time. Here, the meeting request information (input) is suggested as the optimal meeting date and time (output).

[1222] Step 8:

[1223] The server notifies all participants of the optimal meeting date and time. Email and calendar app APIs are used for notification. This ensures that the meeting date and time suggested by the server (input) is notified to all participants (output).

[1224] Step 9:

[1225] After the meeting, the server adds tasks extracted from the generated meeting minutes to the management system and manages their schedules. Here, the task information (input) from the meeting minutes is added to the task list (output) in the management system, and the schedule is updated.

[1226] Detailed processing at each step automates the process, allowing it to proceed more efficiently.

[1227] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1228] This invention is a system that combines the generation of meeting minutes from audio data, task management, checking the wording and formatting of documents, summarizing emails and documents and providing explanations of technical terms, and adjusting meeting schedules with an emotion engine that recognizes the user's emotions. Specific embodiments for carrying out this invention are described below.

[1229] Meeting minutes and task management support during meetings.

[1230] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[1231] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[1232] Combination with the emotion engine

[1233] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[1234] Support for checking the wording and formatting when creating documents.

[1235] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[1236] Combination with the emotion engine

[1237] The emotion engine recognizes the user's emotional state and provides feedback that takes this into account if stress or anxiety is detected. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[1238] Support for summarizing sent emails and documents, and explaining technical terms.

[1239] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1240] Combination with the emotion engine

[1241] The emotion engine evaluates the user's emotional state when analyzing emails and documents, and adjusts the content and expression of summaries and explanations based on that evaluation. For example, if it determines that "the user is tired," it can provide a clear and concise summary.

[1242] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1243] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1244] Combination with the emotion engine

[1245] The emotion engine analyzes participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[1246] As described above, the present invention is a system that combines an emotion engine to significantly improve work efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[1247] The following describes the processing flow.

[1248] Meeting minutes and task management support during meetings.

[1249] Step 1:

[1250] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[1251] Step 2:

[1252] Terminal: Sends recorded audio data to the server.

[1253] Step 3:

[1254] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[1255] Step 4:

[1256] Server: Uses an emotion engine to analyze the user's emotional state and adds relevant information to the text data.

[1257] Step 5:

[1258] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[1259] Step 6:

[1260] Server: Use a task extraction algorithm to extract specific tasks from meeting minutes and add them to the management system.

[1261] Step 7:

[1262] Server: Sends the generated meeting minutes and task list to the terminal.

[1263] Step 8:

[1264] Terminal: Displays received meeting minutes and task lists to the user.

[1265] Support for checking the wording and formatting when creating documents.

[1266] Step 1:

[1267] User: Create documents using a dedicated application.

[1268] Step 2:

[1269] Terminal: Performs grammar checks and formatting consistency in real time, and provides feedback of the check results to the user.

[1270] Step 3:

[1271] Terminal: Uses an emotion engine to analyze the user's emotional state and reflect it in the check feedback.

[1272] Step 4:

[1273] Terminal: Sends a draft of the created document to the server.

[1274] Step 5:

[1275] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[1276] Step 6:

[1277] Server: Sends check results and suggestions to the terminal.

[1278] Step 7:

[1279] Terminal: Displays the received check results and suggestions to the user.

[1280] Support for summarizing sent emails and documents, and explaining technical terms.

[1281] Step 1:

[1282] User: Upload emails and documents to the server.

[1283] Step 2:

[1284] Server: Analyzes the content of emails and documents using an automated summarization algorithm and a technical term detection algorithm.

[1285] Step 3:

[1286] Server: Generates a summary from the analysis results.

[1287] Step 4:

[1288] Server: Uses an emotion engine to adjust the content and expression of summaries and explanations according to the user's emotional state.

[1289] Step 5:

[1290] Server: Detects technical terms from analysis results and generates explanations.

[1291] Step 6:

[1292] Server: Sends the generated summary and explanations of technical terms to the terminal.

[1293] Step 7:

[1294] Terminal: Displays the received summary and explanations of technical terms to the user.

[1295] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1296] Step 1:

[1297] User: Enter the meeting title, participant list, and preferred date and time options in the meeting request form.

[1298] Step 2:

[1299] Terminal: Sends the entered information to the server.

[1300] Step 3:

[1301] Server: Uses the participant's calendar API to check each participant's availability.

[1302] Step 4:

[1303] Server: Uses an emotion engine to analyze participants' emotional states based on historical data and current conditions.

[1304] Step 5:

[1305] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[1306] Step 6:

[1307] Server: Receive feedback from participants and make adjustments as needed.

[1308] Step 7:

[1309] Server: Notifies all participants of the finalized meeting date and time.

[1310] Step 8:

[1311] Terminal: Displays the finalized meeting date and time to the user.

[1312] The above outlines the specific processing steps of the present invention, which incorporates an emotion engine. This system enables the entire business process to proceed efficiently and provides meticulous support that takes into account the user's emotions.

[1313] (Example 2)

[1314] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1315] Traditional business support systems, such as those for meeting minute generation, task management, document creation, email summarization, and meeting scheduling, are often provided individually, resulting in problems with the time and effort required for integrated operation and management. Furthermore, the difficulty in providing feedback and support that takes into account the user's emotional state posed a risk of decreased work efficiency and user satisfaction.

[1316] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting audio data, means for transmitting audio data to the server, means for converting audio data to text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for monitoring the user's emotional state, means for reflecting the user's emotional information in the meeting minutes, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for adjusting the meeting date and time considering the user's emotional state. As a result, the user can process tasks integrally and efficiently with a single system, and furthermore, receiving feedback and support that takes into account their emotional state improves work efficiency and increases user satisfaction.

[1317] "Audio data" refers to data collected using recording devices, such as audio from meetings or conferences.

[1318] A "server" is a computer system that processes, records, analyzes, and notifies audio data.

[1319] A "speech recognition API" is an application programming interface for converting speech data into text data.

[1320] "Text data" refers to character information converted using a speech recognition API.

[1321] "Meeting minutes" are documents that record the content of meetings or discussions.

[1322] A "task management system" is a system for managing extracted tasks and tracking their progress.

[1323] An "emotion engine" is software that analyzes a user's emotions from their voice, facial expressions, and other data.

[1324] "Grammar check" is the process of verifying whether the wording in a document conforms to linguistic rules.

[1325] "Format checking" is the process of verifying whether a document adheres to a certain format and layout.

[1326] An "automatic summarization algorithm" is a program that extracts important parts from long texts and summarizes them concisely.

[1327] A "technical term detection algorithm" is a program that identifies technical terms within a document and provides their definitions and explanations.

[1328] The "Calendar API" is an application programming interface for checking each participant's schedule.

[1329] "Feedback" refers to the evaluations and improvement suggestions that a system provides to its users.

[1330] Modes for carrying out the invention

[1331] This invention is a system that combines the generation of meeting minutes from audio data, task management, text formatting checks for document creation, summarization of emails and documents, explanation of technical terms, and scheduling of meetings with an emotion engine that recognizes user emotions. Specific embodiments for carrying out this invention are described below.

[1332] Meeting minutes and task management support during meetings.

[1333] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[1334] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[1335] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[1336] Support for checking the wording and formatting when creating documents.

[1337] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[1338] The server uses an emotion engine to recognize the user's emotional state and provides feedback that takes stress or anxiety into account. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[1339] Support for summarizing sent emails and documents, and explaining technical terms.

[1340] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1341] The server uses an emotion engine to assess the user's emotional state when analyzing emails and documents, and adjusts the content and wording of summaries and explanations based on that assessment. For example, if it determines that the user is tired, it can provide a clear and concise summary.

[1342] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1343] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1344] The server uses an emotion engine to analyze participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[1345] Examples of specific cases and prompt statements

[1346] The following prompt statements are possible as concrete examples of how this system is operated:

[1347] 1. "Record meetings and automatically generate meeting minutes and tasks. Analyze user sentiment as well."

[1348] 2. "Please check the materials in real time and suggest appropriate wording and formatting. We will also take user sentiment into consideration."

[1349] 3. "Upload emails and documents, and generate summaries of the content and explanations of technical terms. User sentiment will also be reflected as appropriate."

[1350] 4. "Please coordinate the meeting and propose the most suitable date and time. Please also consider the feelings of the participants."

[1351] As described above, the system of the present invention provides an integrated solution that combines an emotion engine to significantly improve operational efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[1352] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1353] Meeting minutes and task management support during meetings.

[1354] Step 1:

[1355] The user activates the recording device and collects audio data.

[1356] Input: Audio data of a meeting initiated by the user.

[1357] Specific operation: The user manually activates a dedicated recording device and uses a high-sensitivity microphone to collect audio from the meeting.

[1358] Output: Recorded audio data.

[1359] Step 2:

[1360] The device sends voice data to the server.

[1361] Input: Audio data collected by a recording device.

[1362] Specific operation: The device uploads the recorded audio data to the server using Wi-Fi or mobile data communication.

[1363] Output: Audio data uploaded to the server.

[1364] Step 3:

[1365] The server uses a speech recognition API to convert the audio data into text.

[1366] Input: Audio data uploaded to the server.

[1367] Specific operation: The server uses speech recognition APIs such as the Google Cloud Speech-to-Text API to convert speech data into text data.

[1368] Output: Converted text data.

[1369] Step 4:

[1370] The server extracts key points from the text and generates a summary of the meeting minutes.

[1371] Input: Converted text data.

[1372] Specific operation: The server uses natural language processing algorithms to extract important information from text data and generate a summary.

[1373] Output: A summary of the generated meeting minutes.

[1374] Step 5:

[1375] The server generates tasks using a task extraction algorithm and adds them to the management system.

[1376] Input: A summary of the generated meeting minutes.

[1377] Specific operation: The server applies a task extraction algorithm to extract specific tasks from the summary and add them to the management system.

[1378] Output: Tasks added to the management system.

[1379] Step 6:

[1380] The server uses an emotion engine to monitor the user's emotional state and reflect it in the meeting minutes.

[1381] Input: Audio recordings, facial expressions, and text input during meetings.

[1382] Specific operation: The emotion engine analyzes this data and recognizes the user's emotional state.

[1383] Output: Meeting minutes containing emotional information.

[1384] Support for checking the wording and formatting when creating documents.

[1385] Step 1:

[1386] Users create documents using a dedicated application.

[1387] Input: Drafts or documents of materials.

[1388] Specific operation: The user creates a document using a document creation application, and grammar and formatting checks are performed in real time.

[1389] Output: Draft version of the document.

[1390] Step 2:

[1391] The terminal sends a draft of the document to the server.

[1392] Input: Draft version of the document.

[1393] Specific operation: The terminal temporarily stores the draft of the document on the server and then securely transmits it.

[1394] Output: Draft version of the document sent to the server.

[1395] Step 3:

[1396] The server checks the wording and formatting and suggests revisions.

[1397] Input: Draft document sent to the server.

[1398] Specific operation: The server applies advanced natural language processing algorithms to check the wording and formatting. For example, it might suggest changing "This document is very easy to understand" to "This document is extremely easy to understand."

[1399] Output: Check results of the document including the proposed revisions.

[1400] Step 4:

[1401] The server sends the correction results to the terminal and provides feedback to the user.

[1402] Input: Check results of the document containing the proposed revisions.

[1403] Specific operation: The check results are reflected in real time on the interface of the dedicated application.

[1404] Output: Correction results provided to the user as feedback.

[1405] Step 5:

[1406] The server uses an emotion engine to adjust correction suggestions based on the user's emotions.

[1407] Input: User emotional state data (e.g., whether the user is tense, stressed, etc.).

[1408] Specific operation: The emotion engine analyzes emotional data and adjusts the suggested corrections based on the analysis results. For example, if the user is feeling stressed, it will suggest corrections using gentler language.

[1409] Output: Revised correction suggestions.

[1410] Support for summarizing sent emails and documents, and explaining technical terms.

[1411] Step 1:

[1412] Users upload emails and documents to the server.

[1413] Input: Emails or documents to send.

[1414] Specific operation: Users use a dedicated upload form to upload emails and documents to the server.

[1415] Output: Emails and documents uploaded to the server.

[1416] Step 2:

[1417] The server performs the analysis using an automatic summarization algorithm and a technical term detection algorithm.

[1418] Input: Emails and documents uploaded to the server.

[1419] Specific operation: The server uses an automatic summarization algorithm to summarize the content and a terminology detection algorithm to extract technical terms.

[1420] Output: Summary and list of technical terms.

[1421] Step 3:

[1422] The server generates a summary and explanations of technical terms.

[1423] Input: Summary and list of technical terms.

[1424] Specific operation: The server generates explanations of technical terms and formats them together with summaries of emails and documents.

[1425] Output: Summary and explanation of technical terms.

[1426] Step 4:

[1427] The server sends the generated summary and explanation to the terminal and provides feedback to the user.

[1428] Input: Summary and explanation of technical terms.

[1429] Specific operation: The summary and explanation results are sent to the terminal and displayed to the user.

[1430] Output: Summary and explanation of technical terms displayed to the user.

[1431] Step 5:

[1432] The server uses an emotion engine to adjust the content and expression of summaries and explanations.

[1433] Input: User's emotional state data.

[1434] Specific operation: The emotion engine assesses the user's emotional state and adjusts the content and expression of the summary and explanation based on that assessment. For example, if the user is tired, it will provide a clear and concise summary.

[1435] Output: Adjusted summary and explanation of technical terms.

[1436] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1437] Step 1:

[1438] The user enters the meeting details into a meeting request form and sends it from their device to the server.

[1439] Input: Details such as the meeting title, participant list, and preferred date and time options.

[1440] Specific operation: The user enters the required information into the meeting request form and sends it from their device to the server.

[1441] Output: Meeting details sent to the server.

[1442] Step 2:

[1443] The server uses the participants' calendar API to check their schedules.

[1444] Input: Participant list.

[1445] Specific operation: The server uses the Calendar API to check each participant's schedule.

[1446] Output: A list of available time slots for all participants.

[1447] Step 3:

[1448] The server will suggest the optimal meeting date and time.

[1449] Input: A list of available time slots for all participants.

[1450] Specific operation: The server calculates and proposes the optimal meeting date and time when all participants can attend.

[1451] Output: Proposed meeting date and time.

[1452] Step 4:

[1453] The server sends the adjustment results to the terminal and notifies the user.

[1454] Input: Proposed meeting date and time.

[1455] Specific operation: The adjustment results are sent to the terminal and notified to the user.

[1456] Output: The adjustment results notified to the user.

[1457] Step 5:

[1458] The server uses an emotion engine to analyze the emotional state of participants and suggest the optimal meeting date and time.

[1459] Input: Participant emotional state data.

[1460] Specific operation: The emotion engine analyzes the participants' busyness and stress levels and suggests the optimal meeting date and time, taking these factors into consideration.

[1461] Output: Results of adjusting the meeting date and time, taking emotional states into consideration.

[1462] The above describes the specific processing steps and operations related to the program of this system.

[1463] (Application Example 2)

[1464] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1465] In today's business environment, there is a growing demand for increased efficiency in various meetings and conferences. In particular, meetings conducted within autonomous mobile devices face challenges due to the lack of automation in tasks such as generating meeting minutes from audio data, task management, formatting checks for documents, and summarizing emails and documents while providing explanations of technical terms. Furthermore, the lack of means to provide appropriate feedback tailored to the emotional state of users impacts the quality and efficiency of work.

[1466] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for collecting audio data in real time within an autonomous mobile device, means for analyzing the user's emotions from the collected audio and text using an emotion recognition engine, means for providing appropriate feedback and summaries based on the user's emotional state, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This makes it possible to improve the efficiency and quality of meetings and conferences.

[1467] "Audio data" refers to data that records audio in digital format.

[1468] "Means of converting to text" refers to technology or equipment for analyzing audio data and converting it into text format.

[1469] "Means for extracting important information" refers to technologies or devices that automatically select key points or important information from text.

[1470] "Means for generating meeting minutes" refers to the technology or device that compiles extracted important information into a format suitable for meeting minutes.

[1471] "Means for automatically extracting tasks and adding them to a management system" refers to a technology or device that identifies tasks from meeting minutes and automatically registers them in a management system.

[1472] "Autonomous mobile devices" refer to all mobile devices that can move automatically without human intervention.

[1473] "Means for collecting audio data in real time" refers to technology or equipment for recording audio in real time within an autonomous mobile device.

[1474] An "emotion recognition engine" is a technology or software used to analyze a user's emotions from voice or text data.

[1475] "Means of providing feedback" refers to technologies or devices that communicate analysis results and support details to users.

[1476] "Methods for checking wording and formatting in real time" refers to technologies or devices that allow for real-time verification of the accuracy and formatting of text in documents and materials, and the provision of revision suggestions.

[1477] "Means for analyzing the content of emails and documents and generating summaries" refers to technology or devices that read the text of emails and documents and concisely summarize their main points.

[1478] "Means for detecting and providing explanations of technical terms" refers to technology or devices that find technical terms within a document and automatically generate explanations related to them.

[1479] "A means of checking participants' schedules and proposing the optimal meeting date and time" refers to a technology or device that obtains the schedule information of meeting participants and calculates the optimal meeting date and time.

[1480] "Means of notifying participants of the optimal meeting date and time" refers to technology or equipment for notifying meeting participants of the decided meeting date and time.

[1481] The system for realizing this invention consists of a combination of various software and hardware for collecting, converting, analyzing, and providing feedback on audio data. The following describes specific embodiments for implementing this invention.

[1482] First, when a user initiates a meeting or conference within the autonomous mobile device, a dedicated recording device is activated to collect audio data. This recording device includes a microphone and a speech recognition module, and collects audio data in real time. The collected audio data is then transmitted to a server via the terminal.

[1483] The server converts the collected audio data into text using a speech recognition API (e.g., Google Speech API). For example, audio data such as "In today's meeting, we discussed the project's progress. The person in charge decided to create documentation as the next step" would be identified as text.

[1484] Next, the server extracts key points from the text data using an automated summarization algorithm (e.g., Hugging Face's Transformers library) and generates a summary in the form of meeting minutes. This summary might look something like, "Project progress, responsible person created the document."

[1485] After the meeting minutes are generated, the server uses a task extraction algorithm to identify important tasks and automatically add them to the management system. For example, "The person in charge creates the document" is registered as a specific task in the management system.

[1486] An emotion recognition engine (e.g., Hugging Face's emotion recognition model) is also integrated into the server, analyzing the user's emotional state from collected voice and text data. This emotional information is reflected in meeting minutes and feedback, and is recorded, for example, as "the user is feeling stressed."

[1487] When creating documents, users create them on a dedicated application. This application checks the wording and formatting in real time and sends it to a server as needed to apply advanced natural language processing algorithms. For example, in response to the sentence "This document is very easy to understand," a suggested revision would be made to "This document is extremely easy to understand."

[1488] Furthermore, by uploading emails and documents sent by users to the server, automatic summarization and explanations of technical terms are performed. Based on the analysis results, the server provides easy-to-understand summaries and explanations of technical terms. For example, "document" might be explained as "document for a specific project."

[1489] When scheduling a meeting, the user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the most suitable meeting date and time. The decided meeting date and time are then notified to all participants.

[1490] Specific example

[1491] Meeting agenda:

[1492] "At today's meeting, we discussed the project's progress. The person in charge will now prepare the necessary documents as the next step."

[1493] Generated summary:

[1494] "Project progress: The person in charge creates the document."

[1495] Results of emotion recognition:

[1496] "Users are feeling stressed."

[1497] Example of a prompt

[1498] Meeting Minutes Generation: "Summarize the key points based on the following text: 'Today's meeting discussed project progress. The person in charge decided to prepare the document as the next step.'"

[1499] Sentiment Analysis: "Analyze the user's emotional state from the following text: 'In today's meeting, we discussed the project's progress. The person in charge decided to create a document as the next step.'"

[1500] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1501] Step 1:

[1502] Audio data is collected within the autonomous mobile device.

[1503] A dedicated recording device is activated to collect audio data from meetings and conferences in real time. The collected audio data is transmitted to the terminal via a microphone.

[1504] Input: Audio data of the user's conversation

[1505] Output: Audio data sent to the terminal

[1506] Step 2:

[1507] Convert audio data to text.

[1508] The device sends the collected audio data to the server, which then uses a speech recognition API (e.g., Google Speech API) to convert the audio data into text.

[1509] Input: Audio data sent from the device

[1510] Output: Text data

[1511] Step 3:

[1512] Extract important information from text data and generate a summary.

[1513] The server uses an automated summarization algorithm (e.g., Hugging Face's Transformers library) to extract key points from text data and generate a summary.

[1514] Input: Text data

[1515] Output: Summary text

[1516] Step 4:

[1517] Based on the summarized text, generate meeting minutes and extract tasks.

[1518] The server generates meeting minutes based on the generated summary text, identifies key tasks using a task extraction algorithm, and automatically adds them to the management system.

[1519] Input: Summary text

[1520] Output: Meeting minutes, task list

[1521] Step 5:

[1522] The system uses an emotion recognition engine to analyze the user's emotions.

[1523] The server uses an emotion recognition engine (e.g., Hugging Face's emotion recognition model) to analyze the user's emotional state from the collected audio and text data and record that state.

[1524] Input: Audio data, text data

[1525] Output: Emotional state

[1526] Step 6:

[1527] We provide users with analysis results and feedback.

[1528] The terminal receives meeting minutes, task lists, and emotional state analysis results sent from the server, and displays them on the user's screen.

[1529] Input: Meeting minutes, task list, emotional state

[1530] Output: Feedback information displayed on the user's screen

[1531] Step 7:

[1532] Check the wording and formatting of the document in real time.

[1533] When creating documents using a dedicated application, the wording and formatting are checked in real time, and if necessary, the documents are sent to a server where advanced natural language processing algorithms are applied to suggest revisions.

[1534] Input: Document being created

[1535] Output: Proposed revisions

[1536] Step 8:

[1537] It analyzes the content of emails and documents, and provides summaries and explanations of technical terms.

[1538] When a user uploads an email or document to the server, the server analyzes the content using automatic summarization and terminology detection algorithms, generating a summary and explanation.

[1539] Input: Email, documents

[1540] Output: Summary and explanation of technical terms

[1541] Step 9:

[1542] We will adjust the meeting schedule and propose the most suitable date and time.

[1543] The user fills out a meeting request form and sends it from their device to the server. The server uses the participants' calendar API to check their schedules, suggests the best meeting time, and notifies all participants.

[1544] Input: Meeting request form, participant calendar information

[1545] Output: Notification of optimal meeting date and time

[1546] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1547] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1548] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1549] [Fourth Embodiment]

[1550] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1551] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1552] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1553] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1554] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1555] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1556] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1557] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1558] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1559] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1560] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1561] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1562] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1563] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[1564] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[1565] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. It also uses a task extraction algorithm to automatically extract tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in Charge A created Document Y" is summarized and managed as specific tasks.

[1566] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[1567] When sending emails or documents, users upload the content they send to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1568] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the scheduling results, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1569] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[1570] The following describes the processing flow.

[1571] Meeting minutes and task management support during meetings.

[1572] Step 1:

[1573] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[1574] Step 2:

[1575] Terminal: Sends recorded audio data to the server.

[1576] Step 3:

[1577] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[1578] Step 4:

[1579] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[1580] Step 5:

[1581] Server: Use a task extraction algorithm to extract specific tasks from summarized meeting minutes and add them to the management system.

[1582] Step 6:

[1583] Server: Sends the generated meeting minutes and task list to the terminal.

[1584] Step 7:

[1585] Terminal: Displays received meeting minutes and task lists to the user.

[1586] Support for checking the wording and formatting when creating documents.

[1587] Step 1:

[1588] User: Create documents using a dedicated application.

[1589] Step 2:

[1590] Terminal: Performs real-time grammar checks and formatting consistency, and provides feedback to the user.

[1591] Step 3:

[1592] Terminal: Sends a draft of the created document to the server.

[1593] Step 4:

[1594] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[1595] Step 5:

[1596] Server: Sends check results and suggestions to the terminal.

[1597] Step 6:

[1598] Terminal: Displays the received check results and suggestions to the user.

[1599] Support for summarizing sent emails and documents, and explaining technical terms.

[1600] Step 1:

[1601] User: Upload emails and documents to the server.

[1602] Step 2:

[1603] Server: Analyzes the content of emails and documents using an automated summarization algorithm and a technical term detection algorithm.

[1604] Step 3:

[1605] Server: Generates a summary from the analysis results.

[1606] Step 4:

[1607] Server: Detects technical terms from analysis results and generates explanations.

[1608] Step 5:

[1609] Server: Sends the generated summary and explanations of technical terms to the terminal.

[1610] Step 6:

[1611] Terminal: Displays the received summary and explanations of technical terms to the user.

[1612] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1613] Step 1:

[1614] User: Enter the meeting title, participant list, and preferred date and time options in the meeting request form.

[1615] Step 2:

[1616] Terminal: Sends the entered information to the server.

[1617] Step 3:

[1618] Server: Uses the participant's calendar API to check each participant's availability.

[1619] Step 4:

[1620] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[1621] Step 5:

[1622] Server: Receive feedback from participants and make adjustments as needed.

[1623] Step 6:

[1624] Server: Notifies all participants of the finalized meeting date and time.

[1625] Step 7:

[1626] Terminal: Displays the finalized meeting date and time to the user.

[1627] The above outlines the specific processing steps for implementing the present invention. This system will improve the efficiency of business activities.

[1628] (Example 1)

[1629] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1630] In today's business environment, tasks such as conducting meetings, taking minutes, managing tasks, creating documents, and scheduling are complex and multifaceted challenges. In particular, there is a need for a way to efficiently automate tasks by converting vast amounts of audio data into text, extracting important information, and streamlining the process. Furthermore, it is necessary to improve work efficiency by analyzing and summarizing the content of user-generated documents and electronic messages, and providing explanations of technical terms. Additionally, a system that checks participants' schedules, automatically suggests optimal meeting times, and effectively coordinates them is crucial. There is a need for a system that can centrally address all of these challenges.

[1631] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1632] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of electronic messages and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This enables centralized management and efficiency of tasks, from the automatic processing of vast amounts of audio data to scheduling.

[1633] "Audio data" refers to conversations and voice information recorded in digital format.

[1634] "Text" refers to written information converted from audio data.

[1635] "Important information" refers to particularly noteworthy points extracted from the content of a meeting or similar event.

[1636] "Meeting minutes" are documents that record the content of meetings and other gatherings in text format.

[1637] "Tasks" refer to specific tasks and activities extracted from meeting minutes and registered in the management system.

[1638] A "management system" is a software system used to manage business operations in an organized manner.

[1639] "Wording" refers to the words and phrases used in a document.

[1640] "Format" refers to the way a document is presented in terms of its format and appearance.

[1641] "Real-time" refers to a situation where processing is performed instantly without delay.

[1642] "Electronic messages" refer to content communicated in digital format, such as email or chat messages.

[1643] A "summary" is a brief overview of a document, statement, or other similar content.

[1644] "Technical terms" are specific words or phrases used in a particular field.

[1645] "Explanation" refers to a commentary that clarifies the meaning and usage of technical terms.

[1646] "Participants" are people who attend meetings or similar events.

[1647] A "schedule" is a time-based plan based on the participants' schedules.

[1648] "Meeting date and time" refers to the date and time the meeting will be held.

[1649] A "notification" is a message or alert used to inform others of information.

[1650] This invention is a system that supports a wide range of business activities, from processing audio data to scheduling meetings. The specific implementation of this system will be described below.

[1651] At the start of the system, the user first activates a dedicated recording device to collect audio data. This audio data is sent to the server via the terminal. The server uses a speech recognition API, such as Amazon Transcribe, to convert the audio data into text data. For example, audio data such as "At today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step" would be converted into text.

[1652] Next, the server uses an automated summarization algorithm (e.g., a BERT-based model) to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to automatically extract tasks from the meeting minutes and add them to the management system (e.g., JIRA). As a result, important information such as "Progress of Project X" and "Person A created Document Y" is summarized and managed as specific tasks.

[1653] When creating documents, users use a dedicated application. This application checks grammar and formatting consistency in real time and provides feedback to the user. Furthermore, the draft of the document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, a user might be suggested to revise a sentence like "This document is very easy to understand" to "This document is extremely easy to understand."

[1654] When sending emails or documents, the user uploads the content to the server. The server analyzes the content using an automatic summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1655] For scheduling meetings, users enter the meeting title, participant list, and preferred date and time into a meeting request form and send it from their device to the server. The server uses participants' calendar APIs (e.g., Google Calendar API) to check their schedules and suggests the best meeting date and time. The scheduling results are notified to the device, and further adjustments are made as needed. The final meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, determines and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1656] Specific examples of prompt statements are as follows:

[1657] "Please convert the following audio data to text and generate a meeting minutes summary: 'Today's meeting agenda is the progress of Project X. It was decided that person A will create document Y as the next step.' Also, please register the proposed task in the management system."

[1658] Thus, the system of the present invention enables the efficient execution of a series of tasks, from generating meeting minutes from audio data to scheduling, thereby significantly improving operational efficiency.

[1659] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1660] Step 1:

[1661] The user activates a dedicated recording device at the start of the meeting to collect audio data. The collected audio data is sent to the server via the terminal. Specifically, the user presses the record button, saves the recording data after the meeting ends, and uploads it to the server. In this process, the input is audio data, and the output is an audio file stored on the server.

[1662] Step 2:

[1663] The server uses a speech recognition API to convert the received audio data into text data. Specifically, the server sends the audio data to a speech recognition service such as Amazon Transcribe and receives the text data in return. In this process, the input is an audio file on the server, and the output is the text data of the audio.

[1664] Step 3:

[1665] The server uses an automated summarization algorithm to extract key points from the generated text data and produce a summary of the meeting minutes. Furthermore, it uses a task extraction algorithm to extract tasks from the meeting minutes and add them to the management system. Specifically, the server inputs text data into the summarization algorithm to extract key points. In this process, the input is text data generated by speech recognition, and the output is the summarized meeting minutes and tasks.

[1666] Step 4:

[1667] Users create documents using a dedicated application. This application checks grammar and formatting in real time and provides feedback to the user. Specifically, the user edits the document, and the application checks the entered text in real time. In this process, the input is the text data entered by the user, and the output is the feedback information.

[1668] Step 5:

[1669] When a user sends an email or document, they upload the content to the server. The server uses an automatic summarization algorithm and a technical term detection algorithm to analyze the content and generate a summary and explanations of technical terms. Specifically, the server inputs the uploaded content into the analysis algorithm and generates the analysis results. In this process, the input is the uploaded content, and the output is the summary and explanations of technical terms.

[1670] Step 6:

[1671] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it from their device to the server. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. Specifically, the server uses the calendar API to check participants' schedules and notifies the user of the suggested date and time. In this process, the input is the data from the meeting request form, and the output is the suggested meeting date and time.

[1672] Step 7:

[1673] If final adjustments are needed, the user receives feedback from participants and makes further adjustments. The finalized meeting date and time are notified to all participants by the server. Specifically, the user inputs feedback, the server makes adjustments, and notifies the final decision. In this process, the input is feedback information, and the output is a notification of the finalized meeting date and time.

[1674] (Application Example 1)

[1675] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1676] Traditional meeting management systems often require manual processes for transcribing audio data, creating meeting minutes, extracting tasks, and scheduling meetings, which can be particularly inefficient in production meetings within factories. Furthermore, the lack of real-time minute generation and task management can lead to delays in post-meeting follow-up. This results in a decrease in overall operational efficiency. While advanced technologies such as automating audio data and scheduling are necessary, currently, no integrated system exists that combines these functions.

[1677] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1678] In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for checking the wording and format of documents in real time, means for analyzing the content of emails and documents and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for checking the schedules of personnel participating in the meeting and adjusting the meeting time appropriately. This makes it possible to improve the efficiency of meetings.

[1679] "Audio data" refers to data that records audio in digital format.

[1680] "Text" refers to written information converted from audio data.

[1681] "Important information" refers to the key points or essential content that deserve particular attention during a meeting or discussion.

[1682] "Meeting minutes" refers to a document that records the content of a meeting or discussion.

[1683] A "task" refers to an individual task or activity that must be performed to achieve a specific objective.

[1684] A "management system" is software or a system used to comprehensively manage tasks, schedules, documents, and other similar items.

[1685] "Wording and style" refers to the wording, layout, and formatting used in a document.

[1686] A "summary" is a document or piece of information that concisely summarizes long texts or complex information.

[1687] "Technical terms" refer to special words or terms used in a particular field or area of ​​expertise.

[1688] "Explanation" refers to explaining specialized or complex topics in an easy-to-understand manner, or to the document containing such an explanation.

[1689] "Participants" refer to the people who attend a meeting or discussion.

[1690] A "schedule" is a plan that outlines the planned and scheduled times for specific activities or tasks.

[1691] The "optimal meeting date and time" refers to the most suitable date and time, taking into account the schedules of all meeting participants.

[1692] "Real-time" refers to processing and responding in accordance with the actual moment in which an event is unfolding.

[1693] An "advanced natural language processing algorithm" is an algorithm that uses advanced techniques to perform highly accurate semantic analysis and generation of text.

[1694] "Feedback" refers to the act of returning a response or evaluation of an activity or its results.

[1695] A system for implementing this invention can be realized with the following configuration and procedure.

[1696] 1. Collection of audio data and conversion to text:

[1697] The user first uses a dedicated device to collect audio data during meetings and discussions. A robot equipped with a microphone records the meeting audio, and this data is sent to a server via the terminal. The server uses a speech recognition API (for example, Google's speech recognition API) to convert the audio data into text.

[1698] 2. Extraction of important information and generation of meeting minutes:

[1699] The server extracts key information from the converted text using an automated summarization algorithm (such as Hugging Face's NLP model) and generates meeting minutes. These minutes include important points from the meeting and tasks to be taken as the next steps.

[1700] 3. Task extraction and addition to the management system:

[1701] An algorithm is applied to automatically extract tasks from meeting minutes, and the extracted tasks are added to the management system. This enables efficient task management.

[1702] 4. Real-time checking of wording and formatting:

[1703] When users create meeting materials or reports, they use a dedicated application. This application checks the wording and formatting in real time and provides feedback to the user. The check results are sent back to the server, where advanced natural language processing algorithms are applied.

[1704] 5. Content analysis and explanation of technical terms:

[1705] When emails or documents are uploaded to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. Simultaneously, an algorithm that detects technical terms is activated and provides explanations to the user.

[1706] 6. Scheduling and notifications:

[1707] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the best meeting date and time. Once the best date and time are determined, all participants are notified. Scheduling meetings is particularly important, as this improves productivity.

[1708] As a concrete example, the following prompt statement is used:

[1709] Audio recording: "At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step."

[1710] Summarized text: "Production line adjustments: Person in charge submits revised proposal."

[1711] Example of a prompt:

[1712] "Please summarize this meeting: 'Today's meeting discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[1713] "Extract the task from this text: 'At today's meeting, we discussed adjustments to the production line. The person in charge will submit a revised plan as the next step.'"

[1714] In this way, the system implementing the present invention efficiently supports a wide range of tasks in business activities.

[1715] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1716] Step 1:

[1717] The user collects audio data using a dedicated device during meetings and discussions. A robot equipped with a microphone records the meeting audio and transmits the audio data to a server via a terminal. In this step, the audio data (input) is transmitted to the server (output).

[1718] Step 2:

[1719] The server uses a speech recognition API (for example, Google's speech recognition API) to convert the received audio data into text. This text conversion process transforms the audio data (input) into text data (output). This conversion includes parsing the audio data and generating a string.

[1720] Step 3:

[1721] The server analyzes the text data using an automated summarization algorithm (e.g., Hugging Face's NLP model) to extract important information from the text. In this step, the text data (input) is converted into summarized text (output). The process then extracts keywords and important phrases from the text.

[1722] Step 4:

[1723] The server generates meeting minutes and applies an algorithm to extract tasks from those minutes. Here, summarized text (input) is generated as meeting minutes (output), and tasks (output) are further extracted. The extracted tasks are added to the management system.

[1724] Step 5:

[1725] A dedicated application used by users to create documents checks the wording and formatting in real time and provides feedback. The application analyzes text data (input) and provides suggested wording revisions and formatting check results (output). The feedback content is generated and notified to the user.

[1726] Step 6:

[1727] When a user uploads emails or documents to the server, the server uses an automated summarization algorithm to analyze the content and generate a summary. A technical term detection algorithm is also applied to generate explanations of technical terms. In this process, the uploaded content (input) is transformed into a summary and explanations of technical terms (output).

[1728] Step 7:

[1729] The user sends a meeting request to the server, including the meeting title, participant list, and preferred date and time options. The server checks the participants' schedules and suggests the most suitable meeting date and time. Here, the meeting request information (input) is suggested as the optimal meeting date and time (output).

[1730] Step 8:

[1731] The server notifies all participants of the optimal meeting date and time. Email and calendar app APIs are used for notification. This ensures that the meeting date and time suggested by the server (input) is notified to all participants (output).

[1732] Step 9:

[1733] After the meeting, the server adds tasks extracted from the generated meeting minutes to the management system and manages their schedules. Here, the task information (input) from the meeting minutes is added to the task list (output) in the management system, and the schedule is updated.

[1734] Detailed processing at each step automates the process, allowing it to proceed more efficiently.

[1735] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1736] This invention is a system that combines the generation of meeting minutes from audio data, task management, checking the wording and formatting of documents, summarizing emails and documents and providing explanations of technical terms, and adjusting meeting schedules with an emotion engine that recognizes the user's emotions. Specific embodiments for carrying out this invention are described below.

[1737] Meeting minutes and task management support during meetings.

[1738] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[1739] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[1740] Combination with the emotion engine

[1741] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[1742] Support for checking the wording and formatting when creating documents.

[1743] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[1744] Combination with the emotion engine

[1745] The emotion engine recognizes the user's emotional state and provides feedback that takes this into account if stress or anxiety is detected. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[1746] Support for summarizing sent emails and documents, and explaining technical terms.

[1747] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1748] Combination with the emotion engine

[1749] The emotion engine evaluates the user's emotional state when analyzing emails and documents, and adjusts the content and expression of summaries and explanations based on that evaluation. For example, if it determines that "the user is tired," it can provide a clear and concise summary.

[1750] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1751] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1752] Combination with the emotion engine

[1753] The emotion engine analyzes participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[1754] As described above, the present invention is a system that combines an emotion engine to significantly improve work efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[1755] The following describes the processing flow.

[1756] Meeting minutes and task management support during meetings.

[1757] Step 1:

[1758] User: At the start of the meeting, activate the dedicated recording device and collect audio data.

[1759] Step 2:

[1760] Terminal: Sends recorded audio data to the server.

[1761] Step 3:

[1762] Server: Uses a speech recognition API to convert the transmitted audio data into text.

[1763] Step 4:

[1764] Server: Uses an emotion engine to analyze the user's emotional state and adds relevant information to the text data.

[1765] Step 5:

[1766] Server: Uses an automated summarization algorithm to extract key points from text and generate a summary of the meeting minutes.

[1767] Step 6:

[1768] Server: Use a task extraction algorithm to extract specific tasks from meeting minutes and add them to the management system.

[1769] Step 7:

[1770] Server: Sends the generated meeting minutes and task list to the terminal.

[1771] Step 8:

[1772] Terminal: Displays received meeting minutes and task lists to the user.

[1773] Support for checking the wording and formatting when creating documents.

[1774] Step 1:

[1775] User: Create documents using a dedicated application.

[1776] Step 2:

[1777] Terminal: Performs grammar checks and formatting consistency in real time, and provides feedback of the check results to the user.

[1778] Step 3:

[1779] Terminal: Uses an emotion engine to analyze the user's emotional state and reflect it in the check feedback.

[1780] Step 4:

[1781] Terminal: Sends a draft of the created document to the server.

[1782] Step 5:

[1783] Server: Applies advanced natural language processing algorithms to generate results for checking the wording and formatting.

[1784] Step 6:

[1785] Server: Sends check results and suggestions to the terminal.

[1786] Step 7:

[1787] Terminal: Displays the received check results and suggestions to the user.

[1788] Support for summarizing sent emails and documents, and explaining technical terms.

[1789] Step 1:

[1790] User: Upload emails and documents to the server.

[1791] Step 2:

[1792] Server: Analyzes the content of emails and documents using an automated summarization algorithm and a technical term detection algorithm.

[1793] Step 3:

[1794] Server: Generates a summary from the analysis results.

[1795] Step 4:

[1796] Server: Uses an emotion engine to adjust the content and expression of summaries and explanations according to the user's emotional state.

[1797] Step 5:

[1798] Server: Detects technical terms from analysis results and generates explanations.

[1799] Step 6:

[1800] Server: Sends the generated summary and explanations of technical terms to the terminal.

[1801] Step 7:

[1802] Terminal: Displays the received summary and explanations of technical terms to the user.

[1803] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1804] Step 1:

[1805] User: Enter the meeting title, participant list, and preferred date and time options in the meeting request form.

[1806] Step 2:

[1807] Terminal: Sends the entered information to the server.

[1808] Step 3:

[1809] Server: Uses the participant's calendar API to check each participant's availability.

[1810] Step 4:

[1811] Server: Uses an emotion engine to analyze participants' emotional states based on historical data and current conditions.

[1812] Step 5:

[1813] Server: Based on the scheduling results, it proposes the optimal meeting date and time and notifies the participants.

[1814] Step 6:

[1815] Server: Receive feedback from participants and make adjustments as needed.

[1816] Step 7:

[1817] Server: Notifies all participants of the finalized meeting date and time.

[1818] Step 8:

[1819] Terminal: Displays the finalized meeting date and time to the user.

[1820] The above outlines the specific processing steps of the present invention, which incorporates an emotion engine. This system enables the entire business process to proceed efficiently and provides meticulous support that takes into account the user's emotions.

[1821] (Example 2)

[1822] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1823] Traditional business support systems, such as those for meeting minute generation, task management, document creation, email summarization, and meeting scheduling, are often provided individually, resulting in problems with the time and effort required for integrated operation and management. Furthermore, the difficulty in providing feedback and support that takes into account the user's emotional state posed a risk of decreased work efficiency and user satisfaction.

[1824] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for collecting audio data, means for transmitting audio data to the server, means for converting audio data to text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for monitoring the user's emotional state, means for reflecting the user's emotional information in the meeting minutes, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, means for notifying participants of the optimal meeting date and time, and means for adjusting the meeting date and time considering the user's emotional state. As a result, the user can process tasks integrally and efficiently with a single system, and furthermore, receiving feedback and support that takes into account their emotional state improves work efficiency and increases user satisfaction.

[1825] "Audio data" refers to data collected using recording devices, such as audio from meetings or conferences.

[1826] A "server" is a computer system that processes, records, analyzes, and notifies audio data.

[1827] A "speech recognition API" is an application programming interface for converting speech data into text data.

[1828] "Text data" refers to character information converted using a speech recognition API.

[1829] "Meeting minutes" are documents that record the content of meetings or discussions.

[1830] A "task management system" is a system for managing extracted tasks and tracking their progress.

[1831] An "emotion engine" is software that analyzes a user's emotions from their voice, facial expressions, and other data.

[1832] "Grammar check" is the process of verifying whether the wording in a document conforms to linguistic rules.

[1833] "Format checking" is the process of verifying whether a document adheres to a certain format and layout.

[1834] An "automatic summarization algorithm" is a program that extracts important parts from long texts and summarizes them concisely.

[1835] A "technical term detection algorithm" is a program that identifies technical terms within a document and provides their definitions and explanations.

[1836] The "Calendar API" is an application programming interface for checking each participant's schedule.

[1837] "Feedback" refers to the evaluations and improvement suggestions that a system provides to its users.

[1838] Modes for carrying out the invention

[1839] This invention is a system that combines the generation of meeting minutes from audio data, task management, text formatting checks for document creation, summarization of emails and documents, explanation of technical terms, and scheduling of meetings with an emotion engine that recognizes user emotions. Specific embodiments for carrying out this invention are described below.

[1840] Meeting minutes and task management support during meetings.

[1841] First, when a user conducts a meeting, they activate a dedicated recording device to collect audio data. This audio data is sent to a server via the terminal. The server uses a speech recognition API to convert the audio data into text. For example, audio data such as, "In today's meeting, we discussed the progress of Project X. Person in charge A decided to create document Y as the next step," would be converted into text.

[1842] Next, the server uses an automated summarization algorithm to extract key points from the generated text and create a summary of the meeting minutes. Furthermore, a task extraction algorithm is used to extract specific tasks from the minutes and add them to the management system. As a result, important information such as "Progress of Project X" and "Person in charge A created document Y" is summarized and managed as specific tasks.

[1843] The server uses an emotion engine to monitor the user's emotional state during meetings. The emotion engine analyzes emotions from voice, facial expressions, and text input, obtaining information such as "the user is stressed" or "the user is satisfied." This emotional information can be included as supplementary information in the meeting minutes.

[1844] Support for checking the wording and formatting when creating documents.

[1845] Users create documents using a dedicated application. This application performs grammar checks and formatting consistency checks in real time and provides feedback to the user. The draft document is sent to a server, where advanced natural language processing algorithms are applied to provide the results of the text formatting check. For example, if a user writes "This document is very easy to understand," the application might suggest correcting it to "This document is extremely easy to understand."

[1846] The server uses an emotion engine to recognize the user's emotional state and provides feedback that takes stress or anxiety into account. For example, if the user is feeling tense, it can offer suggestions for improvement using gentler language.

[1847] Support for summarizing sent emails and documents, and explaining technical terms.

[1848] Users upload emails and documents to the server. The server analyzes the content using an automated summarization algorithm and a technical term detection algorithm, generating a summary and explanations of technical terms. The results are sent to the terminal and displayed to the user. For example, an email stating, "Regarding the progress of Project X, the next step is to create document Y," would be summarized as "Project X: Creating document Y is the next step," and "document Y" would be explained as "a document for a specific project."

[1849] The server uses an emotion engine to assess the user's emotional state when analyzing emails and documents, and adjusts the content and wording of summaries and explanations based on that assessment. For example, if it determines that the user is tired, it can provide a clear and concise summary.

[1850] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1851] The user enters the meeting title, participant list, and preferred date and time into a meeting request form and sends it to the server from their device. The server uses the participants' calendar API to check their schedules and suggests the best meeting date and time. The user is notified of the adjustment results, and further adjustments are made if necessary. The final decided meeting date and time are notified to all participants. For example, the server checks the calendars of participants A and B and, based on their availability, decides and notifies them that "the next meeting will be held on October 15th from 15:00 to 16:00."

[1852] The server uses an emotion engine to analyze participants' emotional states when scheduling meetings and suggests the optimal date and time. For example, if "Participant A is stressed due to busyness," it can avoid that situation and find a more convenient time slot.

[1853] Examples of specific cases and prompt statements

[1854] The following prompt statements are possible as concrete examples of how this system is operated:

[1855] 1. "Record meetings and automatically generate meeting minutes and tasks. Analyze user sentiment as well."

[1856] 2. "Please check the materials in real time and suggest appropriate wording and formatting. We will also take user sentiment into consideration."

[1857] 3. "Upload emails and documents, and generate summaries of the content and explanations of technical terms. User sentiment will also be reflected as appropriate."

[1858] 4. "Please coordinate the meeting and propose the most suitable date and time. Please also consider the feelings of the participants."

[1859] As described above, the system of the present invention provides an integrated solution that combines an emotion engine to significantly improve operational efficiency. This provides detailed support that takes into account the user's emotional state, thereby improving the quality and efficiency of work.

[1860] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1861] Meeting minutes and task management support during meetings.

[1862] Step 1:

[1863] The user activates the recording device and collects audio data.

[1864] Input: Audio data of a meeting initiated by the user.

[1865] Specific operation: The user manually activates a dedicated recording device and uses a high-sensitivity microphone to collect audio from the meeting.

[1866] Output: Recorded audio data.

[1867] Step 2:

[1868] The device sends voice data to the server.

[1869] Input: Audio data collected by a recording device.

[1870] Specific operation: The device uploads the recorded audio data to the server using Wi-Fi or mobile data communication.

[1871] Output: Audio data uploaded to the server.

[1872] Step 3:

[1873] The server uses a speech recognition API to convert the audio data into text.

[1874] Input: Audio data uploaded to the server.

[1875] Specific operation: The server uses speech recognition APIs such as the Google Cloud Speech-to-Text API to convert speech data into text data.

[1876] Output: Converted text data.

[1877] Step 4:

[1878] The server extracts key points from the text and generates a summary of the meeting minutes.

[1879] Input: Converted text data.

[1880] Specific operation: The server uses natural language processing algorithms to extract important information from text data and generate a summary.

[1881] Output: A summary of the generated meeting minutes.

[1882] Step 5:

[1883] The server generates tasks using a task extraction algorithm and adds them to the management system.

[1884] Input: A summary of the generated meeting minutes.

[1885] Specific operation: The server applies a task extraction algorithm to extract specific tasks from the summary and add them to the management system.

[1886] Output: Tasks added to the management system.

[1887] Step 6:

[1888] The server uses an emotion engine to monitor the user's emotional state and reflect it in the meeting minutes.

[1889] Input: Audio recordings, facial expressions, and text input during meetings.

[1890] Specific operation: The emotion engine analyzes this data and recognizes the user's emotional state.

[1891] Output: Meeting minutes containing emotional information.

[1892] Support for checking the wording and formatting when creating documents.

[1893] Step 1:

[1894] Users create documents using a dedicated application.

[1895] Input: Drafts or documents of materials.

[1896] Specific operation: The user creates a document using a document creation application, and grammar and formatting checks are performed in real time.

[1897] Output: Draft version of the document.

[1898] Step 2:

[1899] The terminal sends a draft of the document to the server.

[1900] Input: Draft version of the document.

[1901] Specific operation: The terminal temporarily stores the draft of the document on the server and then securely transmits it.

[1902] Output: Draft version of the document sent to the server.

[1903] Step 3:

[1904] The server checks the wording and formatting and suggests revisions.

[1905] Input: Draft document sent to the server.

[1906] Specific operation: The server applies advanced natural language processing algorithms to check the wording and formatting. For example, it might suggest changing "This document is very easy to understand" to "This document is extremely easy to understand."

[1907] Output: Check results of the document including the proposed revisions.

[1908] Step 4:

[1909] The server sends the correction results to the terminal and provides feedback to the user.

[1910] Input: Check results of the document containing the proposed revisions.

[1911] Specific operation: The check results are reflected in real time on the interface of the dedicated application.

[1912] Output: Correction results provided to the user as feedback.

[1913] Step 5:

[1914] The server uses an emotion engine to adjust correction suggestions based on the user's emotions.

[1915] Input: User emotional state data (e.g., whether the user is tense, stressed, etc.).

[1916] Specific operation: The emotion engine analyzes emotional data and adjusts the suggested corrections based on the analysis results. For example, if the user is feeling stressed, it will suggest corrections using gentler language.

[1917] Output: Revised correction suggestions.

[1918] Support for summarizing sent emails and documents, and explaining technical terms.

[1919] Step 1:

[1920] Users upload emails and documents to the server.

[1921] Input: Emails or documents to send.

[1922] Specific operation: Users use a dedicated upload form to upload emails and documents to the server.

[1923] Output: Emails and documents uploaded to the server.

[1924] Step 2:

[1925] The server performs the analysis using an automatic summarization algorithm and a technical term detection algorithm.

[1926] Input: Emails and documents uploaded to the server.

[1927] Specific operation: The server uses an automatic summarization algorithm to summarize the content and a terminology detection algorithm to extract technical terms.

[1928] Output: Summary and list of technical terms.

[1929] Step 3:

[1930] The server generates a summary and explanations of technical terms.

[1931] Input: Summary and list of technical terms.

[1932] Specific operation: The server generates explanations of technical terms and formats them together with summaries of emails and documents.

[1933] Output: Summary and explanation of technical terms.

[1934] Step 4:

[1935] The server sends the generated summary and explanation to the terminal and provides feedback to the user.

[1936] Input: Summary and explanation of technical terms.

[1937] Specific operation: The summary and explanation results are sent to the terminal and displayed to the user.

[1938] Output: Summary and explanation of technical terms displayed to the user.

[1939] Step 5:

[1940] The server uses an emotion engine to adjust the content and expression of summaries and explanations.

[1941] Input: User's emotional state data.

[1942] Specific operation: The emotion engine assesses the user's emotional state and adjusts the content and expression of the summary and explanation based on that assessment. For example, if the user is tired, it will provide a clear and concise summary.

[1943] Output: Adjusted summary and explanation of technical terms.

[1944] Automated coordination support for meeting scheduling, including communication between the requesting party and the requesting party.

[1945] Step 1:

[1946] The user enters the meeting details into a meeting request form and sends it from their device to the server.

[1947] Input: Details such as the meeting title, participant list, and preferred date and time options.

[1948] Specific operation: The user enters the required information into the meeting request form and sends it from their device to the server.

[1949] Output: Meeting details sent to the server.

[1950] Step 2:

[1951] The server uses the participants' calendar API to check their schedules.

[1952] Input: Participant list.

[1953] Specific operation: The server uses the Calendar API to check each participant's schedule.

[1954] Output: A list of available time slots for all participants.

[1955] Step 3:

[1956] The server will suggest the optimal meeting date and time.

[1957] Input: A list of available time slots for all participants.

[1958] Specific operation: The server calculates and proposes the optimal meeting date and time when all participants can attend.

[1959] Output: Proposed meeting date and time.

[1960] Step 4:

[1961] The server sends the adjustment results to the terminal and notifies the user.

[1962] Input: Proposed meeting date and time.

[1963] Specific operation: The adjustment results are sent to the terminal and notified to the user.

[1964] Output: The adjustment results notified to the user.

[1965] Step 5:

[1966] The server uses an emotion engine to analyze the emotional state of participants and suggest the optimal meeting date and time.

[1967] Input: Participant emotional state data.

[1968] Specific operation: The emotion engine analyzes the participants' busyness and stress levels and suggests the optimal meeting date and time, taking these factors into consideration.

[1969] Output: Results of adjusting the meeting date and time, taking emotional states into consideration.

[1970] The above describes the specific processing steps and operations related to the program of this system.

[1971] (Application Example 2)

[1972] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1973] In today's business environment, there is a growing demand for increased efficiency in various meetings and conferences. In particular, meetings conducted within autonomous mobile devices face challenges due to the lack of automation in tasks such as generating meeting minutes from audio data, task management, formatting checks for documents, and summarizing emails and documents while providing explanations of technical terms. Furthermore, the lack of means to provide appropriate feedback tailored to the emotional state of users impacts the quality and efficiency of work.

[1974] In Application Example 2, the specific processing performed by the specific processing unit 290 of the data processing device 12 is realized by the following means. In this invention, the server includes means for converting audio data into text, means for extracting important information from the text and generating meeting minutes, means for automatically extracting tasks from the meeting minutes and adding them to the management system, means for collecting audio data in real time within an autonomous mobile device, means for analyzing the user's emotions from the collected audio and text using an emotion recognition engine, means for providing appropriate feedback and summaries based on the user's emotional state, means for checking the wording and format of materials in real time, means for analyzing the content of emails and materials and generating summaries, means for detecting technical terms from the analyzed content and providing explanations, means for checking participants' schedules and proposing the optimal meeting date and time, and means for notifying participants of the optimal meeting date and time. This makes it possible to improve the efficiency and quality of meetings and conferences.

[1975] "Audio data" refers to data that records audio in digital format.

[1976] "Means of converting to text" refers to technology or equipment for analyzing audio data and converting it into text format.

[1977] "Means for extracting important information" refers to technologies or devices that automatically select key points or important information from text.

[1978] "Means for generating meeting minutes" refers to the technology or device that compiles extracted important information into a format suitable for meeting minutes.

[1979] "Means for automatically extracting tasks and adding them to a management system" refers to a technology or device that identifies tasks from meeting minutes and automatically registers them in a management system.

[1980] "Autonomous mobile devices" refer to all mobile devices that can move automatically without human intervention.

[1981] "Means for collecting audio data in real time" refers to technology or equipment for recording audio in real time within an autonomous mobile device.

[1982] An "emotion recognition engine" is a technology or software used to analyze a user's emotions from voice or text data.

[1983] "Means of providing feedback" refers to technologies or devices that communicate analysis results and support details to users.

[1984] "Methods for checking wording and formatting in real time" refers to technologies or devices that allow for real-time verification of the accuracy and formatting of text in documents and materials, and the provision of revision suggestions.

[1985] "Means for analyzing the content of emails and documents and generating summaries" refers to technology or devices that read the text of emails and documents and concisely summarize their main points.

[1986] "Means for detecting and providing explanations of technical terms" refers to technology or devices that find technical terms within a document and automatically generate explanations related to them.

[1987] "A means of checking participants' schedules and proposing the optimal meeting date and time" refers to a technology or device that obtains the schedule information of meeting participants and calculates the optimal meeting date and time.

[1988] "Means of notifying participants of the optimal meeting date and time" refers to technology or equipment for notifying meeting participants of the decided meeting date and time.

[1989] The system for realizing this invention consists of a combination of various software and hardware for collecting, converting, analyzing, and providing f...

Claims

1. A means of converting audio data into text, A means of extracting important information from text and generating meeting minutes, A method for automatically extracting tasks from meeting minutes and adding them to the management system, A means of checking the wording and format of documents in real time, A means of analyzing the content of emails and documents to generate summaries, A means of detecting technical terms from the analyzed content and providing explanations, A means of checking participants' schedules and suggesting the most suitable meeting date and time, A means of notifying participants of the optimal meeting date and time, A system that includes this.

2. A means of providing real-time feedback to the user on the results of checking the wording and formatting, A means of applying advanced natural language processing algorithms to provide results for checking the formatting of text, The system according to claim 1, further comprising:

3. A means of displaying the summary and explanation of technical terms to the user, If readjustments are needed, there are means to receive feedback from participants and make adjustments, A means of notifying all participants of the finalized meeting date and time, The system according to claim 1, further comprising:

Images