System

Abstract

A system is provided.SOLUTION: A system for assisting in the progress of a meeting, comprising: means for generating an agenda based on meeting themes and objectives; means for transcribing audio data and summarizing speech content during the meeting; means for analyzing speech rates during the meeting and generating alerts for balancing; means for managing timekeeping and sending notifications depending on time remaining; and means for collecting and analyzing feedback after the meeting ends.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In modern business, meetings are an important means of decision-making and information sharing, but many meetings are inefficient and waste time. This situation is mainly due to a lack of appropriate agenda creation, stagnant discussions, biased comments, poor time management, and a lack of feedback collection after meetings. There is a need to solve these issues and improve the quality and efficiency of meetings. [Means for solving the problem]

[0005] In order to solve these problems, the present invention provides the following means:

[0006] A means of generating an agenda based on the conference theme and objectives;

[0007] A means of transcribing audio data during meetings and summarizing what is being said;

[0008] A means to analyze speaking rates during meetings and generate alerts to balance them;

[0009] A means to manage timekeeping and send notifications depending on the time remaining;

[0010] A means of collecting and analyzing feedback after the meeting.

[0011] This will create a system that provides efficient end-to-end support during each phase of a meeting, from preparation, through to completion.

[0012] A "system to support the progress of meetings" is a system that uses technological means to streamline and optimize all processes from before the start of a meeting to after it ends.

[0013] An "agenda" is a list of the themes, objectives, and topics of a meeting, and is a schedule to facilitate the smooth progress of the meeting.

[0014] "Transcription" is the process of converting audio data into text data.

[0015] "Methods for summarizing speech content" refers to methods for shortening transcribed speech content and extracting only the main points.

[0016] "Means for analyzing speaking rates" refers to a method for measuring the speaking time and frequency of each participant during a meeting and analyzing the data.

[0017] An "alert" is a notification or warning message that alerts the user to a specific situation.

[0018] "Timekeeping" is a function that manages the time for each segment of a meeting and ensures that the meeting proceeds within the specified time.

[0019] The "means for sending a notification according to the remaining time" is a means for notifying the user of the remaining time set according to the progress of the conference.

[0020] "Feedback" refers to information such as opinions, impressions, and areas for improvement collected from conference participants.

[0021] "Methods for collecting and analyzing feedback" refers to methods for systematically collecting feedback, analyzing the content, and identifying areas for improvement in meetings.

[0022] A "natural language processing model" is an algorithm or technology that allows a computer to analyze and understand the natural language used by humans.

[0023] The "means for receiving audio data" is a method for capturing audio occurring during a conference in digital form.

[0024] The "means for generating an alert" is a method for automatically creating a notification to alert the user based on the analysis results. [Brief explanation of the drawings]

[0025] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0026] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0027] First, the terms used in the following description will be explained.

[0028] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0029] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0030] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0031] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0032] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0033] [First embodiment]

[0034] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0035] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0036] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0037] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0038] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0039] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0040] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0041] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0042] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0043] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0044] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0045] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0046] The present invention provides a system for supporting the progress of a conference, specifically providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction between a server, terminals, and users.

[0047] System Configuration

[0048] 1. Server

[0049] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[0050] 2. Terminal

[0051] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, and enter feedback.

[0052] 3. Users

[0053] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[0054] Program processing

[0055] Creating an agenda

[0056] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[0057] Summary of remarks

[0058] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[0059] Progress management and speech rate analysis

[0060] The server analyzes the speech rate of each participant during the meeting. Based on this analysis, if a specific participant is speaking excessively or the discussion is stagnating, the server generates an alert and sends it to the device. The device then displays this alert to the user, prompting them to take the necessary action.

[0061] Timekeeping

[0062] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[0063] Collecting and analyzing feedback

[0064] After the meeting, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback, extracts areas for improvement and important opinions, and generates a feedback report to help prepare for the next meeting.

[0065] Specific examples

[0066] Creating and reviewing the agenda

[0067] When a user prepares for a monthly report meeting, they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses an AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can review the generated agenda on their device, make any necessary additions or corrections, and then resubmit and save the final version.

[0068] Meeting management and management

[0069] When a meeting begins, the server receives and transcribes audio data in real time. It then uses a natural language processing model to summarize the results, which users can view on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[0070] Post-meeting evaluation and feedback

[0071] After the meeting ends, the server creates a feedback form and sends it to each participant. Users fill out their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report is provided to users to help them prepare for the next meeting.

[0072] In this way, the present invention provides a system that efficiently and effectively supports meetings from preparation to progress and evaluation after completion.

[0073] The processing flow will be explained below.

[0074] Creating an agenda

[0075] Step 1:

[0076] The user uses the terminal to input the conference theme and purpose, and transmits them to the server.

[0077] Step 2:

[0078] Based on the received theme and purpose, the server references past meeting data and uses an AI model to generate a draft agenda.

[0079] Step 3:

[0080] The generated agenda plan is sent from the server to the terminal and displayed to the user.

[0081] Step 4:

[0082] The user checks the draft agenda on the device and makes corrections as necessary.

[0083] Step 5:

[0084] The revised agenda is sent back to the server and saved as the final version.

[0085] Summary of remarks

[0086] Step 1:

[0087] The server receives the audio data in real time during the conference.

[0088] Step 2:

[0089] The server automatically transcribes the received audio data.

[0090] Step 3:

[0091] The server passes the transcribed data to a natural language processing model to generate a summary of what was said.

[0092] Step 4:

[0093] The generated summary is sent from the server to the terminal and displayed to the user.

[0094] Analysis of speech rate and generation of alerts

[0095] Step 1:

[0096] The server analyzes each participant's speaking rate in real time during the meeting.

[0097] Step 2:

[0098] Based on the analysis results, if a particular participant is speaking excessively, the server will generate an alert to balance things out.

[0099] Step 3:

[0100] The generated alert is sent from the server to the terminal and displayed to the user.

[0101] Timekeeping

[0102] Step 1:

[0103] The server sets a timer for each segment of the conference.

[0104] Step 2:

[0105] When the time approaches the specified time remaining, the server generates a timekeeping alert.

[0106] Step 3:

[0107] The generated timekeeping alert is sent from the server to the terminal and displayed to the user.

[0108] Collecting and analyzing feedback

[0109] Step 1:

[0110] After the meeting ends, the server automatically generates a feedback form.

[0111] Step 2:

[0112] The generated feedback form is sent from the server to the participant's terminal.

[0113] Step 3:

[0114] The user fills in a feedback form on the terminal and sends it to the server.

[0115] Step 4:

[0116] The server analyzes the received feedback and extracts areas for improvement and important opinions.

[0117] Step 5:

[0118] Based on the extracted information, the server generates a feedback report and sends it to the terminal to help prepare for the next meeting.

[0119] As described above, the server, terminals, and users work together in each step to realize a system that efficiently prepares for, progresses, and evaluates a conference after it has ended.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] In today's business environment, effective meeting management and proceedings are crucial. However, the entire process, from preparing for meetings to conducting them and evaluating them after they have finished, requires a significant amount of time and effort due to manual work. In particular, tasks such as creating agendas, summarizing comments, balancing speaking ratios, timekeeping, and collecting and analyzing feedback are complex and difficult to carry out efficiently. A consistent automated system to solve these issues is needed.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes: means for a user to input the meeting theme and purpose and send the input information to the server; means for the server to refer to past meeting data and generate a draft agenda using a generative AI model and send the generated draft agenda to the terminal; means for the user to check the draft agenda on the terminal and resend the revised agenda to the server for storage; means for the server to receive audio data from the meeting and transcribe it in real time; means for the server to pass the transcribed speech content to a natural language processing model to generate a summary and send the generated summary to the terminal; means for the server to analyze the speech rate of each participant in the meeting in real time, generate an alert based on the analysis result, and send the generated alert to the terminal; means for a user to set a timer for the agenda from the terminal, the server to monitor the timer, generate an alert when the timeout is approaching, and send the alert to the terminal; means for the server to automatically generate a feedback form after the meeting ends and send the feedback to the user; and means for the user to input feedback and send it to the server, the server to analyze the collected feedback, generate a feedback report, and provide the generated report to the user. This allows the entire process from preparing for a meeting to proceeding and evaluation after it has ended to be efficiently automated, reducing the burden on users.

[0125] A "user" is a person who operates a terminal to prepare for a conference, manage its progress, and evaluate it after it has ended.

[0126] "Terminal" means a device that provides an interface for a User to interact with the Server, allowing the User to view and modify the agenda, review comments, track progress, receive timekeeping alerts, and enter feedback.

[0127] The "server" is a central processing unit responsible for meeting information management, data analysis, agenda generation, speech transcription and summarization, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[0128] An "agenda" is a list of topics to be discussed to facilitate the progress of a meeting, and is generated based on the theme and purpose of the meeting.

[0129] A "generative AI model" is an artificial intelligence model that learns from past meeting data and refers to an algorithm for automatically generating a draft agenda based on the meeting theme and purpose entered by the user.

[0130] "Audio data" refers to data that records what is said during a meeting and is later transcribed.

[0131] "Transcription" refers to the process of converting audio data into text data.

[0132] A "natural language processing model" refers to an algorithm for analyzing and summarizing transcribed text data, and it performs text analysis by learning from a large amount of text data.

[0133] The "speaking rate" is an index showing the proportion of time each participant spoke during the conference.

[0134] An "alert" is a notification that is generated based on the progress of the conference and the balance of speech among the participants, and is used to alert the user.

[0135] A "timer" is a device that measures the time allocated to each agenda item in a meeting and manages the progress to be made within the specified time.

[0136] A "feedback form" is an electronic form containing questions to collect opinions and feedback from participants after the meeting.

[0137] A "feedback report" is a report that analyzes the collected feedback and summarizes areas for improvement in the meeting and important opinions from participants.

[0138] The present invention is a system for supporting the progress of a conference, providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction of a server, terminals, and users.

[0139] System Configuration

[0140] server

[0141] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis. Specifically, the server transcribes speech data using Google® Cloud Speech-to-Text API or AWS® Transcribe, summarizes speech content using natural language processing models (e.g., BERT), generates agendas using generative AI models (e.g., GPT-3®), and references past meeting data.

[0142] Terminal

[0143] The terminal provides an interface for users to interact with the server. Through the terminal, users can check and modify the agenda, check what has been said, track progress, receive timekeeping alerts, and enter feedback. The terminal receives data from the server and displays it to the user.

[0144] User

[0145] Users operate the device to prepare for the meeting, manage its progress, and evaluate it after it has ended. Specifically, users input the meeting theme and purpose from the device and send them to the server. During the meeting, users can check summaries and alerts generated in real time through the device and take appropriate action. After the meeting, users can use the device to enter feedback and send it to the server.

[0146] Specific examples

[0147] Creating and reviewing the agenda

[0148] When a user prepares for a monthly report meeting, they send the theme "Monthly Report Meeting" and the purpose "Check Project Progress" from their device to the server. The server uses a generative AI model (e.g., GPT-3) based on past meeting data to generate a draft agenda, creating items such as "Check Project Progress," "Introduce New Projects," and "Share Issues." The user then reviews the generated agenda on their device, makes any necessary corrections, and resubmits it to save the final version.

[0149] Meeting management and management

[0150] When the meeting begins, the server receives and transcribes audio data in real time. For example, it uses the Google Cloud Speech-to-Text API to transcribe the audio data. The results are summarized using a natural language processing model (e.g., BERT), and users can view the summary on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal speaking. Furthermore, a timekeeping function is used to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[0151] Post-meeting evaluation and feedback

[0152] After the meeting ends, the server automatically generates a feedback form and sends it to the user. The user enters feedback such as "The meeting went smoothly" or "The comments were well-balanced" into the feedback form on their device. The server analyzes the collected feedback using a natural language processing model and generates a feedback report that extracts areas for improvement and important opinions. This report is provided to the user to help them prepare for the next meeting.

[0153] Prompt Sentence Examples

[0154] Creating an agenda: "Please create an agenda for the next monthly report meeting with the theme 'Monthly Report' and the purpose 'Checking project progress.'"

[0155] Summary Generation: "Generate a summary of the meeting based on the following transcription: [transcription]"

[0156] Feedback collection: "Automatically generate a form to collect feedback from participants after the meeting, analyze the input data, and create a feedback report."

[0157] By integrating the above functions, the present invention provides a comprehensive system for efficiently and effectively supporting meetings from preparation to progress and evaluation after completion.

[0158] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0159] Program processing flow

[0160] Creating an agenda

[0161] Step 1:

[0162] The user inputs the meeting theme "Project progress review meeting" and the purpose "Project progress review" via the terminal.

[0163] Input: Conference theme and objectives

[0164] Output: Notification of completion of input to the terminal

[0165] Step 2:

[0166] The terminal sends this input information to the server.

[0167] Input: User input information

[0168] Output: Data transfer to the server

[0169] Step 3:

[0170] The server retrieves past meeting data from a database and inputs it into a generative AI model (e.g., GPT-3).

[0171] Input: User input information, past meeting data

[0172] Output: Agenda proposal from the AI ​​model

[0173] Step 4:

[0174] The server sends the generated agenda plan to the terminal so that the user can check it.

[0175] Input: Agenda proposal from AI model

[0176] Output: Send the agenda to the terminal

[0177] Step 5:

[0178] The user checks the draft agenda on the device and modifies it as necessary.

[0179] Input: Agenda proposal from server

[0180] Output: User's modified data

[0181] Step 6:

[0182] The terminal sends the modified agenda back to the server, which stores it.

[0183] Input: User's revision agenda

[0184] Output: Save modified data to the server

[0185] Summary of remarks

[0186] Step 1:

[0187] The server receives the conference audio data in real time.

[0188] Input: Meeting audio data

[0189] Output: Real-time speech-to-text request

[0190] Step 2:

[0191] The server uses the Google Cloud Speech-to-Text API to transcribe the audio data.

[0192] Input: Meeting audio data

[0193] Output: Transcript

[0194] Step 3:

[0195] The server passes the transcribed text to a natural language processing model (e.g., BERT) to generate a summary.

[0196] Input: Transcription text

[0197] Output: Meeting summary text

[0198] Step 4:

[0199] The server sends the generated summary to the terminal.

[0200] Input: Meeting summary text

[0201] Output: Sends text to the terminal

[0202] Step 5:

[0203] The user checks the summary on the terminal and manages the progress of the conference.

[0204] Input: Meeting summary text from the server

[0205] Output: Information for managing the meeting progress

[0206] Progress management and speech rate analysis

[0207] Step 1:

[0208] The server monitors each participant's comments in real time.

[0209] Input: Conference audio data

[0210] Output: Speaker identification data

[0211] Step 2:

[0212] The server tallies the speaking time of each participant and calculates the speaking rate.

[0213] Input: Speaker identification data

[0214] Output: Speech rate data

[0215] Step 3:

[0216] The server generates an alert if a particular participant is speaking too much.

[0217] Input: Speech rate data

[0218] Output: Alert generation request

[0219] Step 4:

[0220] The server sends the generated alert to the terminal and notifies the user.

[0221] Input: Alert information

[0222] Output: Notification to terminal

[0223] Step 5:

[0224] The user checks the alert and adjusts the meeting progress if necessary.

[0225] Input: Alert Notification

[0226] Output: Coordination of meeting progress

[0227] Timekeeping

[0228] Step 1:

[0229] The user sets the timer for each agenda item.

[0230] Input: Set time for each agenda item

[0231] Output: Terminal sends setting values ​​to server

[0232] Step 2:

[0233] The server monitors the timer and measures the progress of each agenda item.

[0234] Input: Set time

[0235] Output: Timer measurement value

[0236] Step 3:

[0237] Generate an alert if the server is running low on time.

[0238] Input: Timer measurement value

[0239] Output: Timekeeping alert

[0240] Step 4:

[0241] The server sends the generated alert to the terminal and notifies the user.

[0242] Input: Timekeeping Alert

[0243] Output: Notification to terminal

[0244] Step 5:

[0245] The user acknowledges the alert and moves on to the next agenda item.

[0246] Input: Timekeeping Alert Notification

[0247] Output: Meeting agenda

[0248] Collecting and analyzing feedback

[0249] Step 1:

[0250] The server automatically generates a feedback form after the meeting ends.

[0251] Input: Meeting end information

[0252] Output: An automatically generated feedback form

[0253] Step 2:

[0254] The server sends the feedback form to the user.

[0255] Input: Auto-generated feedback form

[0256] Output: Feedback form sent to terminal

[0257] Step 3:

[0258] The user inputs feedback on the terminal and transmits it to the server.

[0259] Input: User feedback

[0260] Output: Send feedback to the server

[0261] Step 4:

[0262] The server analyzes the collected feedback.

[0263] Input: User feedback data

[0264] Output: Parsed feedback results

[0265] Step 5:

[0266] The server generates a feedback report and provides it to the user.

[0267] Input: Parsed feedback results

[0268] Output: Feedback report

[0269] Step 6:

[0270] Users can review the feedback report to help them prepare for their next meeting.

[0271] Input: Feedback report from the server

[0272] Output: Capture improvements and feedback

[0273] (Application example 1)

[0274] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0275] In factory maintenance meetings, inefficiencies in progress and information sharing are common, leading to reduced work efficiency and miscommunication. It's particularly difficult to grasp the content of comments made during meetings and the progress of work in real time, and to provide appropriate feedback and time management. Given these issues, a system is needed to ensure effective meeting progress and real-time information sharing.

[0276] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0277] In this invention, the server includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speech, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speech and work status in real time during a factory maintenance meeting and displaying it on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it. This enables more efficient meeting progress and real-time information sharing.

[0278] A "conference theme" is the main subject or topic that will be the focus of discussion at a conference.

[0279] "Purpose" refers to the goal or intention that the meeting is trying to achieve.

[0280] An "agenda" is a list of topics or matters to be discussed at a meeting and indicates the plan of action.

[0281] "Audio data" is a recording of speech spoken during a conference.

[0282] "Transcription" is the process of converting audio data into text data.

[0283] "Speech content" refers to the specific content or information spoken by participants during a meeting.

[0284] A "summary" is a text that condenses the content of a statement or information and extracts its main points.

[0285] The "speaking rate" indicates the rate or frequency at which each participant spoke during the meeting.

[0286] An "alert" is a notification or warning that is issued when a specific condition is met.

[0287] "Timekeeping" means managing the progress of meetings and work.

[0288] "Feedback" refers to the opinions and evaluations collected from participants after the meeting has ended.

[0289] "Analysis" refers to the detailed analysis of collected data to derive patterns and trends.

[0290] A "factory maintenance meeting" is a meeting held within a factory to discuss equipment maintenance and improvements.

[0291] A "smart device" is an electronic device that can connect to the Internet and display and obtain information.

[0292] "Progress management" means managing the arrangements and progress of meetings and work.

[0293] "Real-time" refers to data being processed immediately, without delay.

[0294] "Information sharing" means that multiple people share information and deepen their understanding of each other.

[0295] This invention is a system for supporting the progress of factory maintenance meetings, which is realized by the interaction of a server, a terminal, and a user. The system includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speeches, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speeches and work status during the factory maintenance meeting in real time and displaying them on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it.

[0296] Server Features

[0297] The server generates an agenda based on the meeting theme and purpose by inputting past meeting data into the AI ​​model. By inputting the meeting theme "Monthly maintenance meeting" and the purpose "Checking equipment status and planning the next maintenance" as prompts, the AI ​​model generates agendas such as "Reporting equipment status," "Planning the next maintenance," and "Sharing issues."

[0298] In addition, the system transcribes audio data during meetings in real time and summarizes it using natural language processing (NLP) models. It continuously collects audio during meetings, instantly converts it into text, and sends the summary to a smart device.

[0299] The server analyzes the speaking rate of each participant and generates an alert if there is an imbalance. For example, if a particular participant speaks excessively, an alert will be generated based on that information and a notification will be sent to the smart device.

[0300] Use the timekeeping function to manage the progress of meetings. Based on the time set for each agenda item, when the remaining time is running low, a timekeeping alert will be generated and a notification will be sent to the smart device. In this way, users can review the progress plan in a timely manner.

[0301] After the meeting, the server generates a feedback form and sends it to each participant. The collected feedback data is analyzed and a feedback report is generated to extract improvements for the next maintenance meeting.

[0302] Device and user operations

[0303] Users operate their terminals (smart devices) to check and modify the agenda, check what is being said, understand the progress, receive timekeeping alerts, and enter feedback. For example, when preparing for a monthly maintenance meeting, a user sends the theme of the "monthly maintenance meeting" and the purpose of "checking the equipment status and planning the next maintenance" to the server from their terminal.

[0304] Smart devices have the ability to display real-time transcriptions and summaries to users, specifically, by displaying the transcriptions, summaries, and progress management alerts on the user's smart glasses or head-mounted display.

[0305] Hardware and software used

[0306] The hardware used includes a server, smart glasses, a head-mounted display, and a microphone. The software used includes an AI model, a natural language processing (NLP) model, and a voice recognition library. Specifically, "some_ai_library" and "some_voice_recognition_library" are used.

[0307] Prompt Sentence Examples

[0308] "Topic: Monthly maintenance meeting, Purpose: Checking equipment status and planning the next maintenance"

[0309] In this way, the present invention provides a system that efficiently and effectively supports the preparation, progress, and evaluation of a factory maintenance meeting.

[0310] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0311] Step 1:

[0312] The server receives the meeting theme and purpose from the user. Specifically, the user uses a terminal (smart device) to input the theme, "Monthly Maintenance Meeting," and the purpose, "Checking the equipment status and planning the next maintenance." The input information is sent to the server. The input is text data of the theme and purpose, and the output is the text data sent to the server.

[0313] Step 2:

[0314] The server inputs the received theme and purpose into the generative AI model as prompt sentences. The server references past meeting data and generates a draft agenda using the AI ​​model. Specific data processing involves generating prompt sentences and searching past data, while data calculation involves generating an agenda using the AI ​​model. The input is text data of the theme and purpose, and the output is the generated agenda.

[0315] Step 3:

[0316] The server sends the generated agenda to the terminal, where the user can confirm and modify its contents. The user confirms the agenda on the terminal (smart device) and modifies it as necessary. The modified agenda is then sent back to the server and saved. The input is the text data of the generated agenda, and the output is the text data of the confirmed and modified agenda.

[0317] Step 4:

[0318] The server collects voice data in real time and transcribes it using a speech recognition library. Specifically, the server converts voice data collected by a microphone during a meeting into text data in real time. The input is voice data, and the output is transcribed text data.

[0319] Step 5:

[0320] The server passes the transcribed text data to a natural language processing (NLP) model to generate a summary. The server uses the NLP model to extract and summarize the key points of the text data. The input is the transcribed text data, and the output is the summarized text data.

[0321] Step 6:

[0322] The server sends the generated summary to the terminal and displays it to the user. The user can check the summary in real time during the meeting through their smart device. The input is the summarized text data, and the output is the summary displayed on the smart device.

[0323] Step 7:

[0324] The server analyzes the speech rate of each participant and generates alerts to balance the rate. Specifically, it analyzes audio data, measures the speaking time of each participant, and generates an alert if a specific participant is speaking excessively. The input is audio data, and the output is alert data.

[0325] Step 8:

[0326] The server measures the remaining time based on the timekeeping setting and generates a timekeeping alert. The server monitors the progress of the conference and generates an alert when the specified time is reached and sends it to the terminal. The input is the set time and the current time, and the output is the timekeeping alert.

[0327] Step 9:

[0328] After the conference ends, the server generates a feedback form and sends it to each participant. Users fill out feedback using their terminals and send it to the server. The input is the feedback form, and the output is the user's feedback data.

[0329] Step 10:

[0330] The server analyzes the collected feedback data and generates a feedback report. The server analyzes the feedback data and generates a report that extracts areas for improvement and important opinions, which is useful for preparing for the next meeting. The input is feedback data and the output is a feedback report.

[0331] The above processing steps make it possible to realize a system that efficiently and effectively supports the entire process from preparation to progress and evaluation after a meeting.

[0332] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0333] This invention is a system for supporting the progress of a conference. Specifically, it provides multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended, and further incorporates an emotion engine that recognizes the user's emotions to improve the quality of the conference and the satisfaction of the participants. The system is realized through the interaction between a server, terminals, and users.

[0334] System Configuration

[0335] 1. Server

[0336] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, feedback collection and analysis, and emotion recognition using an emotion engine.

[0337] 2. Terminal

[0338] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, view emotional tone, and enter feedback.

[0339] 3. Users

[0340] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[0341] Program processing

[0342] Creating an agenda

[0343] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[0344] Summary of remarks

[0345] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[0346] Emotion recognition by emotion engine

[0347] During the meeting, the server uses an emotion engine to analyze the voice and facial expressions of each participant in real time and recognize their emotional tone. This emotional tone is sent to the server as an emotion parameter and is used to generate alerts and manage the progress of the meeting.

[0348] Progress management and speech rate analysis

[0349] The server analyzes the speaking rate of each participant during the meeting. Based on the results of this analysis, if a particular participant is speaking excessively or the discussion is stagnating, the server generates an alert to balance the situation. In addition, the server appropriately adjusts the progress based on the output of the emotion engine. The alerts generated by the server are sent to the terminal and displayed to the user.

[0350] Timekeeping

[0351] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[0352] Collecting and analyzing feedback

[0353] After the meeting ends, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback, extracts areas for improvement and important opinions, and generates a feedback report to help prepare for the next meeting. During this process, data from the emotion engine is also used in the feedback analysis, taking into account the emotional reactions of the participants.

[0354] Specific examples

[0355] Creating and reviewing the agenda

[0356] When a user prepares for a monthly report meeting, they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses an AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can review the generated agenda on their device, make any necessary additions or corrections, and then resubmit and save the final version.

[0357] Meeting management and management

[0358] When a meeting begins, the server receives and transcribes audio data in real time. The results are summarized using a natural language processing model, and users can view the summary on their devices. The server also monitors the speaking rate of each participant and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses an emotion engine to analyze participants' emotions and adjust the progress of the meeting, for example, by making suggestions to ease the discussion if stress is building. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[0359] Post-meeting evaluation and feedback

[0360] After the meeting ends, the server creates a feedback form and sends it to each participant. Users fill out their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report is provided to users to help them prepare for the next meeting. At this time, the server also takes into account the emotional data analyzed by the emotion engine and suggests improvements to address emotional reactions and stress points.

[0361] In this way, the present invention provides a system that effectively and efficiently supports meetings from preparation to progress and evaluation after completion, and further uses an emotion engine to grasp the emotional state of participants and improve the quality of the meeting.

[0362] The processing flow will be explained below.

[0363] Creating an agenda

[0364] Step 1:

[0365] The user uses the terminal to input the conference theme and purpose, and transmits them to the server.

[0366] Step 2:

[0367] Based on the received theme and purpose, the server references past meeting data and uses an AI model to generate a draft agenda.

[0368] Step 3:

[0369] The generated agenda plan is sent from the server to the terminal and displayed to the user.

[0370] Step 4:

[0371] The user checks the draft agenda on the device and makes corrections as necessary.

[0372] Step 5:

[0373] The revised agenda is sent back to the server and saved as the final version.

[0374] Summary of remarks

[0375] Step 1:

[0376] The server receives the audio data in real time during the conference.

[0377] Step 2:

[0378] The server automatically transcribes the received audio data.

[0379] Step 3:

[0380] The server passes the transcribed data to a natural language processing model to generate a summary of what was said.

[0381] Step 4:

[0382] The generated summary is sent from the server to the terminal and displayed to the user.

[0383] Emotion recognition by emotion engine

[0384] Step 1:

[0385] The server receives the audio and video data in real time.

[0386] Step 2:

[0387] Based on the received data, the emotion engine analyzes the emotions of each participant.

[0388] Step 3:

[0389] The analyzed emotional tones are sent to the server as emotion parameters.

[0390] Step 4:

[0391] Based on the results of the emotional tone analysis, the server uses this information for progress management and alert generation.

[0392] Analysis of speech rate and generation of alerts

[0393] Step 1:

[0394] The server analyzes the speaking rate of each participant in the meeting in real time.

[0395] Step 2:

[0396] Based on the analysis results, if a particular participant is speaking excessively, the server will generate an alert to balance things out.

[0397] Step 3:

[0398] The generated alert is sent from the server to the terminal and displayed to the user.

[0399] Timekeeping

[0400] Step 1:

[0401] The server sets a timer for each segment of the conference.

[0402] Step 2:

[0403] When the time approaches the specified time remaining, the server generates a timekeeping alert.

[0404] Step 3:

[0405] The generated timekeeping alert is sent from the server to the terminal and displayed to the user.

[0406] Collecting and analyzing feedback

[0407] Step 1:

[0408] After the meeting ends, the server automatically generates a feedback form.

[0409] Step 2:

[0410] The generated feedback form is sent from the server to the participant's terminal.

[0411] Step 3:

[0412] The user fills in a feedback form on the terminal and sends it to the server.

[0413] Step 4:

[0414] The server analyzes the received feedback and extracts areas for improvement and important opinions.

[0415] Step 5:

[0416] Based on the extracted information, the server generates a feedback report and sends it to the terminal to help prepare for the next meeting.

[0417] Specific examples

[0418] Creating and reviewing the agenda

[0419] Step 1:

[0420] The user transmits the theme of the "monthly report meeting" and the purpose of "checking the progress of the project" from the terminal to the server.

[0421] Step 2:

[0422] The server uses an AI model based on past meeting data to generate a draft agenda.

[0423] Step 3:

[0424] For example, create items such as "Check project progress," "Introduce new projects," and "Share issues."

[0425] Step 4:

[0426] The user can review the generated agenda on their device and make any necessary additions or corrections.

[0427] Step 5:

[0428] The revised agenda is sent back to the server and the final version is saved.

[0429] Meeting management and management

[0430] Step 1:

[0431] When the meeting begins, the server receives audio and video data in real time and transcribes it.

[0432] Step 2:

[0433] The results are summarized using a natural language processing model, and users can view the summary on their device.

[0434] Step 3:

[0435] The server monitors the speaking rate of each participant and generates an alert and sends it to the terminal if excessive speaking is detected to encourage equal speaking.

[0436] Step 4:

[0437] The server uses an emotion engine to analyze participants' emotions and uses the results to manage the discussion, for example by making suggestions to ease the discussion if stress levels are rising.

[0438] Step 5:

[0439] Use the timekeeping function to manage the progress of each agenda item, and when time is approaching, an alert will be issued to prompt you to move on to the next agenda item.

[0440] Post-meeting evaluation and feedback

[0441] Step 1:

[0442] After the conference ends, the server automatically generates a feedback form and sends it to the participants.

[0443] Step 2:

[0444] The user fills in a feedback form from the terminal and sends it to the server.

[0445] Step 3:

[0446] The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions.

[0447] Step 4:

[0448] A feedback report is provided to the user to help prepare for the next meeting.

[0449] Step 5:

[0450] In addition, the emotional data analyzed by the emotion engine is taken into account and improvements are suggested to address emotional reactions and stress points.

[0451] Example 2

[0452] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0453] Conventional meeting support systems have had difficulty in providing consistent support for everything from meeting preparation to progress management and post-meeting evaluation. Furthermore, progress management did not take into account the emotional state of participants, which led to problems that could lead to a decline in meeting quality and participant satisfaction. Furthermore, balance of speech and timekeeping were inadequate. There is a need for a system that can solve these issues.

[0454] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0455] In this invention, the server includes a means for generating an agenda based on the conference theme and purpose, a means for transcribing audio data during the conference and summarizing the content of speeches, a means for analyzing the speaking rate during the conference and generating alerts for balancing speaking rates, a means for managing timekeeping and sending notifications according to the remaining time, a means for collecting and analyzing feedback after the conference ends, and a means for analyzing participants' emotions during the conference and recognizing their emotional tone. This provides comprehensive support for preparing, conducting, and evaluating the conference after it ends, and enables progress management that takes participants' emotions into account. Furthermore, speaking balance and timekeeping can also be effectively achieved.

[0456] The "conference theme" is a fundamental element of the content and purpose of the conference, and refers to the main issues to be discussed.

[0457] An "agenda" is a plan that shows the order and timetable of each topic in a meeting.

[0458] "Transcription" is the process of converting audio data into text format.

[0459] A "summary of speech content" is a text that briefly summarizes each statement made during the meeting.

[0460] The "speaking rate" is a numerical representation of the rate at which each participant spoke during the conference.

[0461] An "alert" is a warning or caution message that notifies the user when a specific condition occurs.

[0462] "Timekeeping" is the act of managing the progress of each segment of a meeting and adjusting it so that it stays within the scheduled time.

[0463] "Feedback" refers to the evaluations and opinions of the meeting collected from participants after the meeting has ended.

[0464] "Emotional tone" refers to the emotional state of each participant, analyzed from their voice and facial expressions, and indicates stress, relief, excitement, etc.

[0465] A "generative AI model" is a program that uses artificial intelligence techniques to generate new data or information based on a specific task.

[0466] A "prompt sentence" is an input sentence that instructs a generative AI model on the desired results or task content.

[0467] This invention is a system for effectively supporting the progress of meetings, providing consistent support from preparation to progress and evaluation after the meeting. The system is implemented through the interaction of a server, terminals, and users. Its main functions include generating an agenda, summarizing speech content, analyzing speech rate, recognizing emotional tone, timekeeping, and collecting and analyzing feedback.

[0468] server

[0469] The server is responsible for managing information and analyzing data for the entire conference. Specifically, it performs the following processes:

[0470] Generate the agenda:

[0471] The server generates a draft agenda based on the meeting theme and purpose entered by the user, using a generative AI model and referencing past meeting data. The user can then review and edit the draft agenda on their device, and it is finally saved on the server.

[0472] Example: A user inputs the theme of a "Monthly Report Meeting" and the purpose of "Checking Project Progress" and sends them to the server. The server generates agenda proposals such as "Checking Project Progress," "Introducing New Projects," and "Sharing Issues."

[0473] Summary of what was said:

[0474] The server transcribes the audio data received in real time during the conference and generates a summary using a natural language processing model, which is then sent to the terminal and displayed to the user.

[0475] Example: Speech during a meeting is transcribed in real time to generate text such as "Project A is progressing smoothly." This text is summarized to become "Progress on Project A has been reported."

[0476] Emotional Tone Recognition:

[0477] The server uses an emotion engine to analyze the voice and facial expressions of each participant in the meeting to recognize their emotional tone, and uses this data to generate alerts and suggestions for adjusting the progress of the meeting.

[0478] Example: An emotion engine detects that a particular participant is feeling stressed and sends out an alert saying, "We suggest you take a break."

[0479] Speech rate analysis:

[0480] The server analyzes each participant's speech rate in real time and generates alerts to maintain balance.

[0481] Example: If person A speaks too much, an alert will be sent to the device saying, "Please refrain from speaking, and give other participants a chance to speak."

[0482] Timekeeping:

[0483] The server manages the timer for each segment of the conference and sends timekeeping alerts as time passes.

[0484] Example: When there is little time left, an alert such as "3 minutes remaining" is sent to the device.

[0485] Feedback collection and analysis:

[0486] After the meeting, the server automatically generates a feedback form and sends it to each participant. The collected feedback is then analyzed by the server to extract important opinions and areas for improvement.

[0487] For example, if the feedback includes many comments such as "the discussion was biased," use this information to improve the next meeting.

[0488] Terminal

[0489] The terminal provides an interface for users to interact with the server. Users can perform the following operations through the terminal:

[0490] Agenda review and revision:

[0491] The user checks the agenda sent from the server on the device and modifies it as necessary. The modified agenda is then sent back to the server and saved.

[0492] Check what was said:

[0493] During the conference, users can check the summary sent from the server on their terminals and keep track of the progress.

[0494] Check your emotional tone:

[0495] The user can check the emotional tone information of the participants on the terminal and use it as a reference for conducting the appropriate proceedings.

[0496] Timekeeping:

[0497] The terminal displays the timekeeping alert sent from the server to notify the user of the need for time management.

[0498] Enter your feedback:

[0499] After the conference ends, users input feedback through their terminals, which is then sent to the server.

[0500] User

[0501] Users are primarily responsible for preparing, conducting, and evaluating meetings, including creating and modifying the agenda, checking comments made during the meeting, keeping time, and entering feedback.

[0502] Examples of concrete examples and prompts

[0503] Creating and reviewing the agenda

[0504] The user sends the theme of the "monthly report meeting" and the purpose of "checking the progress of the project" to the server.

[0505] The server uses a generative AI model to generate agenda proposals such as "checking project progress," "introducing new projects," and "sharing issues."

[0506] The user checks the agenda on the device and modifies it if necessary.

[0507] Example prompt sentence:

[0508] "Based on data from past monthly reporting meetings, please create an agenda that matches the following theme and purpose. Theme: Monthly reporting meeting, Purpose: Reviewing project progress."

[0509] Meeting management and management

[0510] The server transcribes the audio data in real time and summarizes the text using a natural language processing model.

[0511] The server uses an emotion engine to analyze the emotional tone of participants and generate alerts as needed.

[0512] Example prompt sentence:

[0513] Please transcribe and summarize the following conference audio:

[0514] Post-meeting evaluation and feedback

[0515] The server generates a feedback form and sends it to each participant.

[0516] The user inputs feedback through the terminal and transmits it to the server.

[0517] The server analyzes the feedback and extracts important improvements and suggestions.

[0518] Example prompt sentence:

[0519] "Collect feedback from participants after the meeting, analyze it, and generate a report."

[0520] In this way, the system of the present invention provides comprehensive support for the entire process from preparation to progress and evaluation of the meeting after it has concluded, and by taking into account the emotional state of the participants using an emotion engine, it is possible to significantly improve the quality of the meeting and the satisfaction of the participants.

[0521] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0522] Creating an agenda

[0523] Step 1:

[0524] The user inputs the meeting theme and purpose. The input contents are "Monthly report meeting" and "Project progress confirmation." The device receives this data as input and sends it to the server.

[0525] Step 2:

[0526] Based on the conference theme and purpose received by the server, past conference data is retrieved from the database, and related data is searched for and retrieved from the database.

[0527] Step 3:

[0528] The server uses the AI ​​model to analyze past meeting data and generate appropriate agenda proposals, such as "checking project progress," "introducing new projects," and "sharing issues."

[0529] Step 4:

[0530] The generated agenda plan is sent from the server to the device, where the user can check it.

[0531] Step 5:

[0532] The user modifies the agenda as necessary by inputting "additional items to be shared," and the terminal transmits the modified agenda to the server again.

[0533] Step 6:

[0534] The server saves the modified agenda.

[0535] Summary of what was said during the meeting

[0536] Step 1:

[0537] The device sends the audio data recorded during the meeting to the server in real time. The audio data is transmitted to the server in its original format (audio data format).

[0538] Step 2:

[0539] The server passes the received audio data to a transcription engine, which converts the audio data into text format. The resulting transcribed text data is generated.

[0540] Step 3:

[0541] The server inputs the transcribed text data into a natural language processing model to generate a summary, which is a short text such as "Progress on Project A has been reported."

[0542] Step 4:

[0543] The generated summary is sent from the server to the terminal and displayed to the user.

[0544] Emotion recognition by emotion engine

[0545] Step 1:

[0546] The terminals transmit the audio and video data of each participant recorded during the conference to the server, where the audio and video data is transmitted to the server in its original format.

[0547] Step 2:

[0548] The server passes the received audio and video data to the emotion engine, which analyzes it in real time. The analysis results in emotional tones (stress, relief, excitement, etc.) being output as emotion parameters.

[0549] Step 3:

[0550] The emotional tone data is stored on the server, and an alert is generated as needed. For example, if it is determined that "Participant A is feeling stressed," an alert is generated.

[0551] Step 4:

[0552] The generated alert is sent from the server to the terminal and displayed to the user.

[0553] Progress management and speech rate analysis

[0554] Step 1:

[0555] During a conference, the terminal sends each participant's speech data to the server. The speech data is transmitted to the server in its original format (text data format).

[0556] Step 2:

[0557] The server passes the received speech data to the analysis engine, which calculates the speech rate of each participant. The calculation results in numerical data, such as Participant A's speech rate being 70%.

[0558] Step 3:

[0559] The server generates alerts based on the speech rate data. For example, if a particular participant speaks too much, an alert will be output saying, "Please refrain from speaking, and give other participants a chance to speak."

[0560] Step 4:

[0561] The generated alert is sent from the server to the terminal and displayed to the user.

[0562] Timekeeping

[0563] Step 1:

[0564] The server sets a timer for each segment of the conference, based on the scheduled duration of each segment.

[0565] Step 2:

[0566] As the meeting progresses, the server monitors a timer that you set and generates a timekeeping alert when the specified time has elapsed.

[0567] Step 3:

[0568] The generated timekeeping alert is sent from the server to the device, and includes a notification such as "3 minutes remaining."

[0569] Step 4:

[0570] The user checks the alert on the terminal and adjusts the progress of the meeting.

[0571] Collecting and analyzing feedback

[0572] Step 1:

[0573] After the conference ends, the server automatically generates a feedback form and sends it to each participant via their terminal. The trigger for generating the form is the end of the conference.

[0574] Step 2:

[0575] Each participant fills out a feedback form using their device, and the input is sent to the server. For example, feedback such as "The meeting went smoothly" is provided as input.

[0576] Step 3:

[0577] The server passes the received feedback data to an analysis engine, which extracts important opinions and areas for improvement. The extracted data outputs areas for improvement such as "the discussion was biased."

[0578] Step 4:

[0579] The server generates a feedback report based on the analysis results, which contains information useful for preparing for the next meeting.

[0580] Step 5:

[0581] The generated feedback report is sent from the server to the terminal and provided to the user.

[0582] Through these specific processing steps, the system provides comprehensive support for meetings, from preparation to progress and post-conclusion evaluation, and uses an emotion engine and generative AI model to improve meeting quality and participant satisfaction.

[0583] (Application example 2)

[0584] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0585] While conventional meeting support systems automate the progress of meetings, summarizing comments, and timekeeping, they do not take into account the emotional state of participants in their progress management, making it difficult to run meetings with high levels of satisfaction.In particular, in situations where comments are biased or emotional stress is high, smooth discussions become difficult, hindering efficient problem-solving and decision-making.

[0586] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0587] In this invention, the server includes means for generating an agenda based on the conference theme and purpose, means for transcribing audio data during the conference and summarizing the content of speech, means for analyzing the speaking rate during the conference and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the conference ends, means for analyzing the content of speech and emotional tone in real time and using an emotion engine to reflect the emotional state of participants, and means for analyzing the emotions of participants as the conference progresses and making suggestions to adjust the progress according to the emotional tone, thereby enabling flexible and smooth conference management according to the emotional state of participants.

[0588] "Conference theme" refers to the subject or topic to be discussed at a conference.

[0589] An "agenda" refers to a list of items and schedules to be discussed at a meeting.

[0590] "Audio data" refers to audio information obtained by recording or acquiring in real time what participants say during a conference.

[0591] "Transcription" refers to the process of converting audio data into text form.

[0592] A "summary of what was said" refers to a short text that extracts important points and summaries based on the transcription data.

[0593] "Speaking rate" refers to data analyzed based on the speaking time and number of times each participant spoke during a meeting.

[0594] An "alert" refers to a notification that alerts a user when a specific condition occurs.

[0595] "Timekeeping" refers to monitoring the progress of each segment of a meeting and ensuring that the discussion is completed within the allotted time.

[0596] "Feedback" refers to information such as opinions, impressions, and areas for improvement collected from participants after the meeting has ended.

[0597] An "emotion engine" refers to a technology or system that analyzes the emotional tone of participants' voices and facial expressions to recognize their emotional state.

[0598] "Progress adjustment suggestions" refers to a function that suggests specific actions or changes to promote smoother discussions based on the results of analyzing participants' emotional states during a meeting.

[0599] A "generative AI model" refers to a mechanism that generates new data using artificial intelligence trained on large amounts of data.

[0600] A "prompt" is a pre-prepared text that prompts an AI to perform a specific action or generate something.

[0601] The present invention is a system for supporting the progress of a conference, and is implemented by the following steps: The system is composed of a server, terminals, and interactions between users.

[0602] System Configuration

[0603] 1. Server

[0604] The server plays a central role in realizing the following functions: managing meeting information, analyzing data, generating agendas, transcribing and summarizing speech content, analyzing speech rates, generating alerts, timekeeping, collecting and analyzing feedback, and recognizing emotions using an emotion engine.

[0605] 2. Terminal

[0606] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, view emotional tone, and enter feedback.

[0607] 3. Users

[0608] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[0609] Creating an agenda

[0610] The user inputs the meeting topic and purpose using a device. This information is sent to the server, which then uses a generative AI model based on past meeting data to generate a draft agenda. This draft agenda is then sent to the device for the user to review and modify.

[0611] Summary of remarks

[0612] During the meeting, the server receives the audio data and transcribes it in real time. This data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[0613] Emotion recognition by emotion engine

[0614] During the meeting, the server uses an emotion engine to analyze the voice and facial expressions of each participant in real time and recognize their emotional tone. This emotional tone is sent to the server as an emotion parameter and is used to generate alerts and manage the progress of the meeting.

[0615] Progress management and speech rate analysis

[0616] The server analyzes the speaking rate of each participant during the meeting. Based on the results of this analysis, if a particular participant is speaking excessively or the discussion is stagnating, the server generates an alert to balance the conversation. In addition, alerts to adjust the progress are also created based on the output of the emotion engine. These alerts are sent to the terminal and displayed to the user.

[0617] Timekeeping

[0618] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[0619] Collecting and analyzing feedback

[0620] After the meeting ends, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report that extracts areas for improvement and important opinions, which can be used to prepare for the next meeting. At this time, data from the emotion engine is also used in the feedback analysis, and the emotional reactions of the participants are also taken into consideration.

[0621] Specific examples

[0622] Creating and reviewing the agenda

[0623] For example, when a user prepares for a "monthly report meeting," they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses a generative AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can then review the generated agenda on their device, make any necessary edits, and save the final version.

[0624] Meeting management and management

[0625] When a meeting begins, the server receives and transcribes audio data in real time. The results are summarized using a natural language processing model, and users can view the summary on their devices. The server also monitors the speaking rate of each participant and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses an emotion engine to analyze participants' emotions and adjust the progress of the meeting, for example, by making suggestions to ease the discussion if stress is building. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[0626] Post-meeting feedback and analysis

[0627] After the meeting ends, the server generates a feedback form and sends it to each participant. Users enter their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report can be used to prepare for the next meeting. It also takes into account emotional data analyzed by the emotion engine and suggests improvements to address emotional reactions and stress points.

[0628] Prompt Sentence Examples

[0629] "The theme of the meeting is 'Considerations on Countermeasures to the Latest Threats.' Please generate an agenda."

[0630] "Perform emotion recognition on the 'Status Report' section."

[0631] As a result, this invention can provide a system that effectively and efficiently supports meetings from preparation to progress and evaluation after completion, and further uses an emotion engine to grasp the emotional states of participants, thereby improving the quality of the meeting.

[0632] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0633] Step 1:

[0634] The user inputs the conference theme and purpose into the terminal and transmits it to the server.

[0635] Input: Theme of "Monthly Report Meeting" and purpose of "Project Progress Review".

[0636] Output: The conference topic and objectives data sent to the server.

[0637] Specific operation: A user accesses the application on their terminal and enters the conference topic and purpose into the input form. The data is then sent to the database on the server.

[0638] Step 2:

[0639] The server generates a draft agenda using a generative AI model based on past meeting data.

[0640] Input: Past conference data and the conference topic and objectives entered in Step 1.

[0641] Output: The generated agenda proposal.

[0642] Specific operation: The AI ​​model in the server analyzes past meeting data stored in a database and generates agenda items related to the meeting theme and purpose. The generated agenda proposal is sent to the device.

[0643] Step 3:

[0644] The user checks the generated agenda on the terminal, modifies it if necessary, and sends the modified agenda draft back to the server.

[0645] Input: Generated agenda proposal and user modifications.

[0646] Output: Final draft agenda.

[0647] Specific operation: The user checks the draft agenda on the device screen and edits it if necessary. The final version of the agenda is sent to the server and saved.

[0648] Step 4:

[0649] During the meeting, the server receives the audio data and transcribes it in real time.

[0650] Input: Audio data during the meeting.

[0651] Output: Transcribed text data.

[0652] How it works: The microphone on the device picks up the voices of the meeting participants, and the voice data is sent to the server, which then transcribes it using a speech recognition system.

[0653] Step 5:

[0654] The transcribed text data is passed to a natural language processing model to generate a summary.

[0655] Input: Transcribed text data.

[0656] Output: Summarized text data.

[0657] How it works: The server inputs the transcription data into a natural language processing model and generates a resulting summary, which is then sent to the device.

[0658] Step 6:

[0659] The server uses an emotion engine to analyze participants' voices and facial expressions in real time and recognize their emotional tone.

[0660] Input: Participant's voice and facial expression data.

[0661] Output: Emotional tone (emotion parameters).

[0662] How it works: The server uses a voice recognition system and an image analysis system to analyze the emotional tone of participants' voices and facial expressions. This information is reflected in real time and recorded as emotion parameters.

[0663] Step 7:

[0664] The server analyzes emotional tone and speech rate to detect excessive speech or stalled discussions and generate alerts.

[0665] Input: Emotional tone data and speech rate data.

[0666] Output: Alert notification.

[0667] Specific operation: The server uses a speech rate analysis algorithm to monitor the volume of speech of each participant and generates an alert if there is an imbalance. It also generates an alert if the emotional tone is negative. The generated alert is sent to the terminal and notifies the user.

[0668] Step 8:

[0669] The server manages the timer for each segment of the meeting and sends timekeeping alerts depending on the time remaining.

[0670] Inputs: Meeting segment start time and current time.

[0671] Output: Timekeeping alert.

[0672] Specific operation: The server calculates the elapsed time from the start time of each segment and manages the timer to ensure that the discussion proceeds within the set time. When the remaining time is running low, an alert is generated and sent to the terminal.

[0673] Step 9:

[0674] After the conference ends, the server automatically generates a feedback form and sends it to the participants. Users can then enter their feedback using their devices and send it to the server.

[0675] Input: Feedback after the meeting.

[0676] Output: Collected feedback data.

[0677] Specific operation: At the end of the conference, the server automatically generates a feedback form and sends it to the participants' devices. The users enter their feedback in the form and submit it. The server then analyzes the received data.

[0678] Step 10:

[0679] The server analyzes the collected feedback data and generates a feedback report that extracts areas for improvement and important opinions.

[0680] Input: Collected feedback data.

[0681] Output: Feedback report.

[0682] How it works: The server inputs the feedback data into a data analysis algorithm to extract areas for improvement and important opinions. Based on the extracted information, a feedback report is generated to help prepare for the next meeting.

[0683] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0684] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0685] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0686] [Second embodiment]

[0687] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0688] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0689] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0690] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0691] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0692] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0693] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0694] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0695] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0696] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0697] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0698] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0699] The present invention provides a system for supporting the progress of a conference, specifically providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction between a server, terminals, and users.

[0700] System Configuration

[0701] 1. Server

[0702] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[0703] 2. Terminal

[0704] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, and enter feedback.

[0705] 3. Users

[0706] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[0707] Program processing

[0708] Creating an agenda

[0709] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[0710] Summary of remarks

[0711] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[0712] Progress management and speech rate analysis

[0713] The server analyzes the speech rate of each participant during the meeting. Based on this analysis, if a specific participant is speaking excessively or the discussion is stagnating, the server generates an alert and sends it to the device. The device then displays this alert to the user, prompting them to take the necessary action.

[0714] Timekeeping

[0715] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[0716] Collecting and analyzing feedback

[0717] After the meeting, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback, extracts areas for improvement and important opinions, and generates a feedback report to help prepare for the next meeting.

[0718] Specific examples

[0719] Creating and reviewing the agenda

[0720] When a user prepares for a monthly report meeting, they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses an AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can review the generated agenda on their device, make any necessary additions or corrections, and then resubmit and save the final version.

[0721] Meeting management and management

[0722] When a meeting begins, the server receives and transcribes audio data in real time. It then uses a natural language processing model to summarize the results, which users can view on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[0723] Post-meeting evaluation and feedback

[0724] After the meeting ends, the server creates a feedback form and sends it to each participant. Users fill out their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report is provided to users to help them prepare for the next meeting.

[0725] In this way, the present invention provides a system that efficiently and effectively supports meetings from preparation to progress and evaluation after completion.

[0726] The processing flow will be explained below.

[0727] Creating an agenda

[0728] Step 1:

[0729] The user uses the terminal to input the conference theme and purpose, and transmits them to the server.

[0730] Step 2:

[0731] Based on the received theme and purpose, the server references past meeting data and uses an AI model to generate a draft agenda.

[0732] Step 3:

[0733] The generated agenda plan is sent from the server to the terminal and displayed to the user.

[0734] Step 4:

[0735] The user checks the draft agenda on the device and makes corrections as necessary.

[0736] Step 5:

[0737] The revised agenda is sent back to the server and saved as the final version.

[0738] Summary of remarks

[0739] Step 1:

[0740] The server receives the audio data in real time during the conference.

[0741] Step 2:

[0742] The server automatically transcribes the received audio data.

[0743] Step 3:

[0744] The server passes the transcribed data to a natural language processing model to generate a summary of what was said.

[0745] Step 4:

[0746] The generated summary is sent from the server to the terminal and displayed to the user.

[0747] Analysis of speech rate and generation of alerts

[0748] Step 1:

[0749] The server analyzes each participant's speaking rate in real time during the meeting.

[0750] Step 2:

[0751] Based on the analysis results, if a particular participant is speaking excessively, the server will generate an alert to balance things out.

[0752] Step 3:

[0753] The generated alert is sent from the server to the terminal and displayed to the user.

[0754] Timekeeping

[0755] Step 1:

[0756] The server sets a timer for each segment of the conference.

[0757] Step 2:

[0758] When the time approaches the specified time remaining, the server generates a timekeeping alert.

[0759] Step 3:

[0760] The generated timekeeping alert is sent from the server to the terminal and displayed to the user.

[0761] Collecting and analyzing feedback

[0762] Step 1:

[0763] After the meeting ends, the server automatically generates a feedback form.

[0764] Step 2:

[0765] The generated feedback form is sent from the server to the participant's terminal.

[0766] Step 3:

[0767] The user fills in a feedback form on the terminal and sends it to the server.

[0768] Step 4:

[0769] The server analyzes the received feedback and extracts areas for improvement and important opinions.

[0770] Step 5:

[0771] Based on the extracted information, the server generates a feedback report and sends it to the terminal to help prepare for the next meeting.

[0772] As described above, the server, terminals, and users work together in each step to realize a system that efficiently prepares for, progresses, and evaluates a conference after it has ended.

[0773] Example 1

[0774] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0775] In today's business environment, effective meeting management and proceedings are crucial. However, the entire process, from preparing for meetings to conducting them and evaluating them after they have finished, requires a significant amount of time and effort due to manual work. In particular, tasks such as creating agendas, summarizing comments, balancing speaking ratios, timekeeping, and collecting and analyzing feedback are complex and difficult to carry out efficiently. A consistent automated system to solve these issues is needed.

[0776] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0777] In this invention, the server includes: means for a user to input the meeting theme and purpose and send the input information to the server; means for the server to refer to past meeting data and generate a draft agenda using a generative AI model and send the generated draft agenda to the terminal; means for the user to check the draft agenda on the terminal and resend the revised agenda to the server for storage; means for the server to receive audio data from the meeting and transcribe it in real time; means for the server to pass the transcribed speech content to a natural language processing model to generate a summary and send the generated summary to the terminal; means for the server to analyze the speech rate of each participant in the meeting in real time, generate an alert based on the analysis result, and send the generated alert to the terminal; means for a user to set a timer for the agenda from the terminal, the server to monitor the timer, generate an alert when the timeout is approaching, and send the alert to the terminal; means for the server to automatically generate a feedback form after the meeting ends and send the feedback to the user; and means for the user to input feedback and send it to the server, the server to analyze the collected feedback, generate a feedback report, and provide the generated report to the user. This allows the entire process from preparing for a meeting to proceeding and evaluation after it has ended to be efficiently automated, reducing the burden on users.

[0778] A "user" is a person who operates a terminal to prepare for a conference, manage its progress, and evaluate it after it has ended.

[0779] "Terminal" means a device that provides an interface for a User to interact with the Server, allowing the User to view and modify the agenda, review comments, track progress, receive timekeeping alerts, and enter feedback.

[0780] The "server" is a central processing unit responsible for meeting information management, data analysis, agenda generation, speech transcription and summarization, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[0781] An "agenda" is a list of topics to be discussed to facilitate the progress of a meeting, and is generated based on the theme and purpose of the meeting.

[0782] A "generative AI model" is an artificial intelligence model that learns from past meeting data and refers to an algorithm for automatically generating a draft agenda based on the meeting theme and purpose entered by the user.

[0783] "Audio data" refers to data that records what is said during a meeting and is later transcribed.

[0784] "Transcription" refers to the process of converting audio data into text data.

[0785] A "natural language processing model" refers to an algorithm for analyzing and summarizing transcribed text data, and it performs text analysis by learning from a large amount of text data.

[0786] The "speaking rate" is an index showing the proportion of time each participant spoke during the conference.

[0787] An "alert" is a notification that is generated based on the progress of the conference and the balance of speech among the participants, and is used to alert the user.

[0788] A "timer" is a device that measures the time allocated to each agenda item in a meeting and manages the progress to be made within the specified time.

[0789] A "feedback form" is an electronic form containing questions to collect opinions and feedback from participants after the meeting.

[0790] A "feedback report" is a report that analyzes the collected feedback and summarizes areas for improvement in the meeting and important opinions from participants.

[0791] The present invention is a system for supporting the progress of a conference, providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction of a server, terminals, and users.

[0792] System Configuration

[0793] server

[0794] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis. Specifically, the server transcribes audio data using Google Cloud Speech-to-Text API or AWS Transcribe, summarizes speech content using natural language processing models (e.g., BERT), generates agendas using generative AI models (e.g., GPT-3), and references past meeting data.

[0795] Terminal

[0796] The terminal provides an interface for users to interact with the server. Through the terminal, users can check and modify the agenda, check what has been said, track progress, receive timekeeping alerts, and enter feedback. The terminal receives data from the server and displays it to the user.

[0797] User

[0798] Users operate the device to prepare for the meeting, manage its progress, and evaluate it after it has ended. Specifically, users input the meeting theme and purpose from the device and send them to the server. During the meeting, users can check summaries and alerts generated in real time through the device and take appropriate action. After the meeting, users can use the device to enter feedback and send it to the server.

[0799] Specific examples

[0800] Creating and reviewing the agenda

[0801] When a user prepares for a monthly report meeting, they send the theme "Monthly Report Meeting" and the purpose "Check Project Progress" from their device to the server. The server uses a generative AI model (e.g., GPT-3) based on past meeting data to generate a draft agenda, creating items such as "Check Project Progress," "Introduce New Projects," and "Share Issues." The user then reviews the generated agenda on their device, makes any necessary corrections, and resubmits it to save the final version.

[0802] Meeting management and management

[0803] When the meeting begins, the server receives and transcribes audio data in real time. For example, it uses the Google Cloud Speech-to-Text API to transcribe the audio data. The results are summarized using a natural language processing model (e.g., BERT), and users can view the summary on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal speaking. Furthermore, a timekeeping function is used to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[0804] Post-meeting evaluation and feedback

[0805] After the meeting ends, the server automatically generates a feedback form and sends it to the user. The user enters feedback such as "The meeting went smoothly" or "The comments were well-balanced" into the feedback form on their device. The server analyzes the collected feedback using a natural language processing model and generates a feedback report that extracts areas for improvement and important opinions. This report is provided to the user to help them prepare for the next meeting.

[0806] Prompt Sentence Examples

[0807] Creating an agenda: "Please create an agenda for the next monthly report meeting with the theme 'Monthly Report' and the purpose 'Checking project progress.'"

[0808] Summary Generation: "Generate a summary of the meeting based on the following transcription: [transcription]"

[0809] Feedback collection: "Automatically generate a form to collect feedback from participants after the meeting, analyze the input data, and create a feedback report."

[0810] By integrating the above functions, the present invention provides a comprehensive system for efficiently and effectively supporting meetings from preparation to progress and evaluation after completion.

[0811] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0812] Program processing flow

[0813] Creating an agenda

[0814] Step 1:

[0815] The user inputs the meeting theme "Project progress review meeting" and the purpose "Project progress review" via the terminal.

[0816] Input: Conference theme and objectives

[0817] Output: Notification of completion of input to the terminal

[0818] Step 2:

[0819] The terminal sends this input information to the server.

[0820] Input: User input information

[0821] Output: Data transfer to the server

[0822] Step 3:

[0823] The server retrieves past meeting data from a database and inputs it into a generative AI model (e.g., GPT-3).

[0824] Input: User input information, past meeting data

[0825] Output: Agenda proposal from the AI ​​model

[0826] Step 4:

[0827] The server sends the generated agenda plan to the terminal so that the user can check it.

[0828] Input: Agenda proposal from AI model

[0829] Output: Send the agenda to the terminal

[0830] Step 5:

[0831] The user checks the draft agenda on the device and modifies it as necessary.

[0832] Input: Agenda proposal from server

[0833] Output: User's modified data

[0834] Step 6:

[0835] The terminal sends the modified agenda back to the server, which stores it.

[0836] Input: User's revision agenda

[0837] Output: Save modified data to the server

[0838] Summary of remarks

[0839] Step 1:

[0840] The server receives the conference audio data in real time.

[0841] Input: Meeting audio data

[0842] Output: Real-time speech-to-text request

[0843] Step 2:

[0844] The server uses the Google Cloud Speech-to-Text API to transcribe the audio data.

[0845] Input: Meeting audio data

[0846] Output: Transcript

[0847] Step 3:

[0848] The server passes the transcribed text to a natural language processing model (e.g., BERT) to generate a summary.

[0849] Input: Transcription text

[0850] Output: Meeting summary text

[0851] Step 4:

[0852] The server sends the generated summary to the terminal.

[0853] Input: Meeting summary text

[0854] Output: Sends text to the terminal

[0855] Step 5:

[0856] The user checks the summary on the terminal and manages the progress of the conference.

[0857] Input: Meeting summary text from the server

[0858] Output: Information for managing the meeting progress

[0859] Progress management and speech rate analysis

[0860] Step 1:

[0861] The server monitors each participant's comments in real time.

[0862] Input: Conference audio data

[0863] Output: Speaker identification data

[0864] Step 2:

[0865] The server tallies the speaking time of each participant and calculates the speaking rate.

[0866] Input: Speaker identification data

[0867] Output: Speech rate data

[0868] Step 3:

[0869] The server generates an alert if a particular participant is speaking too much.

[0870] Input: Speech rate data

[0871] Output: Alert generation request

[0872] Step 4:

[0873] The server sends the generated alert to the terminal and notifies the user.

[0874] Input: Alert information

[0875] Output: Notification to terminal

[0876] Step 5:

[0877] The user checks the alert and adjusts the meeting progress if necessary.

[0878] Input: Alert Notification

[0879] Output: Coordination of meeting progress

[0880] Timekeeping

[0881] Step 1:

[0882] The user sets the timer for each agenda item.

[0883] Input: Set time for each agenda item

[0884] Output: Terminal sends setting values ​​to server

[0885] Step 2:

[0886] The server monitors the timer and measures the progress of each agenda item.

[0887] Input: Set time

[0888] Output: Timer measurement value

[0889] Step 3:

[0890] Generate an alert if the server is running low on time.

[0891] Input: Timer measurement value

[0892] Output: Timekeeping alert

[0893] Step 4:

[0894] The server sends the generated alert to the terminal and notifies the user.

[0895] Input: Timekeeping Alert

[0896] Output: Notification to terminal

[0897] Step 5:

[0898] The user acknowledges the alert and moves on to the next agenda item.

[0899] Input: Timekeeping Alert Notification

[0900] Output: Meeting agenda

[0901] Collecting and analyzing feedback

[0902] Step 1:

[0903] The server automatically generates a feedback form after the meeting ends.

[0904] Input: Meeting end information

[0905] Output: An automatically generated feedback form

[0906] Step 2:

[0907] The server sends the feedback form to the user.

[0908] Input: Auto-generated feedback form

[0909] Output: Feedback form sent to terminal

[0910] Step 3:

[0911] The user inputs feedback on the terminal and transmits it to the server.

[0912] Input: User feedback

[0913] Output: Send feedback to the server

[0914] Step 4:

[0915] The server analyzes the collected feedback.

[0916] Input: User feedback data

[0917] Output: Parsed feedback results

[0918] Step 5:

[0919] The server generates a feedback report and provides it to the user.

[0920] Input: Parsed feedback results

[0921] Output: Feedback report

[0922] Step 6:

[0923] Users can review the feedback report to help them prepare for their next meeting.

[0924] Input: Feedback report from the server

[0925] Output: Capture improvements and feedback

[0926] (Application example 1)

[0927] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0928] In factory maintenance meetings, inefficiencies in progress and information sharing are common, leading to reduced work efficiency and miscommunication. It's particularly difficult to grasp the content of comments made during meetings and the progress of work in real time, and to provide appropriate feedback and time management. Given these issues, a system is needed to ensure effective meeting progress and real-time information sharing.

[0929] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0930] In this invention, the server includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speech, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speech and work status in real time during a factory maintenance meeting and displaying it on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it. This enables more efficient meeting progress and real-time information sharing.

[0931] A "conference theme" is the main subject or topic that will be the focus of discussion at a conference.

[0932] "Purpose" refers to the goal or intention that the meeting is trying to achieve.

[0933] An "agenda" is a list of topics or matters to be discussed at a meeting and indicates the plan of action.

[0934] "Audio data" is a recording of speech spoken during a conference.

[0935] "Transcription" is the process of converting audio data into text data.

[0936] "Speech content" refers to the specific content or information spoken by participants during a meeting.

[0937] A "summary" is a text that condenses the content of a statement or information and extracts its main points.

[0938] The "speaking rate" indicates the rate or frequency at which each participant spoke during the meeting.

[0939] An "alert" is a notification or warning that is issued when a specific condition is met.

[0940] "Timekeeping" means managing the progress of meetings and work.

[0941] "Feedback" refers to the opinions and evaluations collected from participants after the meeting has ended.

[0942] "Analysis" refers to the detailed analysis of collected data to derive patterns and trends.

[0943] A "factory maintenance meeting" is a meeting held within a factory to discuss equipment maintenance and improvements.

[0944] A "smart device" is an electronic device that can connect to the Internet and display and obtain information.

[0945] "Progress management" means managing the arrangements and progress of meetings and work.

[0946] "Real-time" refers to data being processed immediately, without delay.

[0947] "Information sharing" means that multiple people share information and deepen their understanding of each other.

[0948] This invention is a system for supporting the progress of factory maintenance meetings, which is realized by the interaction of a server, a terminal, and a user. The system includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speeches, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speeches and work status during the factory maintenance meeting in real time and displaying them on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it.

[0949] Server Features

[0950] The server generates an agenda based on the meeting theme and purpose by inputting past meeting data into the AI ​​model. By inputting the meeting theme "Monthly maintenance meeting" and the purpose "Checking equipment status and planning the next maintenance" as prompts, the AI ​​model generates agendas such as "Reporting equipment status," "Planning the next maintenance," and "Sharing issues."

[0951] In addition, the system transcribes audio data during meetings in real time and summarizes it using natural language processing (NLP) models. It continuously collects audio during meetings, instantly converts it into text, and sends the summary to a smart device.

[0952] The server analyzes the speaking rate of each participant and generates an alert if there is an imbalance. For example, if a particular participant speaks excessively, an alert will be generated based on that information and a notification will be sent to the smart device.

[0953] Use the timekeeping function to manage the progress of meetings. Based on the time set for each agenda item, when the remaining time is running low, a timekeeping alert will be generated and a notification will be sent to the smart device. In this way, users can review the progress plan in a timely manner.

[0954] After the meeting, the server generates a feedback form and sends it to each participant. The collected feedback data is analyzed and a feedback report is generated to extract improvements for the next maintenance meeting.

[0955] Device and user operations

[0956] Users operate their terminals (smart devices) to check and modify the agenda, check what is being said, understand the progress, receive timekeeping alerts, and enter feedback. For example, when preparing for a monthly maintenance meeting, a user sends the theme of the "monthly maintenance meeting" and the purpose of "checking the equipment status and planning the next maintenance" to the server from their terminal.

[0957] Smart devices have the ability to display real-time transcriptions and summaries to users, specifically, by displaying the transcriptions, summaries, and progress management alerts on the user's smart glasses or head-mounted display.

[0958] Hardware and software used

[0959] The hardware used includes a server, smart glasses, a head-mounted display, and a microphone. The software used includes an AI model, a natural language processing (NLP) model, and a voice recognition library. Specifically, "some_ai_library" and "some_voice_recognition_library" are used.

[0960] Prompt Sentence Examples

[0961] "Topic: Monthly maintenance meeting, Purpose: Checking equipment status and planning the next maintenance"

[0962] In this way, the present invention provides a system that efficiently and effectively supports the preparation, progress, and evaluation of a factory maintenance meeting.

[0963] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0964] Step 1:

[0965] The server receives the meeting theme and purpose from the user. Specifically, the user uses a terminal (smart device) to input the theme, "Monthly Maintenance Meeting," and the purpose, "Checking the equipment status and planning the next maintenance." The input information is sent to the server. The input is text data of the theme and purpose, and the output is the text data sent to the server.

[0966] Step 2:

[0967] The server inputs the received theme and purpose into the generative AI model as prompt sentences. The server references past meeting data and generates a draft agenda using the AI ​​model. Specific data processing involves generating prompt sentences and searching past data, while data calculation involves generating an agenda using the AI ​​model. The input is text data of the theme and purpose, and the output is the generated agenda.

[0968] Step 3:

[0969] The server sends the generated agenda to the terminal, where the user can confirm and modify its contents. The user confirms the agenda on the terminal (smart device) and modifies it as necessary. The modified agenda is then sent back to the server and saved. The input is the text data of the generated agenda, and the output is the text data of the confirmed and modified agenda.

[0970] Step 4:

[0971] The server collects voice data in real time and transcribes it using a speech recognition library. Specifically, the server converts voice data collected by a microphone during a meeting into text data in real time. The input is voice data, and the output is transcribed text data.

[0972] Step 5:

[0973] The server passes the transcribed text data to a natural language processing (NLP) model to generate a summary. The server uses the NLP model to extract and summarize the key points of the text data. The input is the transcribed text data, and the output is the summarized text data.

[0974] Step 6:

[0975] The server sends the generated summary to the terminal and displays it to the user. The user can check the summary in real time during the meeting through their smart device. The input is the summarized text data, and the output is the summary displayed on the smart device.

[0976] Step 7:

[0977] The server analyzes the speech rate of each participant and generates alerts to balance the rate. Specifically, it analyzes audio data, measures the speaking time of each participant, and generates an alert if a specific participant is speaking excessively. The input is audio data, and the output is alert data.

[0978] Step 8:

[0979] The server measures the remaining time based on the timekeeping setting and generates a timekeeping alert. The server monitors the progress of the conference and generates an alert when the specified time is reached and sends it to the terminal. The input is the set time and the current time, and the output is the timekeeping alert.

[0980] Step 9:

[0981] After the conference ends, the server generates a feedback form and sends it to each participant. Users fill out feedback using their terminals and send it to the server. The input is the feedback form, and the output is the user's feedback data.

[0982] Step 10:

[0983] The server analyzes the collected feedback data and generates a feedback report. The server analyzes the feedback data and generates a report that extracts areas for improvement and important opinions, which is useful for preparing for the next meeting. The input is feedback data and the output is a feedback report.

[0984] The above processing steps make it possible to realize a system that efficiently and effectively supports the entire process from preparation to progress and evaluation after a meeting.

[0985] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0986] This invention is a system for supporting the progress of a conference. Specifically, it provides multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended, and further incorporates an emotion engine that recognizes the user's emotions to improve the quality of the conference and the satisfaction of the participants. The system is realized through the interaction between a server, terminals, and users.

[0987] System Configuration

[0988] 1. Server

[0989] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, feedback collection and analysis, and emotion recognition using an emotion engine.

[0990] 2. Terminal

[0991] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, view emotional tone, and enter feedback.

[0992] 3. Users

[0993] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[0994] Program processing

[0995] Creating an agenda

[0996] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[0997] Summary of remarks

[0998] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[0999] Emotion recognition by emotion engine

[1000] During the meeting, the server uses an emotion engine to analyze the voice and facial expressions of each participant in real time and recognize their emotional tone. This emotional tone is sent to the server as an emotion parameter and is used to generate alerts and manage the progress of the meeting.

[1001] Progress management and speech rate analysis

[1002] The server analyzes the speaking rate of each participant during the meeting. Based on the results of this analysis, if a particular participant is speaking excessively or the discussion is stagnating, the server generates an alert to balance the situation. In addition, the server appropriately adjusts the progress based on the output of the emotion engine. The alerts generated by the server are sent to the terminal and displayed to the user.

[1003] Timekeeping

[1004] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[1005] Collecting and analyzing feedback

[1006] After the meeting ends, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback, extracts areas for improvement and important opinions, and generates a feedback report to help prepare for the next meeting. During this process, data from the emotion engine is also used in the feedback analysis, taking into account the emotional reactions of the participants.

[1007] Specific examples

[1008] Creating and reviewing the agenda

[1009] When a user prepares for a monthly report meeting, they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses an AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can review the generated agenda on their device, make any necessary additions or corrections, and then resubmit and save the final version.

[1010] Meeting management and management

[1011] When a meeting begins, the server receives and transcribes audio data in real time. The results are summarized using a natural language processing model, and users can view the summary on their devices. The server also monitors the speaking rate of each participant and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses an emotion engine to analyze participants' emotions and adjust the progress of the meeting, for example, by making suggestions to ease the discussion if stress is building. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[1012] Post-meeting evaluation and feedback

[1013] After the meeting ends, the server creates a feedback form and sends it to each participant. Users fill out their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report is provided to users to help them prepare for the next meeting. At this time, the server also takes into account the emotional data analyzed by the emotion engine and suggests improvements to address emotional reactions and stress points.

[1014] In this way, the present invention provides a system that effectively and efficiently supports meetings from preparation to progress and evaluation after completion, and further uses an emotion engine to grasp the emotional state of participants and improve the quality of the meeting.

[1015] The processing flow will be explained below.

[1016] Creating an agenda

[1017] Step 1:

[1018] The user uses the terminal to input the conference theme and purpose, and transmits them to the server.

[1019] Step 2:

[1020] Based on the received theme and purpose, the server references past meeting data and uses an AI model to generate a draft agenda.

[1021] Step 3:

[1022] The generated agenda plan is sent from the server to the terminal and displayed to the user.

[1023] Step 4:

[1024] The user checks the draft agenda on the device and makes corrections as necessary.

[1025] Step 5:

[1026] The revised agenda is sent back to the server and saved as the final version.

[1027] Summary of remarks

[1028] Step 1:

[1029] The server receives the audio data in real time during the conference.

[1030] Step 2:

[1031] The server automatically transcribes the received audio data.

[1032] Step 3:

[1033] The server passes the transcribed data to a natural language processing model to generate a summary of what was said.

[1034] Step 4:

[1035] The generated summary is sent from the server to the terminal and displayed to the user.

[1036] Emotion recognition by emotion engine

[1037] Step 1:

[1038] The server receives the audio and video data in real time.

[1039] Step 2:

[1040] Based on the received data, the emotion engine analyzes the emotions of each participant.

[1041] Step 3:

[1042] The analyzed emotional tones are sent to the server as emotion parameters.

[1043] Step 4:

[1044] Based on the results of the emotional tone analysis, the server uses this information for progress management and alert generation.

[1045] Analysis of speech rate and generation of alerts

[1046] Step 1:

[1047] The server analyzes the speaking rate of each participant in the meeting in real time.

[1048] Step 2:

[1049] Based on the analysis results, if a particular participant is speaking excessively, the server will generate an alert to balance things out.

[1050] Step 3:

[1051] The generated alert is sent from the server to the terminal and displayed to the user.

[1052] Timekeeping

[1053] Step 1:

[1054] The server sets a timer for each segment of the conference.

[1055] Step 2:

[1056] When the time approaches the specified time remaining, the server generates a timekeeping alert.

[1057] Step 3:

[1058] The generated timekeeping alert is sent from the server to the terminal and displayed to the user.

[1059] Collecting and analyzing feedback

[1060] Step 1:

[1061] After the meeting ends, the server automatically generates a feedback form.

[1062] Step 2:

[1063] The generated feedback form is sent from the server to the participant's terminal.

[1064] Step 3:

[1065] The user fills in a feedback form on the terminal and sends it to the server.

[1066] Step 4:

[1067] The server analyzes the received feedback and extracts areas for improvement and important opinions.

[1068] Step 5:

[1069] Based on the extracted information, the server generates a feedback report and sends it to the terminal to help prepare for the next meeting.

[1070] Specific examples

[1071] Creating and reviewing the agenda

[1072] Step 1:

[1073] The user transmits the theme of the "monthly report meeting" and the purpose of "checking the progress of the project" from the terminal to the server.

[1074] Step 2:

[1075] The server uses an AI model based on past meeting data to generate a draft agenda.

[1076] Step 3:

[1077] For example, create items such as "Check project progress," "Introduce new projects," and "Share issues."

[1078] Step 4:

[1079] The user can review the generated agenda on their device and make any necessary additions or corrections.

[1080] Step 5:

[1081] The revised agenda is sent back to the server and the final version is saved.

[1082] Meeting management and management

[1083] Step 1:

[1084] When the meeting begins, the server receives audio and video data in real time and transcribes it.

[1085] Step 2:

[1086] The results are summarized using a natural language processing model, and users can view the summary on their device.

[1087] Step 3:

[1088] The server monitors the speaking rate of each participant and generates an alert and sends it to the terminal if excessive speaking is detected to encourage equal speaking.

[1089] Step 4:

[1090] The server uses an emotion engine to analyze participants' emotions and uses the results to manage the discussion, for example by making suggestions to ease the discussion if stress levels are rising.

[1091] Step 5:

[1092] Use the timekeeping function to manage the progress of each agenda item, and when time is approaching, an alert will be issued to prompt you to move on to the next agenda item.

[1093] Post-meeting evaluation and feedback

[1094] Step 1:

[1095] After the conference ends, the server automatically generates a feedback form and sends it to the participants.

[1096] Step 2:

[1097] The user fills in a feedback form from the terminal and sends it to the server.

[1098] Step 3:

[1099] The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions.

[1100] Step 4:

[1101] A feedback report is provided to the user to help prepare for the next meeting.

[1102] Step 5:

[1103] In addition, the emotional data analyzed by the emotion engine is taken into account and improvements are suggested to address emotional reactions and stress points.

[1104] Example 2

[1105] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1106] Conventional meeting support systems have had difficulty in providing consistent support for everything from meeting preparation to progress management and post-meeting evaluation. Furthermore, progress management did not take into account the emotional state of participants, which led to problems that could lead to a decline in meeting quality and participant satisfaction. Furthermore, balance of speech and timekeeping were inadequate. There is a need for a system that can solve these issues.

[1107] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1108] In this invention, the server includes a means for generating an agenda based on the conference theme and purpose, a means for transcribing audio data during the conference and summarizing the content of speeches, a means for analyzing the speaking rate during the conference and generating alerts for balancing speaking rates, a means for managing timekeeping and sending notifications according to the remaining time, a means for collecting and analyzing feedback after the conference ends, and a means for analyzing participants' emotions during the conference and recognizing their emotional tone. This provides comprehensive support for preparing, conducting, and evaluating the conference after it ends, and enables progress management that takes participants' emotions into account. Furthermore, speaking balance and timekeeping can also be effectively achieved.

[1109] The "conference theme" is a fundamental element of the content and purpose of the conference, and refers to the main issues to be discussed.

[1110] An "agenda" is a plan that shows the order and timetable of each topic in a meeting.

[1111] "Transcription" is the process of converting audio data into text format.

[1112] A "summary of speech content" is a text that briefly summarizes each statement made during the meeting.

[1113] The "speaking rate" is a numerical representation of the rate at which each participant spoke during the conference.

[1114] An "alert" is a warning or caution message that notifies the user when a specific condition occurs.

[1115] "Timekeeping" is the act of managing the progress of each segment of a meeting and adjusting it so that it stays within the scheduled time.

[1116] "Feedback" refers to the evaluations and opinions of the meeting collected from participants after the meeting has ended.

[1117] "Emotional tone" refers to the emotional state of each participant, analyzed from their voice and facial expressions, and indicates stress, relief, excitement, etc.

[1118] A "generative AI model" is a program that uses artificial intelligence techniques to generate new data or information based on a specific task.

[1119] A "prompt sentence" is an input sentence that instructs a generative AI model on the desired results or task content.

[1120] This invention is a system for effectively supporting the progress of meetings, providing consistent support from preparation to progress and evaluation after the meeting. The system is implemented through the interaction of a server, terminals, and users. Its main functions include generating an agenda, summarizing speech content, analyzing speech rate, recognizing emotional tone, timekeeping, and collecting and analyzing feedback.

[1121] server

[1122] The server is responsible for managing information and analyzing data for the entire conference. Specifically, it performs the following processes:

[1123] Generate the agenda:

[1124] The server generates a draft agenda based on the meeting theme and purpose entered by the user, using a generative AI model and referencing past meeting data. The user can then review and edit the draft agenda on their device, and it is finally saved on the server.

[1125] Example: A user inputs the theme of a "Monthly Report Meeting" and the purpose of "Checking Project Progress" and sends them to the server. The server generates agenda proposals such as "Checking Project Progress," "Introducing New Projects," and "Sharing Issues."

[1126] Summary of what was said:

[1127] The server transcribes the audio data received in real time during the conference and generates a summary using a natural language processing model, which is then sent to the terminal and displayed to the user.

[1128] Example: Speech during a meeting is transcribed in real time to generate text such as "Project A is progressing smoothly." This text is summarized to become "Progress on Project A has been reported."

[1129] Emotional Tone Recognition:

[1130] The server uses an emotion engine to analyze the voice and facial expressions of each participant in the meeting to recognize their emotional tone, and uses this data to generate alerts and suggestions for adjusting the progress of the meeting.

[1131] Example: An emotion engine detects that a particular participant is feeling stressed and sends out an alert saying, "We suggest you take a break."

[1132] Speech rate analysis:

[1133] The server analyzes each participant's speech rate in real time and generates alerts to maintain balance.

[1134] Example: If person A speaks too much, an alert will be sent to the device saying, "Please refrain from speaking, and give other participants a chance to speak."

[1135] Timekeeping:

[1136] The server manages the timer for each segment of the conference and sends timekeeping alerts as time passes.

[1137] Example: When there is little time left, an alert such as "3 minutes remaining" is sent to the device.

[1138] Feedback collection and analysis:

[1139] After the meeting, the server automatically generates a feedback form and sends it to each participant. The collected feedback is then analyzed by the server to extract important opinions and areas for improvement.

[1140] For example, if the feedback includes many comments such as "the discussion was biased," use this information to improve the next meeting.

[1141] Terminal

[1142] The terminal provides an interface for users to interact with the server. Users can perform the following operations through the terminal:

[1143] Agenda review and revision:

[1144] The user checks the agenda sent from the server on the device and modifies it as necessary. The modified agenda is then sent back to the server and saved.

[1145] Check what was said:

[1146] During the conference, users can check the summary sent from the server on their terminals and keep track of the progress.

[1147] Check your emotional tone:

[1148] The user can check the emotional tone information of the participants on the terminal and use it as a reference for conducting the appropriate proceedings.

[1149] Timekeeping:

[1150] The terminal displays the timekeeping alert sent from the server to notify the user of the need for time management.

[1151] Enter your feedback:

[1152] After the conference ends, users input feedback through their terminals, which is then sent to the server.

[1153] User

[1154] Users are primarily responsible for preparing, conducting, and evaluating meetings, including creating and modifying the agenda, checking comments made during the meeting, keeping time, and entering feedback.

[1155] Examples of concrete examples and prompts

[1156] Creating and reviewing the agenda

[1157] The user sends the theme of the "monthly report meeting" and the purpose of "checking the progress of the project" to the server.

[1158] The server uses a generative AI model to generate agenda proposals such as "checking project progress," "introducing new projects," and "sharing issues."

[1159] The user checks the agenda on the device and modifies it if necessary.

[1160] Example prompt sentence:

[1161] "Based on data from past monthly reporting meetings, please create an agenda that matches the following theme and purpose. Theme: Monthly reporting meeting, Purpose: Reviewing project progress."

[1162] Meeting management and management

[1163] The server transcribes the audio data in real time and summarizes the text using a natural language processing model.

[1164] The server uses an emotion engine to analyze the emotional tone of participants and generate alerts as needed.

[1165] Example prompt sentence:

[1166] Please transcribe and summarize the following conference audio:

[1167] Post-meeting evaluation and feedback

[1168] The server generates a feedback form and sends it to each participant.

[1169] The user inputs feedback through the terminal and transmits it to the server.

[1170] The server analyzes the feedback and extracts important improvements and suggestions.

[1171] Example prompt sentence:

[1172] "Collect feedback from participants after the meeting, analyze it, and generate a report."

[1173] In this way, the system of the present invention provides comprehensive support for the entire process from preparation to progress and evaluation of the meeting after it has concluded, and by taking into account the emotional state of the participants using an emotion engine, it is possible to significantly improve the quality of the meeting and the satisfaction of the participants.

[1174] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1175] Creating an agenda

[1176] Step 1:

[1177] The user inputs the meeting theme and purpose. The input contents are "Monthly report meeting" and "Project progress confirmation." The device receives this data as input and sends it to the server.

[1178] Step 2:

[1179] Based on the conference theme and purpose received by the server, past conference data is retrieved from the database, and related data is searched for and retrieved from the database.

[1180] Step 3:

[1181] The server uses the AI ​​model to analyze past meeting data and generate appropriate agenda proposals, such as "checking project progress," "introducing new projects," and "sharing issues."

[1182] Step 4:

[1183] The generated agenda plan is sent from the server to the device, where the user can check it.

[1184] Step 5:

[1185] The user modifies the agenda as necessary by inputting "additional items to be shared," and the terminal transmits the modified agenda to the server again.

[1186] Step 6:

[1187] The server saves the modified agenda.

[1188] Summary of what was said during the meeting

[1189] Step 1:

[1190] The device sends the audio data recorded during the meeting to the server in real time. The audio data is transmitted to the server in its original format (audio data format).

[1191] Step 2:

[1192] The server passes the received audio data to a transcription engine, which converts the audio data into text format. The resulting transcribed text data is generated.

[1193] Step 3:

[1194] The server inputs the transcribed text data into a natural language processing model to generate a summary, which is a short text such as "Progress on Project A has been reported."

[1195] Step 4:

[1196] The generated summary is sent from the server to the terminal and displayed to the user.

[1197] Emotion recognition by emotion engine

[1198] Step 1:

[1199] The terminals transmit the audio and video data of each participant recorded during the conference to the server, where the audio and video data is transmitted to the server in its original format.

[1200] Step 2:

[1201] The server passes the received audio and video data to the emotion engine, which analyzes it in real time. The analysis results in emotional tones (stress, relief, excitement, etc.) being output as emotion parameters.

[1202] Step 3:

[1203] The emotional tone data is stored on the server, and an alert is generated as needed. For example, if it is determined that "Participant A is feeling stressed," an alert is generated.

[1204] Step 4:

[1205] The generated alert is sent from the server to the terminal and displayed to the user.

[1206] Progress management and speech rate analysis

[1207] Step 1:

[1208] During a conference, the terminal sends each participant's speech data to the server. The speech data is transmitted to the server in its original format (text data format).

[1209] Step 2:

[1210] The server passes the received speech data to the analysis engine, which calculates the speech rate of each participant. The calculation results in numerical data, such as Participant A's speech rate being 70%.

[1211] Step 3:

[1212] The server generates alerts based on the speech rate data. For example, if a particular participant speaks too much, an alert will be output saying, "Please refrain from speaking, and give other participants a chance to speak."

[1213] Step 4:

[1214] The generated alert is sent from the server to the terminal and displayed to the user.

[1215] Timekeeping

[1216] Step 1:

[1217] The server sets a timer for each segment of the conference, based on the scheduled duration of each segment.

[1218] Step 2:

[1219] As the meeting progresses, the server monitors a timer that you set and generates a timekeeping alert when the specified time has elapsed.

[1220] Step 3:

[1221] The generated timekeeping alert is sent from the server to the device, and includes a notification such as "3 minutes remaining."

[1222] Step 4:

[1223] The user checks the alert on the terminal and adjusts the progress of the meeting.

[1224] Collecting and analyzing feedback

[1225] Step 1:

[1226] After the conference ends, the server automatically generates a feedback form and sends it to each participant via their terminal. The trigger for generating the form is the end of the conference.

[1227] Step 2:

[1228] Each participant fills out a feedback form using their device, and the input is sent to the server. For example, feedback such as "The meeting went smoothly" is provided as input.

[1229] Step 3:

[1230] The server passes the received feedback data to an analysis engine, which extracts important opinions and areas for improvement. The extracted data outputs areas for improvement such as "the discussion was biased."

[1231] Step 4:

[1232] The server generates a feedback report based on the analysis results, which contains information useful for preparing for the next meeting.

[1233] Step 5:

[1234] The generated feedback report is sent from the server to the terminal and provided to the user.

[1235] Through these specific processing steps, the system provides comprehensive support for meetings, from preparation to progress and post-conclusion evaluation, and uses an emotion engine and generative AI model to improve meeting quality and participant satisfaction.

[1236] (Application example 2)

[1237] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1238] While conventional meeting support systems automate the progress of meetings, summarizing comments, and timekeeping, they do not take into account the emotional state of participants in their progress management, making it difficult to run meetings with high levels of satisfaction.In particular, in situations where comments are biased or emotional stress is high, smooth discussions become difficult, hindering efficient problem-solving and decision-making.

[1239] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1240] In this invention, the server includes means for generating an agenda based on the conference theme and purpose, means for transcribing audio data during the conference and summarizing the content of speech, means for analyzing the speaking rate during the conference and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the conference ends, means for analyzing the content of speech and emotional tone in real time and using an emotion engine to reflect the emotional state of participants, and means for analyzing the emotions of participants as the conference progresses and making suggestions to adjust the progress according to the emotional tone, thereby enabling flexible and smooth conference management according to the emotional state of participants.

[1241] "Conference theme" refers to the subject or topic to be discussed at a conference.

[1242] An "agenda" refers to a list of items and schedules to be discussed at a meeting.

[1243] "Audio data" refers to audio information obtained by recording or acquiring in real time what participants say during a conference.

[1244] "Transcription" refers to the process of converting audio data into text form.

[1245] A "summary of what was said" refers to a short text that extracts important points and summaries based on the transcription data.

[1246] "Speaking rate" refers to data analyzed based on the speaking time and number of times each participant spoke during a meeting.

[1247] An "alert" refers to a notification that alerts a user when a specific condition occurs.

[1248] "Timekeeping" refers to monitoring the progress of each segment of a meeting and ensuring that the discussion is completed within the allotted time.

[1249] "Feedback" refers to information such as opinions, impressions, and areas for improvement collected from participants after the meeting has ended.

[1250] An "emotion engine" refers to a technology or system that analyzes the emotional tone of participants' voices and facial expressions to recognize their emotional state.

[1251] "Progress adjustment suggestions" refers to a function that suggests specific actions or changes to promote smoother discussions based on the results of analyzing participants' emotional states during a meeting.

[1252] A "generative AI model" refers to a mechanism that generates new data using artificial intelligence trained on large amounts of data.

[1253] A "prompt" is a pre-prepared text that prompts an AI to perform a specific action or generate something.

[1254] The present invention is a system for supporting the progress of a conference, and is implemented by the following steps: The system is composed of a server, terminals, and interactions between users.

[1255] System Configuration

[1256] 1. Server

[1257] The server plays a central role in realizing the following functions: managing meeting information, analyzing data, generating agendas, transcribing and summarizing speech content, analyzing speech rates, generating alerts, timekeeping, collecting and analyzing feedback, and recognizing emotions using an emotion engine.

[1258] 2. Terminal

[1259] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, view emotional tone, and enter feedback.

[1260] 3. Users

[1261] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[1262] Creating an agenda

[1263] The user inputs the meeting topic and purpose using a device. This information is sent to the server, which then uses a generative AI model based on past meeting data to generate a draft agenda. This draft agenda is then sent to the device for the user to review and modify.

[1264] Summary of remarks

[1265] During the meeting, the server receives the audio data and transcribes it in real time. This data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[1266] Emotion recognition by emotion engine

[1267] During the meeting, the server uses an emotion engine to analyze the voice and facial expressions of each participant in real time and recognize their emotional tone. This emotional tone is sent to the server as an emotion parameter and is used to generate alerts and manage the progress of the meeting.

[1268] Progress management and speech rate analysis

[1269] The server analyzes the speaking rate of each participant during the meeting. Based on the results of this analysis, if a particular participant is speaking excessively or the discussion is stagnating, the server generates an alert to balance the conversation. In addition, alerts to adjust the progress are also created based on the output of the emotion engine. These alerts are sent to the terminal and displayed to the user.

[1270] Timekeeping

[1271] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[1272] Collecting and analyzing feedback

[1273] After the meeting ends, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report that extracts areas for improvement and important opinions, which can be used to prepare for the next meeting. At this time, data from the emotion engine is also used in the feedback analysis, and the emotional reactions of the participants are also taken into consideration.

[1274] Specific examples

[1275] Creating and reviewing the agenda

[1276] For example, when a user prepares for a "monthly report meeting," they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses a generative AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can then review the generated agenda on their device, make any necessary edits, and save the final version.

[1277] Meeting management and management

[1278] When a meeting begins, the server receives and transcribes audio data in real time. The results are summarized using a natural language processing model, and users can view the summary on their devices. The server also monitors the speaking rate of each participant and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses an emotion engine to analyze participants' emotions and adjust the progress of the meeting, for example, by making suggestions to ease the discussion if stress is building. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[1279] Post-meeting feedback and analysis

[1280] After the meeting ends, the server generates a feedback form and sends it to each participant. Users enter their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report can be used to prepare for the next meeting. It also takes into account emotional data analyzed by the emotion engine and suggests improvements to address emotional reactions and stress points.

[1281] Prompt Sentence Examples

[1282] "The theme of the meeting is 'Considerations on Countermeasures to the Latest Threats.' Please generate an agenda."

[1283] "Perform emotion recognition on the 'Status Report' section."

[1284] As a result, this invention can provide a system that effectively and efficiently supports meetings from preparation to progress and evaluation after completion, and further uses an emotion engine to grasp the emotional states of participants, thereby improving the quality of the meeting.

[1285] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1286] Step 1:

[1287] The user inputs the conference theme and purpose into the terminal and transmits it to the server.

[1288] Input: Theme of "Monthly Report Meeting" and purpose of "Project Progress Review".

[1289] Output: The conference topic and objectives data sent to the server.

[1290] Specific operation: A user accesses the application on their terminal and enters the conference topic and purpose into the input form. The data is then sent to the database on the server.

[1291] Step 2:

[1292] The server generates a draft agenda using a generative AI model based on past meeting data.

[1293] Input: Past conference data and the conference topic and objectives entered in Step 1.

[1294] Output: The generated agenda proposal.

[1295] Specific operation: The AI ​​model in the server analyzes past meeting data stored in a database and generates agenda items related to the meeting theme and purpose. The generated agenda proposal is sent to the device.

[1296] Step 3:

[1297] The user checks the generated agenda on the terminal, modifies it if necessary, and sends the modified agenda draft back to the server.

[1298] Input: Generated agenda proposal and user modifications.

[1299] Output: Final draft agenda.

[1300] Specific operation: The user checks the draft agenda on the device screen and edits it if necessary. The final version of the agenda is sent to the server and saved.

[1301] Step 4:

[1302] During the meeting, the server receives the audio data and transcribes it in real time.

[1303] Input: Audio data during the meeting.

[1304] Output: Transcribed text data.

[1305] How it works: The microphone on the device picks up the voices of the meeting participants, and the voice data is sent to the server, which then transcribes it using a speech recognition system.

[1306] Step 5:

[1307] The transcribed text data is passed to a natural language processing model to generate a summary.

[1308] Input: Transcribed text data.

[1309] Output: Summarized text data.

[1310] How it works: The server inputs the transcription data into a natural language processing model and generates a resulting summary, which is then sent to the device.

[1311] Step 6:

[1312] The server uses an emotion engine to analyze participants' voices and facial expressions in real time and recognize their emotional tone.

[1313] Input: Participant's voice and facial expression data.

[1314] Output: Emotional tone (emotion parameters).

[1315] How it works: The server uses a voice recognition system and an image analysis system to analyze the emotional tone of participants' voices and facial expressions. This information is reflected in real time and recorded as emotion parameters.

[1316] Step 7:

[1317] The server analyzes emotional tone and speech rate to detect excessive speech or stalled discussions and generate alerts.

[1318] Input: Emotional tone data and speech rate data.

[1319] Output: Alert notification.

[1320] Specific operation: The server uses a speech rate analysis algorithm to monitor the volume of speech of each participant and generates an alert if there is an imbalance. It also generates an alert if the emotional tone is negative. The generated alert is sent to the terminal and notifies the user.

[1321] Step 8:

[1322] The server manages the timer for each segment of the meeting and sends timekeeping alerts depending on the time remaining.

[1323] Inputs: Meeting segment start time and current time.

[1324] Output: Timekeeping alert.

[1325] Specific operation: The server calculates the elapsed time from the start time of each segment and manages the timer to ensure that the discussion proceeds within the set time. When the remaining time is running low, an alert is generated and sent to the terminal.

[1326] Step 9:

[1327] After the conference ends, the server automatically generates a feedback form and sends it to the participants. Users can then enter their feedback using their devices and send it to the server.

[1328] Input: Feedback after the meeting.

[1329] Output: Collected feedback data.

[1330] Specific operation: At the end of the conference, the server automatically generates a feedback form and sends it to the participants' devices. The users enter their feedback in the form and submit it. The server then analyzes the received data.

[1331] Step 10:

[1332] The server analyzes the collected feedback data and generates a feedback report that extracts areas for improvement and important opinions.

[1333] Input: Collected feedback data.

[1334] Output: Feedback report.

[1335] How it works: The server inputs the feedback data into a data analysis algorithm to extract areas for improvement and important opinions. Based on the extracted information, a feedback report is generated to help prepare for the next meeting.

[1336] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1337] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1338] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1339] [Third embodiment]

[1340] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1341] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1342] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1343] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1344] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1345] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1346] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1347] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1348] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1349] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1350] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1351] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1352] The present invention provides a system for supporting the progress of a conference, specifically providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction between a server, terminals, and users.

[1353] System Configuration

[1354] 1. Server

[1355] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[1356] 2. Terminal

[1357] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, and enter feedback.

[1358] 3. Users

[1359] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[1360] Program processing

[1361] Creating an agenda

[1362] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[1363] Summary of remarks

[1364] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[1365] Progress management and speech rate analysis

[1366] The server analyzes the speech rate of each participant during the meeting. Based on this analysis, if a specific participant is speaking excessively or the discussion is stagnating, the server generates an alert and sends it to the device. The device then displays this alert to the user, prompting them to take the necessary action.

[1367] Timekeeping

[1368] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[1369] Collecting and analyzing feedback

[1370] After the meeting, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback, extracts areas for improvement and important opinions, and generates a feedback report to help prepare for the next meeting.

[1371] Specific examples

[1372] Creating and reviewing the agenda

[1373] When a user prepares for a monthly report meeting, they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses an AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can review the generated agenda on their device, make any necessary additions or corrections, and then resubmit and save the final version.

[1374] Meeting management and management

[1375] When a meeting begins, the server receives and transcribes audio data in real time. It then uses a natural language processing model to summarize the results, which users can view on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[1376] Post-meeting evaluation and feedback

[1377] After the meeting ends, the server creates a feedback form and sends it to each participant. Users fill out their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report is provided to users to help them prepare for the next meeting.

[1378] In this way, the present invention provides a system that efficiently and effectively supports meetings from preparation to progress and evaluation after completion.

[1379] The processing flow will be explained below.

[1380] Creating an agenda

[1381] Step 1:

[1382] The user uses the terminal to input the conference theme and purpose, and transmits them to the server.

[1383] Step 2:

[1384] Based on the received theme and purpose, the server references past meeting data and uses an AI model to generate a draft agenda.

[1385] Step 3:

[1386] The generated agenda plan is sent from the server to the terminal and displayed to the user.

[1387] Step 4:

[1388] The user checks the draft agenda on the device and makes corrections as necessary.

[1389] Step 5:

[1390] The revised agenda is sent back to the server and saved as the final version.

[1391] Summary of remarks

[1392] Step 1:

[1393] The server receives the audio data in real time during the conference.

[1394] Step 2:

[1395] The server automatically transcribes the received audio data.

[1396] Step 3:

[1397] The server passes the transcribed data to a natural language processing model to generate a summary of what was said.

[1398] Step 4:

[1399] The generated summary is sent from the server to the terminal and displayed to the user.

[1400] Analysis of speech rate and generation of alerts

[1401] Step 1:

[1402] The server analyzes each participant's speaking rate in real time during the meeting.

[1403] Step 2:

[1404] Based on the analysis results, if a particular participant is speaking excessively, the server will generate an alert to balance things out.

[1405] Step 3:

[1406] The generated alert is sent from the server to the terminal and displayed to the user.

[1407] Timekeeping

[1408] Step 1:

[1409] The server sets a timer for each segment of the conference.

[1410] Step 2:

[1411] When the time approaches the specified time remaining, the server generates a timekeeping alert.

[1412] Step 3:

[1413] The generated timekeeping alert is sent from the server to the terminal and displayed to the user.

[1414] Collecting and analyzing feedback

[1415] Step 1:

[1416] After the meeting ends, the server automatically generates a feedback form.

[1417] Step 2:

[1418] The generated feedback form is sent from the server to the participant's terminal.

[1419] Step 3:

[1420] The user fills in a feedback form on the terminal and sends it to the server.

[1421] Step 4:

[1422] The server analyzes the received feedback and extracts areas for improvement and important opinions.

[1423] Step 5:

[1424] Based on the extracted information, the server generates a feedback report and sends it to the terminal to help prepare for the next meeting.

[1425] As described above, the server, terminals, and users work together in each step to realize a system that efficiently prepares for, progresses, and evaluates a conference after it has ended.

[1426] Example 1

[1427] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1428] In today's business environment, effective meeting management and proceedings are crucial. However, the entire process, from preparing for meetings to conducting them and evaluating them after they have finished, requires a significant amount of time and effort due to manual work. In particular, tasks such as creating agendas, summarizing comments, balancing speaking ratios, timekeeping, and collecting and analyzing feedback are complex and difficult to carry out efficiently. A consistent automated system to solve these issues is needed.

[1429] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1430] In this invention, the server includes: means for a user to input the meeting theme and purpose and send the input information to the server; means for the server to refer to past meeting data and generate a draft agenda using a generative AI model and send the generated draft agenda to the terminal; means for the user to check the draft agenda on the terminal and resend the revised agenda to the server for storage; means for the server to receive audio data from the meeting and transcribe it in real time; means for the server to pass the transcribed speech content to a natural language processing model to generate a summary and send the generated summary to the terminal; means for the server to analyze the speech rate of each participant in the meeting in real time, generate an alert based on the analysis result, and send the generated alert to the terminal; means for a user to set a timer for the agenda from the terminal, the server to monitor the timer, generate an alert when the timeout is approaching, and send the alert to the terminal; means for the server to automatically generate a feedback form after the meeting ends and send the feedback to the user; and means for the user to input feedback and send it to the server, the server to analyze the collected feedback, generate a feedback report, and provide the generated report to the user. This allows the entire process from preparing for a meeting to proceeding and evaluation after it has ended to be efficiently automated, reducing the burden on users.

[1431] A "user" is a person who operates a terminal to prepare for a conference, manage its progress, and evaluate it after it has ended.

[1432] "Terminal" means a device that provides an interface for a User to interact with the Server, allowing the User to view and modify the agenda, review comments, track progress, receive timekeeping alerts, and enter feedback.

[1433] The "server" is a central processing unit responsible for meeting information management, data analysis, agenda generation, speech transcription and summarization, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[1434] An "agenda" is a list of topics to be discussed to facilitate the progress of a meeting, and is generated based on the theme and purpose of the meeting.

[1435] A "generative AI model" is an artificial intelligence model that learns from past meeting data and refers to an algorithm for automatically generating a draft agenda based on the meeting theme and purpose entered by the user.

[1436] "Audio data" refers to data that records what is said during a meeting and is later transcribed.

[1437] "Transcription" refers to the process of converting audio data into text data.

[1438] A "natural language processing model" refers to an algorithm for analyzing and summarizing transcribed text data, and it performs text analysis by learning from a large amount of text data.

[1439] The "speaking rate" is an index showing the proportion of time each participant spoke during the conference.

[1440] An "alert" is a notification that is generated based on the progress of the conference and the balance of speech among the participants, and is used to alert the user.

[1441] A "timer" is a device that measures the time allocated to each agenda item in a meeting and manages the progress to be made within the specified time.

[1442] A "feedback form" is an electronic form containing questions to collect opinions and feedback from participants after the meeting.

[1443] A "feedback report" is a report that analyzes the collected feedback and summarizes areas for improvement in the meeting and important opinions from participants.

[1444] The present invention is a system for supporting the progress of a conference, providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction of a server, terminals, and users.

[1445] System Configuration

[1446] server

[1447] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis. Specifically, the server transcribes audio data using Google Cloud Speech-to-Text API or AWS Transcribe, summarizes speech content using natural language processing models (e.g., BERT), generates agendas using generative AI models (e.g., GPT-3), and references past meeting data.

[1448] Terminal

[1449] The terminal provides an interface for users to interact with the server. Through the terminal, users can check and modify the agenda, check what has been said, track progress, receive timekeeping alerts, and enter feedback. The terminal receives data from the server and displays it to the user.

[1450] User

[1451] Users operate the device to prepare for the meeting, manage its progress, and evaluate it after it has ended. Specifically, users input the meeting theme and purpose from the device and send them to the server. During the meeting, users can check summaries and alerts generated in real time through the device and take appropriate action. After the meeting, users can use the device to enter feedback and send it to the server.

[1452] Specific examples

[1453] Creating and reviewing the agenda

[1454] When a user prepares for a monthly report meeting, they send the theme "Monthly Report Meeting" and the purpose "Check Project Progress" from their device to the server. The server uses a generative AI model (e.g., GPT-3) based on past meeting data to generate a draft agenda, creating items such as "Check Project Progress," "Introduce New Projects," and "Share Issues." The user then reviews the generated agenda on their device, makes any necessary corrections, and resubmits it to save the final version.

[1455] Meeting management and management

[1456] When the meeting begins, the server receives and transcribes audio data in real time. For example, it uses the Google Cloud Speech-to-Text API to transcribe the audio data. The results are summarized using a natural language processing model (e.g., BERT), and users can view the summary on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal speaking. Furthermore, a timekeeping function is used to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[1457] Post-meeting evaluation and feedback

[1458] After the meeting ends, the server automatically generates a feedback form and sends it to the user. The user enters feedback such as "The meeting went smoothly" or "The comments were well-balanced" into the feedback form on their device. The server analyzes the collected feedback using a natural language processing model and generates a feedback report that extracts areas for improvement and important opinions. This report is provided to the user to help them prepare for the next meeting.

[1459] Prompt Sentence Examples

[1460] Creating an agenda: "Please create an agenda for the next monthly report meeting with the theme 'Monthly Report' and the purpose 'Checking project progress.'"

[1461] Summary Generation: "Generate a summary of the meeting based on the following transcription: [transcription]"

[1462] Feedback collection: "Automatically generate a form to collect feedback from participants after the meeting, analyze the input data, and create a feedback report."

[1463] By integrating the above functions, the present invention provides a comprehensive system for efficiently and effectively supporting meetings from preparation to progress and evaluation after completion.

[1464] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1465] Program processing flow

[1466] Creating an agenda

[1467] Step 1:

[1468] The user inputs the meeting theme "Project progress review meeting" and the purpose "Project progress review" via the terminal.

[1469] Input: Conference theme and objectives

[1470] Output: Notification of completion of input to the terminal

[1471] Step 2:

[1472] The terminal sends this input information to the server.

[1473] Input: User input information

[1474] Output: Data transfer to the server

[1475] Step 3:

[1476] The server retrieves past meeting data from a database and inputs it into a generative AI model (e.g., GPT-3).

[1477] Input: User input information, past meeting data

[1478] Output: Agenda proposal from the AI ​​model

[1479] Step 4:

[1480] The server sends the generated agenda plan to the terminal so that the user can check it.

[1481] Input: Agenda proposal from AI model

[1482] Output: Send the agenda to the terminal

[1483] Step 5:

[1484] The user checks the draft agenda on the device and modifies it as necessary.

[1485] Input: Agenda proposal from server

[1486] Output: User's modified data

[1487] Step 6:

[1488] The terminal sends the modified agenda back to the server, which stores it.

[1489] Input: User's revision agenda

[1490] Output: Save modified data to the server

[1491] Summary of remarks

[1492] Step 1:

[1493] The server receives the conference audio data in real time.

[1494] Input: Meeting audio data

[1495] Output: Real-time speech-to-text request

[1496] Step 2:

[1497] The server uses the Google Cloud Speech-to-Text API to transcribe the audio data.

[1498] Input: Meeting audio data

[1499] Output: Transcript

[1500] Step 3:

[1501] The server passes the transcribed text to a natural language processing model (e.g., BERT) to generate a summary.

[1502] Input: Transcription text

[1503] Output: Meeting summary text

[1504] Step 4:

[1505] The server sends the generated summary to the terminal.

[1506] Input: Meeting summary text

[1507] Output: Sends text to the terminal

[1508] Step 5:

[1509] The user checks the summary on the terminal and manages the progress of the conference.

[1510] Input: Meeting summary text from the server

[1511] Output: Information for managing the meeting progress

[1512] Progress management and speech rate analysis

[1513] Step 1:

[1514] The server monitors each participant's comments in real time.

[1515] Input: Conference audio data

[1516] Output: Speaker identification data

[1517] Step 2:

[1518] The server tallies the speaking time of each participant and calculates the speaking rate.

[1519] Input: Speaker identification data

[1520] Output: Speech rate data

[1521] Step 3:

[1522] The server generates an alert if a particular participant is speaking too much.

[1523] Input: Speech rate data

[1524] Output: Alert generation request

[1525] Step 4:

[1526] The server sends the generated alert to the terminal and notifies the user.

[1527] Input: Alert information

[1528] Output: Notification to terminal

[1529] Step 5:

[1530] The user checks the alert and adjusts the meeting progress if necessary.

[1531] Input: Alert Notification

[1532] Output: Coordination of meeting progress

[1533] Timekeeping

[1534] Step 1:

[1535] The user sets the timer for each agenda item.

[1536] Input: Set time for each agenda item

[1537] Output: Terminal sends setting values ​​to server

[1538] Step 2:

[1539] The server monitors the timer and measures the progress of each agenda item.

[1540] Input: Set time

[1541] Output: Timer measurement value

[1542] Step 3:

[1543] Generate an alert if the server is running low on time.

[1544] Input: Timer measurement value

[1545] Output: Timekeeping alert

[1546] Step 4:

[1547] The server sends the generated alert to the terminal and notifies the user.

[1548] Input: Timekeeping Alert

[1549] Output: Notification to terminal

[1550] Step 5:

[1551] The user acknowledges the alert and moves on to the next agenda item.

[1552] Input: Timekeeping Alert Notification

[1553] Output: Meeting agenda

[1554] Collecting and analyzing feedback

[1555] Step 1:

[1556] The server automatically generates a feedback form after the meeting ends.

[1557] Input: Meeting end information

[1558] Output: An automatically generated feedback form

[1559] Step 2:

[1560] The server sends the feedback form to the user.

[1561] Input: Auto-generated feedback form

[1562] Output: Feedback form sent to terminal

[1563] Step 3:

[1564] The user inputs feedback on the terminal and transmits it to the server.

[1565] Input: User feedback

[1566] Output: Send feedback to the server

[1567] Step 4:

[1568] The server analyzes the collected feedback.

[1569] Input: User feedback data

[1570] Output: Parsed feedback results

[1571] Step 5:

[1572] The server generates a feedback report and provides it to the user.

[1573] Input: Parsed feedback results

[1574] Output: Feedback report

[1575] Step 6:

[1576] Users can review the feedback report to help them prepare for their next meeting.

[1577] Input: Feedback report from the server

[1578] Output: Capture improvements and feedback

[1579] (Application example 1)

[1580] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1581] In factory maintenance meetings, inefficiencies in progress and information sharing are common, leading to reduced work efficiency and miscommunication. It's particularly difficult to grasp the content of comments made during meetings and the progress of work in real time, and to provide appropriate feedback and time management. Given these issues, a system is needed to ensure effective meeting progress and real-time information sharing.

[1582] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1583] In this invention, the server includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speech, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speech and work status in real time during a factory maintenance meeting and displaying it on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it. This enables more efficient meeting progress and real-time information sharing.

[1584] A "conference theme" is the main subject or topic that will be the focus of discussion at a conference.

[1585] "Purpose" refers to the goal or intention that the meeting is trying to achieve.

[1586] An "agenda" is a list of topics or matters to be discussed at a meeting and indicates the plan of action.

[1587] "Audio data" is a recording of speech spoken during a conference.

[1588] "Transcription" is the process of converting audio data into text data.

[1589] "Speech content" refers to the specific content or information spoken by participants during a meeting.

[1590] A "summary" is a text that condenses the content of a statement or information and extracts its main points.

[1591] The "speaking rate" indicates the rate or frequency at which each participant spoke during the meeting.

[1592] An "alert" is a notification or warning that is issued when a specific condition is met.

[1593] "Timekeeping" means managing the progress of meetings and work.

[1594] "Feedback" refers to the opinions and evaluations collected from participants after the meeting has ended.

[1595] "Analysis" refers to the detailed analysis of collected data to derive patterns and trends.

[1596] A "factory maintenance meeting" is a meeting held within a factory to discuss equipment maintenance and improvements.

[1597] A "smart device" is an electronic device that can connect to the Internet and display and obtain information.

[1598] "Progress management" means managing the arrangements and progress of meetings and work.

[1599] "Real-time" refers to data being processed immediately, without delay.

[1600] "Information sharing" means that multiple people share information and deepen their understanding of each other.

[1601] This invention is a system for supporting the progress of factory maintenance meetings, which is realized by the interaction of a server, a terminal, and a user. The system includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speeches, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speeches and work status during the factory maintenance meeting in real time and displaying them on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it.

[1602] Server Features

[1603] The server generates an agenda based on the meeting theme and purpose by inputting past meeting data into the AI ​​model. By inputting the meeting theme "Monthly maintenance meeting" and the purpose "Checking equipment status and planning the next maintenance" as prompts, the AI ​​model generates agendas such as "Reporting equipment status," "Planning the next maintenance," and "Sharing issues."

[1604] In addition, the system transcribes audio data during meetings in real time and summarizes it using natural language processing (NLP) models. It continuously collects audio during meetings, instantly converts it into text, and sends the summary to a smart device.

[1605] The server analyzes the speaking rate of each participant and generates an alert if there is an imbalance. For example, if a particular participant speaks excessively, an alert will be generated based on that information and a notification will be sent to the smart device.

[1606] Use the timekeeping function to manage the progress of meetings. Based on the time set for each agenda item, when the remaining time is running low, a timekeeping alert will be generated and a notification will be sent to the smart device. In this way, users can review the progress plan in a timely manner.

[1607] After the meeting, the server generates a feedback form and sends it to each participant. The collected feedback data is analyzed and a feedback report is generated to extract improvements for the next maintenance meeting.

[1608] Device and user operations

[1609] Users operate their terminals (smart devices) to check and modify the agenda, check what is being said, understand the progress, receive timekeeping alerts, and enter feedback. For example, when preparing for a monthly maintenance meeting, a user sends the theme of the "monthly maintenance meeting" and the purpose of "checking the equipment status and planning the next maintenance" to the server from their terminal.

[1610] Smart devices have the ability to display real-time transcriptions and summaries to users, specifically, by displaying the transcriptions, summaries, and progress management alerts on the user's smart glasses or head-mounted display.

[1611] Hardware and software used

[1612] The hardware used includes a server, smart glasses, a head-mounted display, and a microphone. The software used includes an AI model, a natural language processing (NLP) model, and a voice recognition library. Specifically, "some_ai_library" and "some_voice_recognition_library" are used.

[1613] Prompt Sentence Examples

[1614] "Topic: Monthly maintenance meeting, Purpose: Checking equipment status and planning the next maintenance"

[1615] In this way, the present invention provides a system that efficiently and effectively supports the preparation, progress, and evaluation of a factory maintenance meeting.

[1616] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1617] Step 1:

[1618] The server receives the meeting theme and purpose from the user. Specifically, the user uses a terminal (smart device) to input the theme, "Monthly Maintenance Meeting," and the purpose, "Checking the equipment status and planning the next maintenance." The input information is sent to the server. The input is text data of the theme and purpose, and the output is the text data sent to the server.

[1619] Step 2:

[1620] The server inputs the received theme and purpose into the generative AI model as prompt sentences. The server references past meeting data and generates a draft agenda using the AI ​​model. Specific data processing involves generating prompt sentences and searching past data, while data calculation involves generating an agenda using the AI ​​model. The input is text data of the theme and purpose, and the output is the generated agenda.

[1621] Step 3:

[1622] The server sends the generated agenda to the terminal, where the user can confirm and modify its contents. The user confirms the agenda on the terminal (smart device) and modifies it as necessary. The modified agenda is then sent back to the server and saved. The input is the text data of the generated agenda, and the output is the text data of the confirmed and modified agenda.

[1623] Step 4:

[1624] The server collects voice data in real time and transcribes it using a speech recognition library. Specifically, the server converts voice data collected by a microphone during a meeting into text data in real time. The input is voice data, and the output is transcribed text data.

[1625] Step 5:

[1626] The server passes the transcribed text data to a natural language processing (NLP) model to generate a summary. The server uses the NLP model to extract and summarize the key points of the text data. The input is the transcribed text data, and the output is the summarized text data.

[1627] Step 6:

[1628] The server sends the generated summary to the terminal and displays it to the user. The user can check the summary in real time during the meeting through their smart device. The input is the summarized text data, and the output is the summary displayed on the smart device.

[1629] Step 7:

[1630] The server analyzes the speech rate of each participant and generates alerts to balance the rate. Specifically, it analyzes audio data, measures the speaking time of each participant, and generates an alert if a specific participant is speaking excessively. The input is audio data, and the output is alert data.

[1631] Step 8:

[1632] The server measures the remaining time based on the timekeeping setting and generates a timekeeping alert. The server monitors the progress of the conference and generates an alert when the specified time is reached and sends it to the terminal. The input is the set time and the current time, and the output is the timekeeping alert.

[1633] Step 9:

[1634] After the conference ends, the server generates a feedback form and sends it to each participant. Users fill out feedback using their terminals and send it to the server. The input is the feedback form, and the output is the user's feedback data.

[1635] Step 10:

[1636] The server analyzes the collected feedback data and generates a feedback report. The server analyzes the feedback data and generates a report that extracts areas for improvement and important opinions, which is useful for preparing for the next meeting. The input is feedback data and the output is a feedback report.

[1637] The above processing steps make it possible to realize a system that efficiently and effectively supports the entire process from preparation to progress and evaluation after a meeting.

[1638] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1639] This invention is a system for supporting the progress of a conference. Specifically, it provides multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended, and further incorporates an emotion engine that recognizes the user's emotions to improve the quality of the conference and the satisfaction of the participants. The system is realized through the interaction between a server, terminals, and users.

[1640] System Configuration

[1641] 1. Server

[1642] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, feedback collection and analysis, and emotion recognition using an emotion engine.

[1643] 2. Terminal

[1644] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, view emotional tone, and enter feedback.

[1645] 3. Users

[1646] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[1647] Program processing

[1648] Creating an agenda

[1649] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[1650] Summary of remarks

[1651] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[1652] Emotion recognition by emotion engine

[1653] During the meeting, the server uses an emotion engine to analyze the voice and facial expressions of each participant in real time and recognize their emotional tone. This emotional tone is sent to the server as an emotion parameter and is used to generate alerts and manage the progress of the meeting.

[1654] Progress management and speech rate analysis

[1655] The server analyzes the speaking rate of each participant during the meeting. Based on the results of this analysis, if a particular participant is speaking excessively or the discussion is stagnating, the server generates an alert to balance the situation. In addition, the server appropriately adjusts the progress based on the output of the emotion engine. The alerts generated by the server are sent to the terminal and displayed to the user.

[1656] Timekeeping

[1657] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[1658] Collecting and analyzing feedback

[1659] After the meeting ends, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback, extracts areas for improvement and important opinions, and generates a feedback report to help prepare for the next meeting. During this process, data from the emotion engine is also used in the feedback analysis, taking into account the emotional reactions of the participants.

[1660] Specific examples

[1661] Creating and reviewing the agenda

[1662] When a user prepares for a monthly report meeting, they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses an AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can review the generated agenda on their device, make any necessary additions or corrections, and then resubmit and save the final version.

[1663] Meeting management and management

[1664] When a meeting begins, the server receives and transcribes audio data in real time. The results are summarized using a natural language processing model, and users can view the summary on their devices. The server also monitors the speaking rate of each participant and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses an emotion engine to analyze participants' emotions and adjust the progress of the meeting, for example, by making suggestions to ease the discussion if stress is building. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[1665] Post-meeting evaluation and feedback

[1666] After the meeting ends, the server creates a feedback form and sends it to each participant. Users fill out their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report is provided to users to help them prepare for the next meeting. At this time, the server also takes into account the emotional data analyzed by the emotion engine and suggests improvements to address emotional reactions and stress points.

[1667] In this way, the present invention provides a system that effectively and efficiently supports meetings from preparation to progress and evaluation after completion, and further uses an emotion engine to grasp the emotional state of participants and improve the quality of the meeting.

[1668] The processing flow will be explained below.

[1669] Creating an agenda

[1670] Step 1:

[1671] The user uses the terminal to input the conference theme and purpose, and transmits them to the server.

[1672] Step 2:

[1673] Based on the received theme and purpose, the server references past meeting data and uses an AI model to generate a draft agenda.

[1674] Step 3:

[1675] The generated agenda plan is sent from the server to the terminal and displayed to the user.

[1676] Step 4:

[1677] The user checks the draft agenda on the device and makes corrections as necessary.

[1678] Step 5:

[1679] The revised agenda is sent back to the server and saved as the final version.

[1680] Summary of remarks

[1681] Step 1:

[1682] The server receives the audio data in real time during the conference.

[1683] Step 2:

[1684] The server automatically transcribes the received audio data.

[1685] Step 3:

[1686] The server passes the transcribed data to a natural language processing model to generate a summary of what was said.

[1687] Step 4:

[1688] The generated summary is sent from the server to the terminal and displayed to the user.

[1689] Emotion recognition by emotion engine

[1690] Step 1:

[1691] The server receives the audio and video data in real time.

[1692] Step 2:

[1693] Based on the received data, the emotion engine analyzes the emotions of each participant.

[1694] Step 3:

[1695] The analyzed emotional tones are sent to the server as emotion parameters.

[1696] Step 4:

[1697] Based on the results of the emotional tone analysis, the server uses this information for progress management and alert generation.

[1698] Analysis of speech rate and generation of alerts

[1699] Step 1:

[1700] The server analyzes the speaking rate of each participant in the meeting in real time.

[1701] Step 2:

[1702] Based on the analysis results, if a particular participant is speaking excessively, the server will generate an alert to balance things out.

[1703] Step 3:

[1704] The generated alert is sent from the server to the terminal and displayed to the user.

[1705] Timekeeping

[1706] Step 1:

[1707] The server sets a timer for each segment of the conference.

[1708] Step 2:

[1709] When the time approaches the specified time remaining, the server generates a timekeeping alert.

[1710] Step 3:

[1711] The generated timekeeping alert is sent from the server to the terminal and displayed to the user.

[1712] Collecting and analyzing feedback

[1713] Step 1:

[1714] After the meeting ends, the server automatically generates a feedback form.

[1715] Step 2:

[1716] The generated feedback form is sent from the server to the participant's terminal.

[1717] Step 3:

[1718] The user fills in a feedback form on the terminal and sends it to the server.

[1719] Step 4:

[1720] The server analyzes the received feedback and extracts areas for improvement and important opinions.

[1721] Step 5:

[1722] Based on the extracted information, the server generates a feedback report and sends it to the terminal to help prepare for the next meeting.

[1723] Specific examples

[1724] Creating and reviewing the agenda

[1725] Step 1:

[1726] The user transmits the theme of the "monthly report meeting" and the purpose of "checking the progress of the project" from the terminal to the server.

[1727] Step 2:

[1728] The server uses an AI model based on past meeting data to generate a draft agenda.

[1729] Step 3:

[1730] For example, create items such as "Check project progress," "Introduce new projects," and "Share issues."

[1731] Step 4:

[1732] The user can review the generated agenda on their device and make any necessary additions or corrections.

[1733] Step 5:

[1734] The revised agenda is sent back to the server and the final version is saved.

[1735] Meeting management and management

[1736] Step 1:

[1737] When the meeting begins, the server receives audio and video data in real time and transcribes it.

[1738] Step 2:

[1739] The results are summarized using a natural language processing model, and users can view the summary on their device.

[1740] Step 3:

[1741] The server monitors the speaking rate of each participant and generates an alert and sends it to the terminal if excessive speaking is detected to encourage equal speaking.

[1742] Step 4:

[1743] The server uses an emotion engine to analyze participants' emotions and uses the results to manage the discussion, for example by making suggestions to ease the discussion if stress levels are rising.

[1744] Step 5:

[1745] Use the timekeeping function to manage the progress of each agenda item, and when time is approaching, an alert will be issued to prompt you to move on to the next agenda item.

[1746] Post-meeting evaluation and feedback

[1747] Step 1:

[1748] After the conference ends, the server automatically generates a feedback form and sends it to the participants.

[1749] Step 2:

[1750] The user fills in a feedback form from the terminal and sends it to the server.

[1751] Step 3:

[1752] The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions.

[1753] Step 4:

[1754] A feedback report is provided to the user to help prepare for the next meeting.

[1755] Step 5:

[1756] In addition, the emotional data analyzed by the emotion engine is taken into account and improvements are suggested to address emotional reactions and stress points.

[1757] Example 2

[1758] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1759] Conventional meeting support systems have had difficulty in providing consistent support for everything from meeting preparation to progress management and post-meeting evaluation. Furthermore, progress management did not take into account the emotional state of participants, which led to problems that could lead to a decline in meeting quality and participant satisfaction. Furthermore, balance of speech and timekeeping were inadequate. There is a need for a system that can solve these issues.

[1760] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1761] In this invention, the server includes a means for generating an agenda based on the conference theme and purpose, a means for transcribing audio data during the conference and summarizing the content of speeches, a means for analyzing the speaking rate during the conference and generating alerts for balancing speaking rates, a means for managing timekeeping and sending notifications according to the remaining time, a means for collecting and analyzing feedback after the conference ends, and a means for analyzing participants' emotions during the conference and recognizing their emotional tone. This provides comprehensive support for preparing, conducting, and evaluating the conference after it ends, and enables progress management that takes participants' emotions into account. Furthermore, speaking balance and timekeeping can also be effectively achieved.

[1762] The "conference theme" is a fundamental element of the content and purpose of the conference, and refers to the main issues to be discussed.

[1763] An "agenda" is a plan that shows the order and timetable of each topic in a meeting.

[1764] "Transcription" is the process of converting audio data into text format.

[1765] A "summary of speech content" is a text that briefly summarizes each statement made during the meeting.

[1766] The "speaking rate" is a numerical representation of the rate at which each participant spoke during the conference.

[1767] An "alert" is a warning or caution message that notifies the user when a specific condition occurs.

[1768] "Timekeeping" is the act of managing the progress of each segment of a meeting and adjusting it so that it stays within the scheduled time.

[1769] "Feedback" refers to the evaluations and opinions of the meeting collected from participants after the meeting has ended.

[1770] "Emotional tone" refers to the emotional state of each participant, analyzed from their voice and facial expressions, and indicates stress, relief, excitement, etc.

[1771] A "generative AI model" is a program that uses artificial intelligence techniques to generate new data or information based on a specific task.

[1772] A "prompt sentence" is an input sentence that instructs a generative AI model on the desired results or task content.

[1773] This invention is a system for effectively supporting the progress of meetings, providing consistent support from preparation to progress and evaluation after the meeting. The system is implemented through the interaction of a server, terminals, and users. Its main functions include generating an agenda, summarizing speech content, analyzing speech rate, recognizing emotional tone, timekeeping, and collecting and analyzing feedback.

[1774] server

[1775] The server is responsible for managing information and analyzing data for the entire conference. Specifically, it performs the following processes:

[1776] Generate the agenda:

[1777] The server generates a draft agenda based on the meeting theme and purpose entered by the user, using a generative AI model and referencing past meeting data. The user can then review and edit the draft agenda on their device, and it is finally saved on the server.

[1778] Example: A user inputs the theme of a "Monthly Report Meeting" and the purpose of "Checking Project Progress" and sends them to the server. The server generates agenda proposals such as "Checking Project Progress," "Introducing New Projects," and "Sharing Issues."

[1779] Summary of what was said:

[1780] The server transcribes the audio data received in real time during the conference and generates a summary using a natural language processing model, which is then sent to the terminal and displayed to the user.

[1781] Example: Speech during a meeting is transcribed in real time to generate text such as "Project A is progressing smoothly." This text is summarized to become "Progress on Project A has been reported."

[1782] Emotional Tone Recognition:

[1783] The server uses an emotion engine to analyze the voice and facial expressions of each participant in the meeting to recognize their emotional tone, and uses this data to generate alerts and suggestions for adjusting the progress of the meeting.

[1784] Example: An emotion engine detects that a particular participant is feeling stressed and sends out an alert saying, "We suggest you take a break."

[1785] Speech rate analysis:

[1786] The server analyzes each participant's speech rate in real time and generates alerts to maintain balance.

[1787] Example: If person A speaks too much, an alert will be sent to the device saying, "Please refrain from speaking, and give other participants a chance to speak."

[1788] Timekeeping:

[1789] The server manages the timer for each segment of the conference and sends timekeeping alerts as time passes.

[1790] Example: When there is little time left, an alert such as "3 minutes remaining" is sent to the device.

[1791] Feedback collection and analysis:

[1792] After the meeting, the server automatically generates a feedback form and sends it to each participant. The collected feedback is then analyzed by the server to extract important opinions and areas for improvement.

[1793] For example, if the feedback includes many comments such as "the discussion was biased," use this information to improve the next meeting.

[1794] Terminal

[1795] The terminal provides an interface for users to interact with the server. Users can perform the following operations through the terminal:

[1796] Agenda review and revision:

[1797] The user checks the agenda sent from the server on the device and modifies it as necessary. The modified agenda is then sent back to the server and saved.

[1798] Check what was said:

[1799] During the conference, users can check the summary sent from the server on their terminals and keep track of the progress.

[1800] Check your emotional tone:

[1801] The user can check the emotional tone information of the participants on the terminal and use it as a reference for conducting the appropriate proceedings.

[1802] Timekeeping:

[1803] The terminal displays the timekeeping alert sent from the server to notify the user of the need for time management.

[1804] Enter your feedback:

[1805] After the conference ends, users input feedback through their terminals, which is then sent to the server.

[1806] User

[1807] Users are primarily responsible for preparing, conducting, and evaluating meetings, including creating and modifying the agenda, checking comments made during the meeting, keeping time, and entering feedback.

[1808] Examples of concrete examples and prompts

[1809] Creating and reviewing the agenda

[1810] The user sends the theme of the "monthly report meeting" and the purpose of "checking the progress of the project" to the server.

[1811] The server uses a generative AI model to generate agenda proposals such as "checking project progress," "introducing new projects," and "sharing issues."

[1812] The user checks the agenda on the device and modifies it if necessary.

[1813] Example prompt sentence:

[1814] "Based on data from past monthly reporting meetings, please create an agenda that matches the following theme and purpose. Theme: Monthly reporting meeting, Purpose: Reviewing project progress."

[1815] Meeting management and management

[1816] The server transcribes the audio data in real time and summarizes the text using a natural language processing model.

[1817] The server uses an emotion engine to analyze the emotional tone of participants and generate alerts as needed.

[1818] Example prompt sentence:

[1819] Please transcribe and summarize the following conference audio:

[1820] Post-meeting evaluation and feedback

[1821] The server generates a feedback form and sends it to each participant.

[1822] The user inputs feedback through the terminal and transmits it to the server.

[1823] The server analyzes the feedback and extracts important improvements and suggestions.

[1824] Example prompt sentence:

[1825] "Collect feedback from participants after the meeting, analyze it, and generate a report."

[1826] In this way, the system of the present invention provides comprehensive support for the entire process from preparation to progress and evaluation of the meeting after it has concluded, and by taking into account the emotional state of the participants using an emotion engine, it is possible to significantly improve the quality of the meeting and the satisfaction of the participants.

[1827] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1828] Creating an agenda

[1829] Step 1:

[1830] The user inputs the meeting theme and purpose. The input contents are "Monthly report meeting" and "Project progress confirmation." The device receives this data as input and sends it to the server.

[1831] Step 2:

[1832] Based on the conference theme and purpose received by the server, past conference data is retrieved from the database, and related data is searched for and retrieved from the database.

[1833] Step 3:

[1834] The server uses the AI ​​model to analyze past meeting data and generate appropriate agenda proposals, such as "checking project progress," "introducing new projects," and "sharing issues."

[1835] Step 4:

[1836] The generated agenda plan is sent from the server to the device, where the user can check it.

[1837] Step 5:

[1838] The user modifies the agenda as necessary by inputting "additional items to be shared," and the terminal transmits the modified agenda to the server again.

[1839] Step 6:

[1840] The server saves the modified agenda.

[1841] Summary of what was said during the meeting

[1842] Step 1:

[1843] The device sends the audio data recorded during the meeting to the server in real time. The audio data is transmitted to the server in its original format (audio data format).

[1844] Step 2:

[1845] The server passes the received audio data to a transcription engine, which converts the audio data into text format. The resulting transcribed text data is generated.

[1846] Step 3:

[1847] The server inputs the transcribed text data into a natural language processing model to generate a summary, which is a short text such as "Progress on Project A has been reported."

[1848] Step 4:

[1849] The generated summary is sent from the server to the terminal and displayed to the user.

[1850] Emotion recognition by emotion engine

[1851] Step 1:

[1852] The terminals transmit the audio and video data of each participant recorded during the conference to the server, where the audio and video data is transmitted to the server in its original format.

[1853] Step 2:

[1854] The server passes the received audio and video data to the emotion engine, which analyzes it in real time. The analysis results in emotional tones (stress, relief, excitement, etc.) being output as emotion parameters.

[1855] Step 3:

[1856] The emotional tone data is stored on the server, and an alert is generated as needed. For example, if it is determined that "Participant A is feeling stressed," an alert is generated.

[1857] Step 4:

[1858] The generated alert is sent from the server to the terminal and displayed to the user.

[1859] Progress management and speech rate analysis

[1860] Step 1:

[1861] During a conference, the terminal sends each participant's speech data to the server. The speech data is transmitted to the server in its original format (text data format).

[1862] Step 2:

[1863] The server passes the received speech data to the analysis engine, which calculates the speech rate of each participant. The calculation results in numerical data, such as Participant A's speech rate being 70%.

[1864] Step 3:

[1865] The server generates alerts based on the speech rate data. For example, if a particular participant speaks too much, an alert will be output saying, "Please refrain from speaking, and give other participants a chance to speak."

[1866] Step 4:

[1867] The generated alert is sent from the server to the terminal and displayed to the user.

[1868] Timekeeping

[1869] Step 1:

[1870] The server sets a timer for each segment of the conference, based on the scheduled duration of each segment.

[1871] Step 2:

[1872] As the meeting progresses, the server monitors a timer that you set and generates a timekeeping alert when the specified time has elapsed.

[1873] Step 3:

[1874] The generated timekeeping alert is sent from the server to the device, and includes a notification such as "3 minutes remaining."

[1875] Step 4:

[1876] The user checks the alert on the terminal and adjusts the progress of the meeting.

[1877] Collecting and analyzing feedback

[1878] Step 1:

[1879] After the conference ends, the server automatically generates a feedback form and sends it to each participant via their terminal. The trigger for generating the form is the end of the conference.

[1880] Step 2:

[1881] Each participant fills out a feedback form using their device, and the input is sent to the server. For example, feedback such as "The meeting went smoothly" is provided as input.

[1882] Step 3:

[1883] The server passes the received feedback data to an analysis engine, which extracts important opinions and areas for improvement. The extracted data outputs areas for improvement such as "the discussion was biased."

[1884] Step 4:

[1885] The server generates a feedback report based on the analysis results, which contains information useful for preparing for the next meeting.

[1886] Step 5:

[1887] The generated feedback report is sent from the server to the terminal and provided to the user.

[1888] Through these specific processing steps, the system provides comprehensive support for meetings, from preparation to progress and post-conclusion evaluation, and uses an emotion engine and generative AI model to improve meeting quality and participant satisfaction.

[1889] (Application example 2)

[1890] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1891] While conventional meeting support systems automate the progress of meetings, summarizing comments, and timekeeping, they do not take into account the emotional state of participants in their progress management, making it difficult to run meetings with high levels of satisfaction.In particular, in situations where comments are biased or emotional stress is high, smooth discussions become difficult, hindering efficient problem-solving and decision-making.

[1892] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1893] In this invention, the server includes means for generating an agenda based on the conference theme and purpose, means for transcribing audio data during the conference and summarizing the content of speech, means for analyzing the speaking rate during the conference and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the conference ends, means for analyzing the content of speech and emotional tone in real time and using an emotion engine to reflect the emotional state of participants, and means for analyzing the emotions of participants as the conference progresses and making suggestions to adjust the progress according to the emotional tone, thereby enabling flexible and smooth conference management according to the emotional state of participants.

[1894] "Conference theme" refers to the subject or topic to be discussed at a conference.

[1895] An "agenda" refers to a list of items and schedules to be discussed at a meeting.

[1896] "Audio data" refers to audio information obtained by recording or acquiring in real time what participants say during a conference.

[1897] "Transcription" refers to the process of converting audio data into text form.

[1898] A "summary of what was said" refers to a short text that extracts important points and summaries based on the transcription data.

[1899] "Speaking rate" refers to data analyzed based on the speaking time and number of times each participant spoke during a meeting.

[1900] An "alert" refers to a notification that alerts a user when a specific condition occurs.

[1901] "Timekeeping" refers to monitoring the progress of each segment of a meeting and ensuring that the discussion is completed within the allotted time.

[1902] "Feedback" refers to information such as opinions, impressions, and areas for improvement collected from participants after the meeting has ended.

[1903] An "emotion engine" refers to a technology or system that analyzes the emotional tone of participants' voices and facial expressions to recognize their emotional state.

[1904] "Progress adjustment suggestions" refers to a function that suggests specific actions or changes to promote smoother discussions based on the results of analyzing participants' emotional states during a meeting.

[1905] A "generative AI model" refers to a mechanism that generates new data using artificial intelligence trained on large amounts of data.

[1906] A "prompt" is a pre-prepared text that prompts an AI to perform a specific action or generate something.

[1907] The present invention is a system for supporting the progress of a conference, and is implemented by the following steps: The system is composed of a server, terminals, and interactions between users.

[1908] System Configuration

[1909] 1. Server

[1910] The server plays a central role in realizing the following functions: managing meeting information, analyzing data, generating agendas, transcribing and summarizing speech content, analyzing speech rates, generating alerts, timekeeping, collecting and analyzing feedback, and recognizing emotions using an emotion engine.

[1911] 2. Terminal

[1912] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, view emotional tone, and enter feedback.

[1913] 3. Users

[1914] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[1915] Creating an agenda

[1916] The user inputs the meeting topic and purpose using a device. This information is sent to the server, which then uses a generative AI model based on past meeting data to generate a draft agenda. This draft agenda is then sent to the device for the user to review and modify.

[1917] Summary of remarks

[1918] During the meeting, the server receives the audio data and transcribes it in real time. This data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[1919] Emotion recognition by emotion engine

[1920] During the meeting, the server uses an emotion engine to analyze the voice and facial expressions of each participant in real time and recognize their emotional tone. This emotional tone is sent to the server as an emotion parameter and is used to generate alerts and manage the progress of the meeting.

[1921] Progress management and speech rate analysis

[1922] The server analyzes the speaking rate of each participant during the meeting. Based on the results of this analysis, if a particular participant is speaking excessively or the discussion is stagnating, the server generates an alert to balance the conversation. In addition, alerts to adjust the progress are also created based on the output of the emotion engine. These alerts are sent to the terminal and displayed to the user.

[1923] Timekeeping

[1924] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[1925] Collecting and analyzing feedback

[1926] After the meeting ends, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report that extracts areas for improvement and important opinions, which can be used to prepare for the next meeting. At this time, data from the emotion engine is also used in the feedback analysis, and the emotional reactions of the participants are also taken into consideration.

[1927] Specific examples

[1928] Creating and reviewing the agenda

[1929] For example, when a user prepares for a "monthly report meeting," they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses a generative AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can then review the generated agenda on their device, make any necessary edits, and save the final version.

[1930] Meeting management and management

[1931] When a meeting begins, the server receives and transcribes audio data in real time. The results are summarized using a natural language processing model, and users can view the summary on their devices. The server also monitors the speaking rate of each participant and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses an emotion engine to analyze participants' emotions and adjust the progress of the meeting, for example, by making suggestions to ease the discussion if stress is building. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[1932] Post-meeting feedback and analysis

[1933] After the meeting ends, the server generates a feedback form and sends it to each participant. Users enter their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report can be used to prepare for the next meeting. It also takes into account emotional data analyzed by the emotion engine and suggests improvements to address emotional reactions and stress points.

[1934] Prompt Sentence Examples

[1935] "The theme of the meeting is 'Considerations on Countermeasures to the Latest Threats.' Please generate an agenda."

[1936] "Perform emotion recognition on the 'Status Report' section."

[1937] As a result, this invention can provide a system that effectively and efficiently supports meetings from preparation to progress and evaluation after completion, and further uses an emotion engine to grasp the emotional states of participants, thereby improving the quality of the meeting.

[1938] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1939] Step 1:

[1940] The user inputs the conference theme and purpose into the terminal and transmits it to the server.

[1941] Input: Theme of "Monthly Report Meeting" and purpose of "Project Progress Review".

[1942] Output: The conference topic and objectives data sent to the server.

[1943] Specific operation: A user accesses the application on their terminal and enters the conference topic and purpose into the input form. The data is then sent to the database on the server.

[1944] Step 2:

[1945] The server generates a draft agenda using a generative AI model based on past meeting data.

[1946] Input: Past conference data and the conference topic and objectives entered in Step 1.

[1947] Output: The generated agenda proposal.

[1948] Specific operation: The AI ​​model in the server analyzes past meeting data stored in a database and generates agenda items related to the meeting theme and purpose. The generated agenda proposal is sent to the device.

[1949] Step 3:

[1950] The user checks the generated agenda on the terminal, modifies it if necessary, and sends the modified agenda draft back to the server.

[1951] Input: Generated agenda proposal and user modifications.

[1952] Output: Final draft agenda.

[1953] Specific operation: The user checks the draft agenda on the device screen and edits it if necessary. The final version of the agenda is sent to the server and saved.

[1954] Step 4:

[1955] During the meeting, the server receives the audio data and transcribes it in real time.

[1956] Input: Audio data during the meeting.

[1957] Output: Transcribed text data.

[1958] How it works: The microphone on the device picks up the voices of the meeting participants, and the voice data is sent to the server, which then transcribes it using a speech recognition system.

[1959] Step 5:

[1960] The transcribed text data is passed to a natural language processing model to generate a summary.

[1961] Input: Transcribed text data.

[1962] Output: Summarized text data.

[1963] How it works: The server inputs the transcription data into a natural language processing model and generates a resulting summary, which is then sent to the device.

[1964] Step 6:

[1965] The server uses an emotion engine to analyze participants' voices and facial expressions in real time and recognize their emotional tone.

[1966] Input: Participant's voice and facial expression data.

[1967] Output: Emotional tone (emotion parameters).

[1968] How it works: The server uses a voice recognition system and an image analysis system to analyze the emotional tone of participants' voices and facial expressions. This information is reflected in real time and recorded as emotion parameters.

[1969] Step 7:

[1970] The server analyzes emotional tone and speech rate to detect excessive speech or stalled discussions and generate alerts.

[1971] Input: Emotional tone data and speech rate data.

[1972] Output: Alert notification.

[1973] Specific operation: The server uses a speech rate analysis algorithm to monitor the volume of speech of each participant and generates an alert if there is an imbalance. It also generates an alert if the emotional tone is negative. The generated alert is sent to the terminal and notifies the user.

[1974] Step 8:

[1975] The server manages the timer for each segment of the meeting and sends timekeeping alerts depending on the time remaining.

[1976] Inputs: Meeting segment start time and current time.

[1977] Output: Timekeeping alert.

[1978] Specific operation: The server calculates the elapsed time from the start time of each segment and manages the timer to ensure that the discussion proceeds within the set time. When the remaining time is running low, an alert is generated and sent to the terminal.

[1979] Step 9:

[1980] After the conference ends, the server automatically generates a feedback form and sends it to the participants. Users can then enter their feedback using their devices and send it to the server.

[1981] Input: Feedback after the meeting.

[1982] Output: Collected feedback data.

[1983] Specific operation: At the end of the conference, the server automatically generates a feedback form and sends it to the participants' devices. The users enter their feedback in the form and submit it. The server then analyzes the received data.

[1984] Step 10:

[1985] The server analyzes the collected feedback data and generates a feedback report that extracts areas for improvement and important opinions.

[1986] Input: Collected feedback data.

[1987] Output: Feedback report.

[1988] How it works: The server inputs the feedback data into a data analysis algorithm to extract areas for improvement and important opinions. Based on the extracted information, a feedback report is generated to help prepare for the next meeting.

[1989] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1990] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1991] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1992] [Fourth embodiment]

[1993] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1994] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1995] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1996] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1997] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1998] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1999] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2000] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2001] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2002] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2003] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2004] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2005] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2006] The present invention provides a system for supporting the progress of a conference, specifically providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction between a server, terminals, and users.

[2007] System Configuration

[2008] 1. Server

[2009] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[2010] 2. Terminal

[2011] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, and enter feedback.

[2012] 3. Users

[2013] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[2014] Program processing

[2015] Creating an agenda

[2016] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[2017] Summary of remarks

[2018] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[2019] Progress management and speech rate analysis

[2020] The server analyzes the speech rate of each participant during the meeting. Based on this analysis, if a specific participant is speaking excessively or the discussion is stagnating, the server generates an alert and sends it to the device. The device then displays this alert to the user, prompting them to take the necessary action.

[2021] Timekeeping

[2022] The server manages the timer for each segment of the conference and sends timekeeping alerts to keep the conference running within the specified time. When the remaining time is running low, the server sends an alert to the terminal, which is displayed to the user.

[2023] Collecting and analyzing feedback

[2024] After the meeting, the server automatically generates a feedback form and sends it to the participants. Users enter their feedback using their devices and send it to the server. The server analyzes the collected feedback, extracts areas for improvement and important opinions, and generates a feedback report to help prepare for the next meeting.

[2025] Specific examples

[2026] Creating and reviewing the agenda

[2027] When a user prepares for a monthly report meeting, they send the theme of the "monthly report meeting" and the purpose of "checking project progress" from their device to the server. The server uses an AI model based on past meeting data to generate a draft agenda, creating items such as "checking project progress," "introducing new projects," and "sharing issues." The user can review the generated agenda on their device, make any necessary additions or corrections, and then resubmit and save the final version.

[2028] Meeting management and management

[2029] When a meeting begins, the server receives and transcribes audio data in real time. It then uses a natural language processing model to summarize the results, which users can view on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal participation. It also uses a timekeeping function to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[2030] Post-meeting evaluation and feedback

[2031] After the meeting ends, the server creates a feedback form and sends it to each participant. Users fill out their feedback on their devices and send it to the server. The server analyzes the collected feedback and generates a feedback report based on the data, extracting areas for improvement and important opinions. This report is provided to users to help them prepare for the next meeting.

[2032] In this way, the present invention provides a system that efficiently and effectively supports meetings from preparation to progress and evaluation after completion.

[2033] The processing flow will be explained below.

[2034] Creating an agenda

[2035] Step 1:

[2036] The user uses the terminal to input the conference theme and purpose, and transmits them to the server.

[2037] Step 2:

[2038] Based on the received theme and purpose, the server references past meeting data and uses an AI model to generate a draft agenda.

[2039] Step 3:

[2040] The generated agenda plan is sent from the server to the terminal and displayed to the user.

[2041] Step 4:

[2042] The user checks the draft agenda on the device and makes corrections as necessary.

[2043] Step 5:

[2044] The revised agenda is sent back to the server and saved as the final version.

[2045] Summary of remarks

[2046] Step 1:

[2047] The server receives the audio data in real time during the conference.

[2048] Step 2:

[2049] The server automatically transcribes the received audio data.

[2050] Step 3:

[2051] The server passes the transcribed data to a natural language processing model to generate a summary of what was said.

[2052] Step 4:

[2053] The generated summary is sent from the server to the terminal and displayed to the user.

[2054] Analysis of speech rate and generation of alerts

[2055] Step 1:

[2056] The server analyzes each participant's speaking rate in real time during the meeting.

[2057] Step 2:

[2058] Based on the analysis results, if a particular participant is speaking excessively, the server will generate an alert to balance things out.

[2059] Step 3:

[2060] The generated alert is sent from the server to the terminal and displayed to the user.

[2061] Timekeeping

[2062] Step 1:

[2063] The server sets a timer for each segment of the conference.

[2064] Step 2:

[2065] When the time approaches the specified time remaining, the server generates a timekeeping alert.

[2066] Step 3:

[2067] The generated timekeeping alert is sent from the server to the terminal and displayed to the user.

[2068] Collecting and analyzing feedback

[2069] Step 1:

[2070] After the meeting ends, the server automatically generates a feedback form.

[2071] Step 2:

[2072] The generated feedback form is sent from the server to the participant's terminal.

[2073] Step 3:

[2074] The user fills in a feedback form on the terminal and sends it to the server.

[2075] Step 4:

[2076] The server analyzes the received feedback and extracts areas for improvement and important opinions.

[2077] Step 5:

[2078] Based on the extracted information, the server generates a feedback report and sends it to the terminal to help prepare for the next meeting.

[2079] As described above, the server, terminals, and users work together in each step to realize a system that efficiently prepares for, progresses, and evaluates a conference after it has ended.

[2080] Example 1

[2081] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2082] In today's business environment, effective meeting management and proceedings are crucial. However, the entire process, from preparing for meetings to conducting them and evaluating them after they have finished, requires a significant amount of time and effort due to manual work. In particular, tasks such as creating agendas, summarizing comments, balancing speaking ratios, timekeeping, and collecting and analyzing feedback are complex and difficult to carry out efficiently. A consistent automated system to solve these issues is needed.

[2083] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2084] In this invention, the server includes: means for a user to input the meeting theme and purpose and send the input information to the server; means for the server to refer to past meeting data and generate a draft agenda using a generative AI model and send the generated draft agenda to the terminal; means for the user to check the draft agenda on the terminal and resend the revised agenda to the server for storage; means for the server to receive audio data from the meeting and transcribe it in real time; means for the server to pass the transcribed speech content to a natural language processing model to generate a summary and send the generated summary to the terminal; means for the server to analyze the speech rate of each participant in the meeting in real time, generate an alert based on the analysis result, and send the generated alert to the terminal; means for a user to set a timer for the agenda from the terminal, the server to monitor the timer, generate an alert when the timeout is approaching, and send the alert to the terminal; means for the server to automatically generate a feedback form after the meeting ends and send the feedback to the user; and means for the user to input feedback and send it to the server, the server to analyze the collected feedback, generate a feedback report, and provide the generated report to the user. This allows the entire process from preparing for a meeting to proceeding and evaluation after it has ended to be efficiently automated, reducing the burden on users.

[2085] A "user" is a person who operates a terminal to prepare for a conference, manage its progress, and evaluate it after it has ended.

[2086] "Terminal" means a device that provides an interface for a User to interact with the Server, allowing the User to view and modify the agenda, review comments, track progress, receive timekeeping alerts, and enter feedback.

[2087] The "server" is a central processing unit responsible for meeting information management, data analysis, agenda generation, speech transcription and summarization, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis.

[2088] An "agenda" is a list of topics to be discussed to facilitate the progress of a meeting, and is generated based on the theme and purpose of the meeting.

[2089] A "generative AI model" is an artificial intelligence model that learns from past meeting data and refers to an algorithm for automatically generating a draft agenda based on the meeting theme and purpose entered by the user.

[2090] "Audio data" refers to data that records what is said during a meeting and is later transcribed.

[2091] "Transcription" refers to the process of converting audio data into text data.

[2092] A "natural language processing model" refers to an algorithm for analyzing and summarizing transcribed text data, and it performs text analysis by learning from a large amount of text data.

[2093] The "speaking rate" is an index showing the proportion of time each participant spoke during the conference.

[2094] An "alert" is a notification that is generated based on the progress of the conference and the balance of speech among the participants, and is used to alert the user.

[2095] A "timer" is a device that measures the time allocated to each agenda item in a meeting and manages the progress to be made within the specified time.

[2096] A "feedback form" is an electronic form containing questions to collect opinions and feedback from participants after the meeting.

[2097] A "feedback report" is a report that analyzes the collected feedback and summarizes areas for improvement in the meeting and important opinions from participants.

[2098] The present invention is a system for supporting the progress of a conference, providing multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended. This system is realized through the interaction of a server, terminals, and users.

[2099] System Configuration

[2100] server

[2101] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, and feedback collection and analysis. Specifically, the server transcribes audio data using Google Cloud Speech-to-Text API or AWS Transcribe, summarizes speech content using natural language processing models (e.g., BERT), generates agendas using generative AI models (e.g., GPT-3), and references past meeting data.

[2102] Terminal

[2103] The terminal provides an interface for users to interact with the server. Through the terminal, users can check and modify the agenda, check what has been said, track progress, receive timekeeping alerts, and enter feedback. The terminal receives data from the server and displays it to the user.

[2104] User

[2105] Users operate the device to prepare for the meeting, manage its progress, and evaluate it after it has ended. Specifically, users input the meeting theme and purpose from the device and send them to the server. During the meeting, users can check summaries and alerts generated in real time through the device and take appropriate action. After the meeting, users can use the device to enter feedback and send it to the server.

[2106] Specific examples

[2107] Creating and reviewing the agenda

[2108] When a user prepares for a monthly report meeting, they send the theme "Monthly Report Meeting" and the purpose "Check Project Progress" from their device to the server. The server uses a generative AI model (e.g., GPT-3) based on past meeting data to generate a draft agenda, creating items such as "Check Project Progress," "Introduce New Projects," and "Share Issues." The user then reviews the generated agenda on their device, makes any necessary corrections, and resubmits it to save the final version.

[2109] Meeting management and management

[2110] When the meeting begins, the server receives and transcribes audio data in real time. For example, it uses the Google Cloud Speech-to-Text API to transcribe the audio data. The results are summarized using a natural language processing model (e.g., BERT), and users can view the summary on their devices. The server also monitors each participant's speaking rate and sends an alert to the device if excessive speaking is detected to encourage equal speaking. Furthermore, a timekeeping function is used to manage the progress of each agenda item, issuing an alert when time is running out to prompt users to move on to the next agenda item.

[2111] Post-meeting evaluation and feedback

[2112] After the meeting ends, the server automatically generates a feedback form and sends it to the user. The user enters feedback such as "The meeting went smoothly" or "The comments were well-balanced" into the feedback form on their device. The server analyzes the collected feedback using a natural language processing model and generates a feedback report that extracts areas for improvement and important opinions. This report is provided to the user to help them prepare for the next meeting.

[2113] Prompt Sentence Examples

[2114] Creating an agenda: "Please create an agenda for the next monthly report meeting with the theme 'Monthly Report' and the purpose 'Checking project progress.'"

[2115] Summary Generation: "Generate a summary of the meeting based on the following transcription: [transcription]"

[2116] Feedback collection: "Automatically generate a form to collect feedback from participants after the meeting, analyze the input data, and create a feedback report."

[2117] By integrating the above functions, the present invention provides a comprehensive system for efficiently and effectively supporting meetings from preparation to progress and evaluation after completion.

[2118] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2119] Program processing flow

[2120] Creating an agenda

[2121] Step 1:

[2122] The user inputs the meeting theme "Project progress review meeting" and the purpose "Project progress review" via the terminal.

[2123] Input: Conference theme and objectives

[2124] Output: Notification of completion of input to the terminal

[2125] Step 2:

[2126] The terminal sends this input information to the server.

[2127] Input: User input information

[2128] Output: Data transfer to the server

[2129] Step 3:

[2130] The server retrieves past meeting data from a database and inputs it into a generative AI model (e.g., GPT-3).

[2131] Input: User input information, past meeting data

[2132] Output: Agenda proposal from the AI ​​model

[2133] Step 4:

[2134] The server sends the generated agenda plan to the terminal so that the user can check it.

[2135] Input: Agenda proposal from AI model

[2136] Output: Send the agenda to the terminal

[2137] Step 5:

[2138] The user checks the draft agenda on the device and modifies it as necessary.

[2139] Input: Agenda proposal from server

[2140] Output: User's modified data

[2141] Step 6:

[2142] The terminal sends the modified agenda back to the server, which stores it.

[2143] Input: User's revision agenda

[2144] Output: Save modified data to the server

[2145] Summary of remarks

[2146] Step 1:

[2147] The server receives the conference audio data in real time.

[2148] Input: Meeting audio data

[2149] Output: Real-time speech-to-text request

[2150] Step 2:

[2151] The server uses the Google Cloud Speech-to-Text API to transcribe the audio data.

[2152] Input: Meeting audio data

[2153] Output: Transcript

[2154] Step 3:

[2155] The server passes the transcribed text to a natural language processing model (e.g., BERT) to generate a summary.

[2156] Input: Transcription text

[2157] Output: Meeting summary text

[2158] Step 4:

[2159] The server sends the generated summary to the terminal.

[2160] Input: Meeting summary text

[2161] Output: Sends text to the terminal

[2162] Step 5:

[2163] The user checks the summary on the terminal and manages the progress of the conference.

[2164] Input: Meeting summary text from the server

[2165] Output: Information for managing the meeting progress

[2166] Progress management and speech rate analysis

[2167] Step 1:

[2168] The server monitors each participant's comments in real time.

[2169] Input: Conference audio data

[2170] Output: Speaker identification data

[2171] Step 2:

[2172] The server tallies the speaking time of each participant and calculates the speaking rate.

[2173] Input: Speaker identification data

[2174] Output: Speech rate data

[2175] Step 3:

[2176] The server generates an alert if a particular participant is speaking too much.

[2177] Input: Speech rate data

[2178] Output: Alert generation request

[2179] Step 4:

[2180] The server sends the generated alert to the terminal and notifies the user.

[2181] Input: Alert information

[2182] Output: Notification to terminal

[2183] Step 5:

[2184] The user checks the alert and adjusts the meeting progress if necessary.

[2185] Input: Alert Notification

[2186] Output: Coordination of meeting progress

[2187] Timekeeping

[2188] Step 1:

[2189] The user sets the timer for each agenda item.

[2190] Input: Set time for each agenda item

[2191] Output: Terminal sends setting values ​​to server

[2192] Step 2:

[2193] The server monitors the timer and measures the progress of each agenda item.

[2194] Input: Set time

[2195] Output: Timer measurement value

[2196] Step 3:

[2197] Generate an alert if the server is running low on time.

[2198] Input: Timer measurement value

[2199] Output: Timekeeping alert

[2200] Step 4:

[2201] The server sends the generated alert to the terminal and notifies the user.

[2202] Input: Timekeeping Alert

[2203] Output: Notification to terminal

[2204] Step 5:

[2205] The user acknowledges the alert and moves on to the next agenda item.

[2206] Input: Timekeeping Alert Notification

[2207] Output: Meeting agenda

[2208] Collecting and analyzing feedback

[2209] Step 1:

[2210] The server automatically generates a feedback form after the meeting ends.

[2211] Input: Meeting end information

[2212] Output: An automatically generated feedback form

[2213] Step 2:

[2214] The server sends the feedback form to the user.

[2215] Input: Auto-generated feedback form

[2216] Output: Feedback form sent to terminal

[2217] Step 3:

[2218] The user inputs feedback on the terminal and transmits it to the server.

[2219] Input: User feedback

[2220] Output: Send feedback to the server

[2221] Step 4:

[2222] The server analyzes the collected feedback.

[2223] Input: User feedback data

[2224] Output: Parsed feedback results

[2225] Step 5:

[2226] The server generates a feedback report and provides it to the user.

[2227] Input: Parsed feedback results

[2228] Output: Feedback report

[2229] Step 6:

[2230] Users can review the feedback report to help them prepare for their next meeting.

[2231] Input: Feedback report from the server

[2232] Output: Capture improvements and feedback

[2233] (Application example 1)

[2234] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2235] In factory maintenance meetings, inefficiencies in progress and information sharing are common, leading to reduced work efficiency and miscommunication. It's particularly difficult to grasp the content of comments made during meetings and the progress of work in real time, and to provide appropriate feedback and time management. Given these issues, a system is needed to ensure effective meeting progress and real-time information sharing.

[2236] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2237] In this invention, the server includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speech, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speech and work status in real time during a factory maintenance meeting and displaying it on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it. This enables more efficient meeting progress and real-time information sharing.

[2238] A "conference theme" is the main subject or topic that will be the focus of discussion at a conference.

[2239] "Purpose" refers to the goal or intention that the meeting is trying to achieve.

[2240] An "agenda" is a list of topics or matters to be discussed at a meeting and indicates the plan of action.

[2241] "Audio data" is a recording of speech spoken during a conference.

[2242] "Transcription" is the process of converting audio data into text data.

[2243] "Speech content" refers to the specific content or information spoken by participants during a meeting.

[2244] A "summary" is a text that condenses the content of a statement or information and extracts its main points.

[2245] The "speaking rate" indicates the rate or frequency at which each participant spoke during the meeting.

[2246] An "alert" is a notification or warning that is issued when a specific condition is met.

[2247] "Timekeeping" means managing the progress of meetings and work.

[2248] "Feedback" refers to the opinions and evaluations collected from participants after the meeting has ended.

[2249] "Analysis" refers to the detailed analysis of collected data to derive patterns and trends.

[2250] A "factory maintenance meeting" is a meeting held within a factory to discuss equipment maintenance and improvements.

[2251] A "smart device" is an electronic device that can connect to the Internet and display and obtain information.

[2252] "Progress management" means managing the arrangements and progress of meetings and work.

[2253] "Real-time" refers to data being processed immediately, without delay.

[2254] "Information sharing" means that multiple people share information and deepen their understanding of each other.

[2255] This invention is a system for supporting the progress of factory maintenance meetings, which is realized by the interaction of a server, a terminal, and a user. The system includes means for generating an agenda based on the meeting theme and purpose, means for transcribing audio data during the meeting and summarizing the content of speeches, means for analyzing the speaking rate during the meeting and generating alerts to maintain balance, means for managing timekeeping and sending notifications according to the remaining time, means for collecting and analyzing feedback after the meeting ends, means for transcribing the content of speeches and work status during the factory maintenance meeting in real time and displaying them on a smart device, and means for displaying the agenda generated based on the meeting theme and purpose on a smart device so that the user can confirm and modify it.

[2256] Server Features

[2257] The server generates an agenda based on the meeting theme and purpose by inputting past meeting data into the AI ​​model. By inputting the meeting theme "Monthly maintenance meeting" and the purpose "Checking equipment status and planning the next maintenance" as prompts, the AI ​​model generates agendas such as "Reporting equipment status," "Planning the next maintenance," and "Sharing issues."

[2258] In addition, the system transcribes audio data during meetings in real time and summarizes it using natural language processing (NLP) models. It continuously collects audio during meetings, instantly converts it into text, and sends the summary to a smart device.

[2259] The server analyzes the speaking rate of each participant and generates an alert if there is an imbalance. For example, if a particular participant speaks excessively, an alert will be generated based on that information and a notification will be sent to the smart device.

[2260] Use the timekeeping function to manage the progress of meetings. Based on the time set for each agenda item, when the remaining time is running low, a timekeeping alert will be generated and a notification will be sent to the smart device. In this way, users can review the progress plan in a timely manner.

[2261] After the meeting, the server generates a feedback form and sends it to each participant. The collected feedback data is analyzed and a feedback report is generated to extract improvements for the next maintenance meeting.

[2262] Device and user operations

[2263] Users operate their terminals (smart devices) to check and modify the agenda, check what is being said, understand the progress, receive timekeeping alerts, and enter feedback. For example, when preparing for a monthly maintenance meeting, a user sends the theme of the "monthly maintenance meeting" and the purpose of "checking the equipment status and planning the next maintenance" to the server from their terminal.

[2264] Smart devices have the ability to display real-time transcriptions and summaries to users, specifically, by displaying the transcriptions, summaries, and progress management alerts on the user's smart glasses or head-mounted display.

[2265] Hardware and software used

[2266] The hardware used includes a server, smart glasses, a head-mounted display, and a microphone. The software used includes an AI model, a natural language processing (NLP) model, and a voice recognition library. Specifically, "some_ai_library" and "some_voice_recognition_library" are used.

[2267] Prompt Sentence Examples

[2268] "Topic: Monthly maintenance meeting, Purpose: Checking equipment status and planning the next maintenance"

[2269] In this way, the present invention provides a system that efficiently and effectively supports the preparation, progress, and evaluation of a factory maintenance meeting.

[2270] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2271] Step 1:

[2272] The server receives the meeting theme and purpose from the user. Specifically, the user uses a terminal (smart device) to input the theme, "Monthly Maintenance Meeting," and the purpose, "Checking the equipment status and planning the next maintenance." The input information is sent to the server. The input is text data of the theme and purpose, and the output is the text data sent to the server.

[2273] Step 2:

[2274] The server inputs the received theme and purpose into the generative AI model as prompt sentences. The server references past meeting data and generates a draft agenda using the AI ​​model. Specific data processing involves generating prompt sentences and searching past data, while data calculation involves generating an agenda using the AI ​​model. The input is text data of the theme and purpose, and the output is the generated agenda.

[2275] Step 3:

[2276] The server sends the generated agenda to the terminal, where the user can confirm and modify its contents. The user confirms the agenda on the terminal (smart device) and modifies it as necessary. The modified agenda is then sent back to the server and saved. The input is the text data of the generated agenda, and the output is the text data of the confirmed and modified agenda.

[2277] Step 4:

[2278] The server collects voice data in real time and transcribes it using a speech recognition library. Specifically, the server converts voice data collected by a microphone during a meeting into text data in real time. The input is voice data, and the output is transcribed text data.

[2279] Step 5:

[2280] The server passes the transcribed text data to a natural language processing (NLP) model to generate a summary. The server uses the NLP model to extract and summarize the key points of the text data. The input is the transcribed text data, and the output is the summarized text data.

[2281] Step 6:

[2282] The server sends the generated summary to the terminal and displays it to the user. The user can check the summary in real time during the meeting through their smart device. The input is the summarized text data, and the output is the summary displayed on the smart device.

[2283] Step 7:

[2284] The server analyzes the speech rate of each participant and generates alerts to balance the rate. Specifically, it analyzes audio data, measures the speaking time of each participant, and generates an alert if a specific participant is speaking excessively. The input is audio data, and the output is alert data.

[2285] Step 8:

[2286] The server measures the remaining time based on the timekeeping setting and generates a timekeeping alert. The server monitors the progress of the conference and generates an alert when the specified time is reached and sends it to the terminal. The input is the set time and the current time, and the output is the timekeeping alert.

[2287] Step 9:

[2288] After the conference ends, the server generates a feedback form and sends it to each participant. Users fill out feedback using their terminals and send it to the server. The input is the feedback form, and the output is the user's feedback data.

[2289] Step 10:

[2290] The server analyzes the collected feedback data and generates a feedback report. The server analyzes the feedback data and generates a report that extracts areas for improvement and important opinions, which is useful for preparing for the next meeting. The input is feedback data and the output is a feedback report.

[2291] The above processing steps make it possible to realize a system that efficiently and effectively supports the entire process from preparation to progress and evaluation after a meeting.

[2292] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2293] This invention is a system for supporting the progress of a conference. Specifically, it provides multiple functions that consistently support the preparation, progress, and evaluation of the conference after it has ended, and further incorporates an emotion engine that recognizes the user's emotions to improve the quality of the conference and the satisfaction of the participants. The system is realized through the interaction between a server, terminals, and users.

[2294] System Configuration

[2295] 1. Server

[2296] The server is responsible for meeting information management, data analysis, agenda generation, transcription and summarization of speech content, speech rate analysis, alert generation, timekeeping, feedback collection and analysis, and emotion recognition using an emotion engine.

[2297] 2. Terminal

[2298] The terminal provides an interface for users to interact with the server, allowing them to view and modify the agenda, review what is being said, track progress, receive timekeeping alerts, view emotional tone, and enter feedback.

[2299] 3. Users

[2300] The user operates the terminal to prepare for the meeting, manage the progress, and evaluate the meeting after it has ended.

[2301] Program processing

[2302] Creating an agenda

[2303] The user uses their device to input the meeting topic and purpose. The input information is sent to the server, which then references past meeting data and uses an AI model to generate a draft agenda. This draft agenda is then sent to the device, where the user can review and modify it. The modified agenda is then sent back to the server and saved.

[2304] Summary of remarks

[2305] During the meeting, the server receives the audio data and transcribes it in real time. The transcribed data is passed to a natural language processing model to generate a summary, which is then sent to the device and displayed to the user.

[2306] Emotion recognition by emotion engine

[2307] During the meeting, the server uses an emotion engine to analyze the voice and facial expressions of each participant in real time and recognize their emotional tone. This emotional tone is sent to the server as an emotion parameter and is used to generate alerts and manage the progress of the meeting.

[2308] Progress management and speech rate analysis

[2309] The server analyzes the speaking rate of each participant during the meeting. Based on the results of this analysis, if a particular participant is speaking excessively ...

Claims

1. A system for supporting the progress of a meeting, A means of generating an agenda based on the conference theme and objectives; A means of transcribing audio data during a meeting and summarizing what is being said; A means to analyze speaking rates during meetings and generate alerts to balance them; a means of managing timekeeping and sending notifications depending on the time remaining; A means to collect and analyze feedback after the meeting; A system including:

2. 10. The system of claim 1, wherein the system uses an AI model with reference to past meeting data to generate an agenda based on meeting themes and objectives.

3. The system of claim 1 , wherein speech content transcribed during a meeting is summarized using a natural language processing model.

4. The system according to claim 1 , wherein the system analyzes speech rates in real time and generates an alert to balance the speech rate when a particular participant is speaking excessively.

5. The system of claim 1 , further comprising: a timer for each segment for timekeeping purposes, said timer sending a notification when the time is approaching.

6. The system of claim 1 , further comprising: automatically generating a feedback form after a conference to collect and analyze feedback.

Images