System

Abstract

A system is provided.SOLUTION: A system comprising: means for sending an acknowledgement notification; means for receiving and displaying user attendance responses; means for sharing agenda lists; means for setting a time limit for each agenda; means for recording user remarks in real-time; means for prompting other participants for comments during periods of silence; means for analyzing the remarks with natural language processing; means for matching against a database of previous meetings; means for displaying duplicate remarks notifications; means for generating meeting summaries and action points; means for transmitting meeting summaries and next steps; and means for confirming meeting closure.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] The following problems arise in corporate meetings:

[0005] 1. Meeting participants do not actively speak up, limiting the exchange of opinions.

[0006] 2. Meetings often proceed without reaching a conclusion on matters that need to be decided.

[0007] 3. It reduces productivity by duplicating content that has already been discussed in previous meetings.

[0008] 4. Meeting progress and results are not properly shared, delaying next steps.

[0009] These problems lead to low productivity in meetings and make it difficult for employees to concentrate on their primary duties of planning and execution. A system that can solve these problems is needed. [Means for solving the problem]

[0010] In order to solve the above problems, the present invention provides the following means.

[0011] The system includes means for sending attendance confirmation notifications, means for receiving and displaying users' attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording users' comments in real time, means for encouraging other participants to give their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for generating a summary of the meeting and action points, means for sending a summary of the meeting and next steps, and means for confirming the end of the meeting.

[0012] This system improves meeting efficiency and productivity, and provides an environment where employees can concentrate on their core business. Specifically, it encourages all participants to speak at the appropriate time on the agenda, prevents duplication of previous meeting content, and ensures that everyone has a common understanding. In this way, the present invention provides a system that is highly effective in improving the quality of meetings and time management.

[0013] An "attendance confirmation notice" is a notice sent from the server at the start of a conference to all participants to prompt them to confirm their attendance.

[0014] The "user attendance response" is information that a user who has received the attendance confirmation notice returns by pressing the attendance button.

[0015] The "agenda list" is information that lists the agenda items to be handled at the meeting.

[0016] "Time limit" refers to the time limit set for each agenda item.

[0017] "Recording in real time" means that what a user says is recorded at that moment and saved in a database.

[0018] A "silent period" refers to a period during which no one speaks during a specified time.

[0019] "Natural language processing" is a technology that analyzes text data such as user comments and extracts important information and keywords.

[0020] A "past meeting database" is a database that stores the contents of discussions and decisions made in past meetings.

[0021] The "notification of duplicated comments" is a notification that notifies the user when the same content as in a past conference is commented.

[0022] "Summary of the meeting" is summary information summarizing the contents of the meeting.

[0023] "Action points" are items that indicate the next actions or things to be done that were decided in the meeting.

[0024] A "meeting summary" is a concise summary of what was discussed and what decisions were made during the meeting.

[0025] "Next steps" refers to the next actions or procedures to be taken based on the results of the meeting.

[0026] "Confirming the end of the meeting" refers to the procedure of having participants confirm that all agenda items have been completed. [Brief explanation of the drawings]

[0027] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0028] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0029] First, the terms used in the following description will be explained.

[0030] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0031] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0032] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0033] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0034] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0035] [First embodiment]

[0036] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0037] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0038] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0039] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0040] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0041] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0042] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0043] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0044] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0045] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0046] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0047] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0048] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments of each function of the system will be described below.

[0049] Starting a meeting

[0050] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent via email or the notification function of the conference system. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[0051] Examples:

[0052] Server: "Attendance confirmation has been sent."

[0053] Terminal: "The user pressed the attend button. A response was sent to the server."

[0054] Server: "Everyone is here. The agenda is as follows..."

[0055] Progress of the agenda

[0056] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[0057] Examples:

[0058] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0059] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0060] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0061] Speech analysis

[0062] The server analyzes what users say using natural language processing, extracting important points and keywords. These points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[0063] Examples:

[0064] Server: "Analyzing user C's comment."

[0065] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0066] Server: "This comment was raised in a previous meeting. We will notify the users."

[0067] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0068] Ending the meeting and sharing

[0069] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[0070] Examples:

[0071] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0072] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0073] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0074] In this way, the system of the present invention improves the productivity of meetings by efficiently managing the entire flow from the start to the end of the meeting. The server, terminals, and users each fulfill their respective roles, and the system operates smoothly as a whole, allowing employees to effectively understand the content of the meeting and quickly return to their main tasks.

[0075] The processing flow will be explained below.

[0076] Step 1: Send attendance confirmations

[0077] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[0078] Step 2: Receiving attendance responses

[0079] Terminal: "When the user presses the attend button, the response is sent to the server."

[0080] Step 3: Complete attendance confirmation

[0081] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[0082] Step 4: Set a time limit for the agenda

[0083] Server: "Set a time limit for each topic and start the timer."

[0084] Step 5: Record what you say

[0085] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[0086] Step 6: Prompt for silence

[0087] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[0088] Step 7: Natural Language Analysis of Speech

[0089] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[0090] Step 8: Match with past meetings

[0091] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[0092] Step 9: Notification of duplicate posts

[0093] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[0094] Step 10: Generate a meeting summary

[0095] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[0096] Step 11: Submit your summary and action plan

[0097] Server: "Send an email to all participants with a meeting summary and next steps."

[0098] Step 12: Confirm the end of the meeting

[0099] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[0100] The above are the specific processing steps of the system. By executing each step in order, the meeting will proceed smoothly and effective decision-making will be possible.

[0101] Example 1

[0102] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0103] Corporate meetings often do not proceed effectively, resulting in wasted time, delays in moving forward with agenda items, and overlooking important comments. This can result in lower meeting productivity and delays in important decisions. It is also difficult to keep track of past meeting content and prevent new comments from overlapping. A system that solves these problems and improves meeting productivity is needed.

[0104] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0105] In this invention, the server includes means for sending attendance confirmation notices, means for receiving and displaying user attendance responses, means for sharing an agenda list, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants for their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for generating a meeting summary and action items, means for sending a meeting summary and next steps, means for confirming the end of the meeting, means for sending a meeting attendance confirmation notice via email or a notification function of the conference system, means for converting user comments into text using speech recognition technology, means for saving the recorded comments in real time in a memory area, means for notifying users of the comment analysis results, means for searching past meeting records for similar comments, and means for generating an agenda list in HTML format and displaying it on a terminal.This enables efficient meeting progress, prevents important comments from being overlooked, reduces duplicate comments, and clarifies summaries and action points.

[0106] 1. "Attendance confirmation notice" means a means of communication to confirm the attendance of meeting participants.

[0107] 2. "User attendance response" refers to a conference participant's indication of intent to attend in response to an attendance confirmation notice.

[0108] 3. An "agenda list" is a list of items or topics to be discussed at a meeting.

[0109] 4. "Time limit" means the specific time limit assigned to proceed with each agenda item.

[0110] 5. "Real-time recording" refers to recording what is said immediately on the spot.

[0111] 6. "Silent period" refers to a period of time during which no participant speaks.

[0112] 7. "Natural language processing" is a technology that uses computers to analyze human language and understand and process its meaning and syntax.

[0113] 8. "Past Meeting Database" means data storage that accumulates and manages records of previously held meetings.

[0114] 9. "Duplicate comment notification" is a function that notifies participants when a comment is made that is the same as something that has already been discussed.

[0115] 10. A "summary" is a summary of the overall content of the meeting.

[0116] 11. "Action items" are specific next steps or tasks determined as a result of a meeting.

[0117] 12. A "summary" is a concise summary of what was discussed and decided at a meeting.

[0118] 13. “Next steps” are specific actions or plans to be taken after the meeting.

[0119] 14. "Sending attendance confirmation notices by email or via the notification function of the conference system" refers to sending attendance confirmation notices using the notification function of an email service or online conference tool.

[0120] 15. "Speech recognition technology" is technology that converts speech into text.

[0121] 16. "Memory Area" means a computer storage area for storing data.

[0122] 17. "Analyzing the results of speech analysis" refers to analyzing the content of users' speech and evaluating and using the results.

[0123] 18. "Searching for similar statements" refers to searching for similar content or opinions in past meeting records.

[0124] 19. "Generating an agenda list in HTML format and displaying it on a device" means creating an agenda list in web page format and displaying it on a participant's device.

[0125] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments thereof will be described below.

[0126] Starting a meeting

[0127] The server first sends attendance confirmation notifications to all participants. Attendance confirmation notifications are sent by email or using the conference system's notification function. Users who receive this notification press the attendance button to indicate their intention to attend. The terminal detects the user's attendance response and sends it to the server. After receiving all attendance responses, the server generates an agenda list and displays it on each participant's terminal.

[0128] Examples:

[0129] Server: "Everyone is here. Here's the agenda..."

[0130] Terminal: "The user pressed the attend button. A response was sent to the server."

[0131] Progress of the agenda

[0132] The server sets a time limit for each agenda item and monitors the progress of the agenda items. When a comment is made, the device records the content in real time and sends it to the server. If no comment is made for a certain period of time, the server sends a notification to other participants urging them to contribute. This process ensures the smooth flow of the meeting.

[0133] Examples:

[0134] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0135] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0136] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0137] Speech analysis

[0138] The server uses natural language processing to analyze what users say. This analysis extracts important points and keywords from the comments. These points and keywords are then compared with a database of past meetings, and if similar comments are made, the system notifies the user.

[0139] Examples:

[0140] Server: "Analyzing user C's comment."

[0141] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0142] Server: "This comment was raised in a previous meeting. We will notify the users."

[0143] Ending the meeting and sharing

[0144] After all the agenda items have been completed, the server generates a summary of the meeting and action points. This summary and action points are sent to all participants by email. The terminal displays a confirmation screen for the end of the meeting to the user, and the user can close the meeting by pressing the end button.

[0145] Examples:

[0146] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0147] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0148] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0149] Hardware and software used

[0150] High-performance database servers are used as servers. Specifically, general-purpose database servers or cloud-based database services (such as Amazon RDS) are used. Laptops and tablets (such as ordinary laptops and tablets) are used as devices. Software used includes the GOOGLE WORKSPACE® email API and the MICROSOFT® TEAMS® notification function. The Google® Cloud NLP API is used for natural language processing.

[0151] Prompt Sentence Examples

[0152] "Please describe in detail a system that improves the productivity of corporate meetings. Explain each step from the start to the end of a meeting from the perspective of the server, device, and user, including specific hardware and software."

[0153] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0154] Step 1:

[0155] The server sends attendance confirmations to all participants at the start of the conference.

[0156] Input: List of conference participants

[0157] Processing: The server uses the Google Workspace email API or the conferencing system's notification function to generate and send attendance confirmation notifications to all participants.

[0158] Output: Notification of completion of sending of attendance confirmation notice

[0159] Step 2:

[0160] The user receives a confirmation of attendance.

[0161] Input: Attendance confirmation notification from the server

[0162] Action: The user checks the notification or email they received and clicks the attend button.

[0163] Output: Click on the attendance button (attendance response)

[0164] Step 3:

[0165] The terminal sends the user's response to the server.

[0166] Input: User's attendance response (clicking the attendance button)

[0167] Processing: The terminal sends the user's response to the server via an HTTP POST request.

[0168] Output: Attendance response data sent to the server

[0169] Step 4:

[0170] The server verifies the attendance of all participants.

[0171] Input: Attendance response data from each participant

[0172] Processing: The server receives responses from all participants and checks whether everyone has confirmed their attendance. Reminder emails are sent again if necessary.

[0173] Output: Attendance confirmation completion notification

[0174] Step 5:

[0175] The server instructs each participant to display the agenda list on their terminal.

[0176] Input: Agenda list data

[0177] Processing: The server generates an agenda list and sends it to each terminal using WebSocket. It also configures it to be displayed in HTML format.

[0178] Output: Send and display agenda list

[0179] Step 6:

[0180] The server sets a time limit for each topic.

[0181] Input: Agenda list data

[0182] Processing: The server sets a timer using Python's time library at the start of each agenda item.

[0183] Output: Timer setting completion notification for each agenda item

[0184] Step 7:

[0185] The server monitors the progress of the agenda.

[0186] Input: Ongoing data for each agenda item

[0187] Processing: The server periodically checks the elapsed time of the agenda and evaluates the progress.

[0188] Output: Progress monitoring data

[0189] Step 8:

[0190] When a user speaks, the content is recorded in real time by the terminal.

[0191] Input: User's voice data

[0192] Processing: The device converts the voice data into text using voice recognition technology and sends it to the server in real time.

[0193] Output: Text data to the server

[0194] Step 9:

[0195] If no one speaks for a certain period of time, the server sends a notification to other participants urging them to speak up.

[0196] Input: Data for silent periods

[0197] Processing: When the silent period exceeds the set time, the server sends a notification to other participants using the Slack API.

[0198] Output: Notice of opinion promotion

[0199] Step 10:

[0200] The server analyzes the content of the user's comments.

[0201] Input: Text data sent to the server

[0202] Processing: The server uses Google Cloud NLP API to parse the text data and extract key points and keywords.

[0203] Output: Analysis results (important points and keywords)

[0204] Step 11:

[0205] The server matches the extracted data with past conference data.

[0206] Input: Analysis results (important points and keywords)

[0207] Processing: The server accesses a database of stored past meetings and uses the Cosine Similarity algorithm to search for similar statements.

[0208] Output: Search results for similar statements

[0209] Step 12:

[0210] The server notifies the user if there is a duplicate message.

[0211] Input: Search results for similar statements

[0212] Processing: When a duplicate message is detected, the server generates a notification and sends it to the relevant user. The device receives the notification and displays it to the user.

[0213] Output: Duplicate message notification

[0214] Step 13:

[0215] The server generates a summary and action points for the meeting.

[0216] Input: All data recorded at the meeting

[0217] Processing: The server summarizes all the agenda items and their conclusions of the meeting, automatically sets action points, saves them in Excel format and generates a summary.

[0218] Output: Meeting summary and action points

[0219] Step 14:

[0220] The server will email the meeting summary and action points to all participants.

[0221] Input: Meeting summary and action points

[0222] Processing: The server uses Google Workspace's email API to send the summary and action points to each participant in PDF format.

[0223] Output: Email with summary and action points

[0224] Step 15:

[0225] The terminal displays a confirmation screen for ending the conference to the user, prompting the user to press the end button.

[0226] Input: Conference end notification from the server

[0227] Processing: The terminal receives the notification of the end of the conference and displays the end confirmation screen. The user clicks the end button to confirm.

[0228] Output: User confirms end of meeting

[0229] By dividing the process into detailed steps in this way, the operation of the entire system becomes clear. These steps efficiently manage the entire flow from the start to the end of the conference.

[0230] (Application example 1)

[0231] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0232] The efficiency of conferences and meetings within factories is extremely important, but it is often difficult to advance productive discussions in an environment where multiple participants simultaneously exchange opinions. Furthermore, revisiting topics and opinions already discussed in previous meetings wastes time and reduces meeting productivity. Furthermore, summarizing and sharing action points after meetings is time-consuming, and there is a risk that important information may be overlooked. There is a need for a system that can solve these issues and improve the productivity and efficiency of meetings.

[0233] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0234] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants to give their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing with a database of past meetings, means for displaying notifications of duplicate comments using a generative AI model, means for generating a meeting summary and action points, means for sending a meeting summary and next steps, means for confirming the end of the meeting, and means for generating prompt sentences to improve the efficiency of meetings within the factory. This makes it possible to improve the efficiency and productivity of factory meetings.

[0235] An "attendance confirmation notice" is a notice sent at the start of a conference or meeting to confirm the attendance of participants.

[0236] "User's attendance response" is response information sent by the user in response to the attendance confirmation notice.

[0237] An "agenda list" is a list of topics to be discussed at a conference or meeting.

[0238] A "time limit" is a time limit set for each agenda item.

[0239] "Real-time recording" means that what a user says is instantly saved in digital form.

[0240] A "silent period" is a period of time during a conference where no one speaks for a certain period of time or more.

[0241] A "notice encouraging opinions" is a notification sent to other participants during a silent period to encourage them to give their opinions.

[0242] "Natural language processing" is a technology that analyzes user statements and extracts important points and keywords.

[0243] "Past Meeting Database" means a database that aggregates and stores records of meetings that have been held in the past.

[0244] A "generative AI model" is a model that uses artificial intelligence in the generation process.

[0245] The "duplicate comment notification" is a notification that notifies the user of content similar to a comment that has already been mentioned in a past conference.

[0246] A "meeting summary" is a summary of what was discussed in the meeting and outlines next steps and action points.

[0247] "Action points" are specific next steps to take as a result of the meeting.

[0248] A "prompt sentence" is a specific instruction sentence for performing a specific task.

[0249] The present invention is a system for improving the efficiency of conferences and meetings in a factory. Specific embodiments of the system will be described below.

[0250] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent to the participant's device (smartphone, smart glasses, head-mounted display, etc.), and the participant responds by pressing the attendance button. This attendance information is immediately sent to the server, and the attendance of all participants is confirmed.

[0251] The server then shares the agenda list with all participants' devices, allowing each participant to prepare for the meeting. A time limit is also set for each agenda item, and the server monitors its progress. When a comment is made, the content is recorded in real time by the device and sent to the server. During periods of silence, the server sends notifications to other participants urging them to contribute, preventing the discussion from stagnating.

[0252] The server analyzes user comments using natural language processing technology. It uses libraries such as Spacy to extract key points and keywords. The extracted information is then compared with a database of past meetings to determine whether there are any overlapping comments. If overlapping comments are detected using a generative AI model, a notification is sent to the user.

[0253] When the meeting ends, the server generates a summary of the meeting and action points, and sends them to all participants. This summary includes the conclusions and next steps for each agenda item. A confirmation screen for the end of the meeting is also displayed on the terminal, and participants can press the confirmation button to end the meeting.

[0254] As a concrete example, the following prompts can be fed into a generative AI model to help guide a meeting:

[0255] Analyze the statements and extract key keywords. Here is an example statement:

[0256] "Do you have any comments on the progress of Project X?"

[0257] In this way, the system of the present invention can improve the efficiency and productivity of conferences and meetings within factories. The hardware used includes factory robots, smartphones, smart glasses, and head-mounted displays, and the software includes a server (Node.js server) and a natural language processing library (Spacy).

[0258] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0259] Step 1:

[0260] The server sends an attendance confirmation notification to all participants at the start of the conference. The inputs are the participant list and the start time of the conference. The attendance notification is sent and displayed on each participant's device.

[0261] Step 2:

[0262] The user responds by pressing the attendance button on the terminal. The input is the user's operation (clicking the attendance button) when the attendance confirmation notification is displayed. When the attendance button is pressed, the response information is sent to the server.

[0263] Step 3:

[0264] The server processes the received attendance responses and confirms the attendance of all participants. The input is the attendance response data sent by each participant. The attendance information is tallied, and once all attendance is confirmed, the server proceeds to the next step.

[0265] Step 4:

[0266] The server shares the agenda list with all participants' devices. The input is the meeting agenda list. The agenda list is displayed on each device and shared with the participants.

[0267] Step 5:

[0268] The server sets a time limit for each agenda item and monitors its progress. The inputs are the time limit and start time for each agenda item. The timer starts and the progress is monitored in real time.

[0269] Step 6:

[0270] Users comment on topics. The input is the content of the user's comments. The content of the comments is recorded in real time on the terminal, and the data is sent to the server.

[0271] Step 7:

[0272] The server sends notifications to other participants during periods of silence, encouraging them to contribute their opinions. An input is a period of no speech. When a period of silence is detected, a notification is sent to other participants encouraging them to contribute their opinions.

[0273] Step 8:

[0274] The server analyzes the received comments using natural language processing technology. The input is the user's comment data. Natural language processing (using the Spacy library) is performed to extract important keywords and key points.

[0275] Step 9:

[0276] The server compares the extracted keywords and points with a database of past conferences. The inputs are the extracted keywords and points and the database of past conferences. Database comparison is performed to detect duplicated comments.

[0277] Step 10:

[0278] The server uses the generative AI model to display a notification of duplicated comments to the user. The inputs are the matching results and the duplicate comment detection information. The detected duplicate comments are notified to the user.

[0279] Step 11:

[0280] After the meeting, the server generates a summary of the meeting and action points. The input is all speech data collected during the meeting and a summary of each agenda item. The summary and action points are automatically generated and sent to all participants by email.

[0281] Step 12:

[0282] The user presses the conference end confirmation button on the terminal. As input, there are buttons displayed on the end confirmation screen. When the end confirmation button is pressed, the server records the end of the conference and the conference is officially ended.

[0283] The above steps will improve the efficiency of meetings within the factory and increase productivity.

[0284] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0285] The present invention is a system for improving the productivity of corporate meetings, which is combined with an emotion engine that recognizes the emotions of users. Specific embodiments of each function of the system including the emotion engine will be described below.

[0286] Starting a meeting

[0287] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[0288] Examples:

[0289] Server: "Attendance confirmation has been sent."

[0290] Terminal: "The user pressed the attend button. A response was sent to the server."

[0291] Server: "Everyone is here. The agenda is as follows..."

[0292] Progress of the agenda

[0293] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[0294] Examples:

[0295] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0296] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0297] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0298] Speech analysis and emotion recognition

[0299] The server uses natural language processing to analyze what users say, extracting important points and keywords. At the same time, an emotion engine analyzes and records the emotions contained in the user's comments. The extracted points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[0300] Examples:

[0301] Server: "Analyzing user C's comment."

[0302] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0303] Server: "This comment was raised in a previous meeting. We will notify the users."

[0304] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0305] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[0306] Emotional feedback

[0307] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. In addition, emotion data is accumulated and compared with past emotion data, allowing changes in emotions during a meeting to be tracked.

[0308] Examples:

[0309] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[0310] Server: "Send feedback on dissatisfaction."

[0311] Device: "The user has notified other participants that they are unhappy."

[0312] Ending the meeting and sharing

[0313] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[0314] Examples:

[0315] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0316] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0317] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0318] As described above, the system of the present invention effectively manages a meeting from start to finish through a series of flows, and by adding an emotion engine, it also takes into account emotional aspects during the meeting, thereby realizing smoother and more constructive meetings.

[0319] The processing flow will be explained below.

[0320] Step 1: Send attendance confirmations

[0321] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[0322] Step 2: Receiving attendance responses

[0323] Terminal: "When the user presses the attend button, the response is sent to the server."

[0324] Step 3: Complete attendance confirmation

[0325] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[0326] Step 4: Set a time limit for the agenda

[0327] Server: "Set a time limit for each topic and start the timer."

[0328] Step 5: Record what you say

[0329] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[0330] Step 6: Prompt for silence

[0331] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[0332] Step 7: Natural Language Analysis of Speech

[0333] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[0334] Step 8: Sentiment Analysis

[0335] Emotion engine: "Based on the content of the user's speech, analyzes the emotions at the time of the speech and generates emotional data."

[0336] Step 9: Provide emotional feedback

[0337] Emotion engine: "Provides appropriate feedback to users in real time based on analyzed emotional data."

[0338] Step 10: Match with past meetings

[0339] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[0340] Step 11: Notification of duplicate posts

[0341] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[0342] Step 12: Generate a meeting summary

[0343] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[0344] Step 13: Submit your summary and action plan

[0345] Server: "Send an email to all participants with a meeting summary and next steps."

[0346] Step 14: Confirm the end of the meeting

[0347] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[0348] Examples:

[0349] The flow from the start to the end of a meeting is shown below.

[0350] Step 1:

[0351] Server: "Attendance confirmation has been sent."

[0352] Step 2:

[0353] Terminal: "The user pressed the attend button. A response was sent to the server."

[0354] Step 3:

[0355] Server: "Everyone is here. The agenda is as follows..."

[0356] Step 4:

[0357] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0358] Step 5:

[0359] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0360] Step 6:

[0361] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0362] Step 7:

[0363] Server: "Analyzing User C's comments. Key points identified."

[0364] Step 8:

[0365] Emotion Engine: "Analyzing emotions from user C's comments. The result is 'Go-heung'."

[0366] Step 9:

[0367] Emotion Engine: "Provide positive feedback because the user is in a 'high-exciting' state."

[0368] Step 10:

[0369] Server: "I checked with past meetings. Similar opinions were expressed in previous meetings."

[0370] Step 11:

[0371] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0372] Step 12:

[0373] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0374] Step 13:

[0375] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0376] Step 14:

[0377] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0378] These are the specific processing steps of the system that includes the emotion engine. By executing each step in order, the meeting will proceed effectively and participants will be emotionally satisfied.

[0379] Example 2

[0380] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0381] Corporate meetings often have a tendency for certain participants to speak, resulting in meetings that don't proceed smoothly or don't end on time. Furthermore, participants' feelings and intentions are not properly shared during the meeting, which can lead to misunderstandings and frustration. Furthermore, many topics and comments overlap with previously discussed content, making it difficult to hold efficient discussions. There is a need to solve these problems and improve meeting productivity.

[0382] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0383] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording and transmitting user comments in real time, means for encouraging other participants to contribute during periods of silence, means for analyzing comments using natural language processing and extracting important points and keywords, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for analyzing and recording emotions contained in comments using an emotion engine, means for providing feedback based on emotion data, means for generating a meeting summary and action points, means for transmitting a meeting summary and next steps, and means for confirming the end of the meeting. This allows for the equal invitation of all participants' opinions, efficient analysis of the content of comments, and progress of the meeting while linking them with past data. Furthermore, by understanding emotion data in real time and taking appropriate measures, smoother and more productive meeting management can be achieved.

[0384] "Means for sending attendance confirmation notices" refers to a device or system for sending attendance confirmation notices to all participants at the start of a conference.

[0385] The "means for receiving and displaying a user's attendance response" refers to a device or system for receiving information in response to a user's attendance notification and displaying the response.

[0386] The "means for sharing an agenda list" is a device or system for sharing the meeting agenda in list form with all participants.

[0387] The "means for setting a time limit for each agenda item" is a device or system for setting a time limit for the discussion of each agenda item and managing the progress thereof.

[0388] The "means for recording and transmitting user comments in real time" refers to a device or system for recording user comments in real time and transmitting them to a server.

[0389] The "means for encouraging other participants to give their opinions during silent periods" refers to a device or system for encouraging other participants to give their opinions when no one has spoken for a certain period of time.

[0390] "Means for analyzing comments using natural language processing and extracting important points and keywords" refers to a device or system that uses natural language processing technology to analyze the content of user comments and extract important points and keywords.

[0391] The "means for checking against a database of past meetings" refers to a device or system for checking the content of current statements against a database of past meetings to identify duplicate or similar statements.

[0392] The "means for displaying a notification of overlapping comments" is a device or system for displaying a notification to the user when overlapping comments are found as a result of comparison with past conferences.

[0393] The "means for analyzing and recording emotions contained in comments using an emotion engine" refers to a device or system for analyzing a user's emotions based on the content of comments and recording the results.

[0394] The "means for providing feedback based on emotional data" is a device or system for providing appropriate feedback or notifications to a user based on emotional data analyzed in real time.

[0395] The "means for summarizing a meeting and generating action points" is a device or system for summarizing the content of the discussion after the meeting and generating the next action steps.

[0396] The "Means for transmitting meeting summary and next steps" is a device or system for transmitting a meeting summary and future action points to all participants.

[0397] The "means for confirming the end of the conference" is a device or system for allowing participants to confirm the end of the conference.

[0398] This invention is a system for improving the productivity of corporate meetings, comprehensively managing meetings from start to finish, and utilizing an emotion engine to grasp the user's emotional state in real time and provide feedback. Specific embodiments are described below.

[0399] Hardware and Software Configuration

[0400] The server is equipped with a database server, a web server, a natural language processing engine, and a sentiment analysis engine. These engines can also use external APIs. For example, the system uses the Google Cloud Natural Language API for natural language processing and the IBM Watson (registered trademark) Tone Analyzer for sentiment analysis. This enables advanced text analysis and sentiment analysis.

[0401] A terminal is a device (such as a PC, tablet, or smartphone) used by a participant, and provides an interface for sending and receiving information through communication with the server and for displaying and inputting information.

[0402] Starting a meeting

[0403] At the start of a meeting, the server sends an attendance confirmation notification to all participants. Specifically, it retrieves participants' email addresses and device information from a database and sends them all emails or push notifications (using SendGrid, for example). When a user receives the notification and presses the "attend" button, the response is sent from the device to the server. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on the device.

[0404] Examples:

[0405] Server: "Attendance confirmation has been sent."

[0406] Terminal: "The user pressed the attend button. A response was sent to the server."

[0407] Server: "Everyone is here. The agenda is as follows..."

[0408] Progress of the agenda

[0409] The server sets a time limit for each topic and monitors its progress. A timer is set at the start and a warning is issued before the time limit is exceeded, ensuring smooth progress in the discussion. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[0410] Examples:

[0411] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0412] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0413] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0414] Speech analysis and emotion recognition

[0415] The server analyzes the user's comments using natural language processing. Specifically, it uses the Google Cloud Natural Language API to analyze the text data and extract important points and keywords. At the same time, the emotion engine uses IBM Watson Tone Analyzer to analyze the sentiment of the comments and record the results. The comments and the extracted points and keywords are compared with a database of past meetings, and if similar comments are found, the user is notified that they are duplicates.

[0416] Examples:

[0417] Server: "Analyzing user C's comment."

[0418] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0419] Server: "This comment was raised in a previous meeting. We will notify the users."

[0420] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0421] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[0422] Emotional feedback

[0423] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. Emotion data is accumulated and compared with past emotion data to track changes in emotions during the meeting and consider how to respond if necessary.

[0424] Examples:

[0425] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[0426] Server: "Send feedback on dissatisfaction."

[0427] Device: "The user has notified other participants that they are unhappy."

[0428] Ending the meeting and sharing

[0429] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this to all participants (using SendGrid, for example), and the terminal displays a meeting end confirmation screen to the user. The user presses the end button to officially end the meeting.

[0430] Examples:

[0431] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0432] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0433] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0434] This system efficiently manages the progress of the agenda and analyzes comments, and also grasps the user's emotional state in real time, making for smoother and more constructive meetings.

[0435] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0436] Step 1: Send attendance confirmations

[0437] Specifically, when the meeting starts, the server retrieves the participants' email addresses and device information from the database. Based on the retrieved information, it uses an email sending API (e.g., SendGrid) to send attendance confirmation notifications to all participants.

[0438] Input: Participant's email address and device information obtained from the database.

[0439] Data processing: Convert email addresses and device information into a format suitable for the API.

[0440] Data output: Attendance confirmation notification sending result (success / failure).

[0441] Specific behavior:

[0442] Server: "Attendance confirmation has been sent."

[0443] Step 2: User's attendance response

[0444] When a user presses the attendance button on the terminal, a response is sent from the terminal to the server, which receives it and updates the attendance information in the database.

[0445] Input: User presses the Attend button.

[0446] Data processing: The server receives and analyzes the user's attendance response.

[0447] Data output: Update attendance information to database.

[0448] Specific behavior:

[0449] - Terminal: "The user pressed the attend button. A response was sent to the server."

[0450] - Server: "Attendance information has been updated to the database."

[0451] Step 3: Share the agenda list

[0452] After the server confirms that everyone is present, it shares the meeting agenda list with the terminals. The server retrieves the agenda list from the database and sends it to each terminal.

[0453] Input: A list of topics retrieved by the server from the database.

[0454] Data processing: The acquired agenda list is organized into a format suitable for the device.

[0455] Data output: Send the agenda list to each device.

[0456] Specific behavior:

[0457] - Server: "Agenda list shared."

[0458] Step 4: Set time limits for each agenda item

[0459] During the meeting, the server sets a time limit for each agenda item, runs a timer, and monitors the progress.

[0460] Input: Notifications at the start of each agenda item.

[0461] Data processing: setting time limits and starting timers.

[0462] Data output: Time management data (such as remaining time notification).

[0463] Specific behavior:

[0464] - Server: "Next on the agenda is 'Progress on Project X'. We'll be running it for 20 minutes."

[0465] Step 5: Record and send user comments

[0466] When a user speaks, the content is recorded in real time by the terminal and transmitted to the server.

[0467] Input: What the user says.

[0468] Data processing: Recorded as text data in real time.

[0469] Data output: Sends the statement to the server.

[0470] Specific behavior:

[0471] - Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0472] Step 6: Encourage feedback during silent periods

[0473] If no comments are made for a certain period of time, the server sends a notification to other participants prompting them to comment.

[0474] Input: Whether or not the speech was recorded and the time.

[0475] Data processing: Analyze periods of silence.

[0476] Data output: Notifications encouraging other participants to give their opinions.

[0477] Specific behavior:

[0478] - Server: "There are no new opinions, so I'll ask User B."

[0479] Step 7: Natural Language Processing Analysis of Speech

[0480] The server analyzes the acquired speech using natural language processing (NLP) technology and extracts important points and keywords.

[0481] Input: What the user says.

[0482] Data processing: Analyze using a natural language processing engine (e.g., Google Cloud Natural Language API).

[0483] Data output: Key points and keywords.

[0484] Specific behavior:

[0485] - Server: "Analyzing what you said. Identifying key points."

[0486] Step 8: Sentiment Analysis

[0487] The server uses an emotion engine to analyze the emotions contained in the user's comments and record the results.

[0488] Input: What the user says.

[0489] Data processing: Analyze using a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[0490] Data output: Sentiment analysis results.

[0491] Specific behavior:

[0492] - Server: "Analyzing emotions from user comments."

[0493] Step 9: Notification of duplicate posts

[0494] The server compares the content of comments with a database of past conferences and notifies the user if there are any duplicate comments.

[0495] Input: Analyzed speech content and historical database.

[0496] Data processing: database matching.

[0497] Data output: Duplicate comment notification.

[0498] Specific behavior:

[0499] - Server: "This comment was raised in a previous meeting. We will notify the user."

[0500] - Device: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0501] Step 10: Provide emotional feedback

[0502] Based on the analysis results, the server provides feedback according to the user's emotional state and also sends notifications to take appropriate action.

[0503] Input: Sentiment analysis results.

[0504] Data processing: Generating feedback content.

[0505] Data output: Sending feedback notifications.

[0506] Specific behavior:

[0507] - Server: "Send feedback on dissatisfaction."

[0508] - Device: "The user has notified other participants that they are unhappy."

[0509] Step 11: Meeting summary and action points generation

[0510] Once all the agenda items have been completed, the server generates a meeting summary and action points and sends this to all participants by email.

[0511] Input: Agenda progress data and discussion content.

[0512] Data processing: summary and generation of action points.

[0513] Data output: sent by email.

[0514] Specific behavior:

[0515] - Server: "All meeting agenda items have been completed. Generate summary and action points."

[0516] - Server: "I've emailed you a summary of the meeting and upcoming actions."

[0517] Step 12: Confirm the end of the meeting

[0518] Finally, the terminal displays a confirmation screen for the user to end the conference and prompts the user to press the end button. When the user presses the end button, the conference is officially ended.

[0519] Input: End of meeting notification.

[0520] Data processing: Display of confirmation screen for completion.

[0521] Data output: Response confirming completion.

[0522] Specific behavior:

[0523] - Device: "The meeting has ended. Please press the confirmation button to end it."

[0524] These detailed steps streamline the overall operation and management of meetings and enable real-time analysis of user comments and emotions.

[0525] (Application example 2)

[0526] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0527] In corporate meetings, attendance confirmation, agenda management, and recording and analysis of comments are often done manually, resulting in problems that reduce meeting efficiency and productivity. It is also difficult to grasp participants' emotions during meetings and reflect their feedback in the progress of the meeting. These problems are particularly pronounced in workplaces such as factories, where effective meeting progress is essential.

[0528] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for sending an attendance confirmation notification, means for receiving and displaying a user's attendance response, means for sharing an agenda list, means for setting a time limit for each agenda, means for recording user comments in real time, means for prompting other participants during periods of silence, means for analyzing comments using natural language processing, means for comparing with a database of past meetings, means for displaying a notification of duplicate comments, means for generating a meeting summary and action points, means for sending a meeting summary and next steps, means for confirming the end of the meeting, means for analyzing emotions from user comments, means for accumulating and comparing emotional data, means for providing feedback according to the emotional state, means for displaying the emotional state, and means for operating a robot to support the progress of the meeting. This improves the efficiency of meetings, allows participants' emotional states to be grasped in real time, and appropriate feedback can be provided, enabling more constructive and effective meeting progress.

[0529] An "attendance confirmation notice" is a notice sent to all participants at the start of a conference to confirm the participants' intention to attend the conference.

[0530] The "user attendance response" is information that the user responds to upon receiving the attendance confirmation notice by pressing the attendance button or other methods, and this response is sent to the server.

[0531] An "agenda list" is a list of topics or themes to be discussed at a meeting that is shared to plan the meeting's progress.

[0532] A "time limit" is a time limit set for each agenda item, and is used to keep the meeting proceeding efficiently.

[0533] "User comments" refer to opinions and comments made by participants during a meeting, which are recorded in real time.

[0534] A "silent period" is a period of time in which no comments are made for a certain period of time or more, during which a means for encouraging other participants to give their opinions is activated.

[0535] "Natural language processing" is a technology that analyzes user statements and extracts important points and keywords, thereby helping to understand the content of the statements.

[0536] A "past meeting database" is a database that stores records of meetings that have taken place in the past and is used to match the content of a current meeting with the content of the previous meeting.

[0537] "Duplicate comment notification" is information that is sent to current speakers when similar comments are identified in past meetings, in order to prevent discussions on the same content.

[0538] A "meeting summary" is a summary of the results of the meeting discussion, including next steps and action points.

[0539] "Action points" are specific action plans or tasks that are derived from the results of the meeting, and clearly indicate the action plan after the meeting.

[0540] "Analyzing emotions" refers to a technique for identifying emotions contained in a user's speech and grasping their emotional state, and is implemented using an emotion engine.

[0541] "Providing feedback" means responding or reacting to the user based on the analyzed emotional state, which contributes to improving the progress of the meeting.

[0542] "Displaying emotional state" refers to visually displaying the analyzed emotional state of the user, allowing other participants to understand the situation.

[0543] "Operating a robot" means using a robot to assist in the progress of a meeting, improving the efficiency of the meeting and managing the emotions of participants.

[0544] The present invention is a system for improving productivity in meetings within a factory, which is combined with an emotion engine that recognizes the emotions of a user. The system of the present invention is configured as follows, and a specific embodiment thereof will be described.

[0545] Components

[0546] 1. Attendance confirmation notice

[0547] At the start of the conference, the server sends an attendance confirmation notification to all participants. Participants receive the notification on their devices and press the attendance button to send the information to the server.

[0548] 2. Share the agenda list

[0549] After confirming everyone's attendance, the server generates an agenda list for the meeting and displays it on each participant's device, allowing participants to understand the progress of the meeting.

[0550] 3. Set a time limit

[0551] A time limit is set for each topic, and the server monitors the progress. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to speak up.

[0552] 4. Recording and Analysis of Speech

[0553] When a user speaks, the content is recorded in real time by the device and sent to the server, which then analyzes the content using natural language processing (e.g., Python's Natural Language Toolkit (NLTK) or spaCy).

[0554] 5. Emotion Analysis

[0555] The server uses an emotion engine to analyze emotions from the content of comments, and the analysis results are accumulated in real time and compared with past emotion data.

[0556] 6. Providing Feedback

[0557] Depending on the analyzed emotional state, the server provides feedback to the participants. For example, if a user is feeling unhappy, the server notifies other participants and adjusts the progress of the meeting.

[0558] 7. Comparison with past meetings

[0559] The server compares the content of comments with a database of past meetings, and if there is any overlap, it notifies participants and makes past meeting records available for reference.

[0560] 8. Summary of the meeting and action points

[0561] Once all the agenda items have been completed, the server generates a meeting summary and action points and shares them with all participants.

[0562] 9. Confirmation of the end of the meeting

[0563] When the conference ends, the server confirms the end and, with the consent of all participants, the conference is officially ended.

[0564] Specific examples

[0565] The server performs the following series of operations:

[0566] Attendance confirmation notification: "Are you attending the meeting? Please press the attend button."

[0567] Sharing the agenda list: "Today's agenda list is as follows. Agenda 1: Improving production efficiency, Agenda 2: Safety measures"

[0568] Setting a time limit: "Next on the agenda is 'Improving Productivity'. We'll have 20 minutes."

[0569] Recording and analysis of speech: A user says, "I want to increase the speed of the production line," and the server records and analyzes this.

[0570] Analyzing emotions and providing feedback: "User A's sentiment was analyzed as 'high excitement'. The situation is good."

[0571] Cross-referencing: "This comment was also raised in a previous meeting. Would you like to review the previous meeting?"

[0572] Summary and action points: "The meeting is over. Summary: Specific action points were generated for 'improving production efficiency'. These have been sent via email."

[0573] Meeting End Confirmation: "The meeting has officially ended. Please press the confirmation button to end it."

[0574] Prompt Sentence Examples

[0575] "We're trying to come up with ideas to speed up the production line. How can we improve it while still maintaining safety?"

[0576] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0577] Step 1:

[0578] The server sends an attendance confirmation notification to all participants at the start of the conference. This notification is necessary to confirm attendance status, and participants who receive the notification press the attendance button on their terminal. The input is the instruction to send the attendance confirmation notification, and the output is the result of the notification being sent. Data processing involves generating a notification based on the participant list and sending it to each participant's terminal.

[0579] Step 2:

[0580] The user receives the attendance confirmation notification and presses the attendance button. This generates an attendance response, and the information is sent to the server. The input is the attendance confirmation notification, and the output is the attendance response. Data processing involves generating the attendance response by pressing the button and sending it to the server.

[0581] Step 3:

[0582] After the server confirms everyone's attendance, it generates an agenda list for the meeting and displays it on each participant's device. The input is the aggregated attendance responses, and the output is the displayed agenda list. Data processing involves creating an agenda list after attendance confirmation and distributing it to each participant.

[0583] Step 4:

[0584] The server sets a time limit for each agenda item and monitors the progress of the meeting while it is in progress. If no one speaks for a certain amount of time, the server sends a notification to other participants encouraging them to comment. The input is the agenda list, and the output is the time limit setting information and a notification encouraging comments. Data processing involves setting a time limit for each agenda item and sending notifications based on the elapsed time.

[0585] Step 5:

[0586] When a user speaks, the content of the speech is recorded in real time by the device and the data is sent to the server. The input is the user's speech and the output is the speech recording data. Data processing involves converting the speech into text data using voice recognition and sending that data to the server.

[0587] Step 6:

[0588] The server analyzes the received speech using natural language processing tools (e.g., Python's Natural Language Toolkit (NLTK) or spaCy). The input is the speech recording data, and the output is the analyzed keywords and key points. Data processing involves analyzing the text data and extracting important points.

[0589] Step 7:

[0590] The server then uses an emotion engine to analyze the analyzed speech content and identify the emotions contained in the speech. The input is the analyzed speech data, and the output is the emotion analysis results. Data processing involves using an emotion estimation algorithm (e.g., EmotionAnalyzer) to identify the emotions.

[0591] Step 8:

[0592] The server provides feedback to participants based on the analyzed emotional state. For example, if a user is feeling dissatisfied, the server notifies the other participants of this information and adjusts the progress of the meeting. The input is the emotion analysis result, and the output is a feedback notification. Data processing involves generating appropriate feedback based on the emotion analysis result and sending it to the necessary participants.

[0593] Step 9:

[0594] The server compares the content of comments with a database of past meetings and notifies participants if there is any overlap. The input is the analyzed comment data and the past meeting database, and the output is a notification of duplicate comments. Data processing involves matching with past meeting records.

[0595] Step 10:

[0596] Once all agenda items have been completed, the server generates a meeting summary and action points and shares them with all participants. The input is the meeting progress data, and the output is the summary and action points. Data processing involves summarizing the discussion and next steps and distributing them to participants.

[0597] Step 11:

[0598] When the meeting ends, the server confirms the end and, with the consent of all participants, the meeting officially ends. The input is the end confirmation instruction, and the output is the end confirmation result. Data processing involves executing the end confirmation process and collecting and displaying the results.

[0599] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0600] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0601] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0602] [Second embodiment]

[0603] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0604] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0605] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0606] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0607] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0608] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0609] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0610] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0611] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0612] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0613] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0614] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0615] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments of each function of the system will be described below.

[0616] Starting a meeting

[0617] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent via email or the notification function of the conference system. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[0618] Examples:

[0619] Server: "Attendance confirmation has been sent."

[0620] Terminal: "The user pressed the attend button. A response was sent to the server."

[0621] Server: "Everyone is here. The agenda is as follows..."

[0622] Progress of the agenda

[0623] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[0624] Examples:

[0625] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0626] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0627] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0628] Speech analysis

[0629] The server analyzes what users say using natural language processing, extracting important points and keywords. These points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[0630] Examples:

[0631] Server: "Analyzing user C's comment."

[0632] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0633] Server: "This comment was raised in a previous meeting. We will notify the users."

[0634] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0635] Ending the meeting and sharing

[0636] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[0637] Examples:

[0638] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0639] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0640] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0641] In this way, the system of the present invention improves the productivity of meetings by efficiently managing the entire flow from the start to the end of the meeting. The server, terminals, and users each fulfill their respective roles, and the system operates smoothly as a whole, allowing employees to effectively understand the content of the meeting and quickly return to their main tasks.

[0642] The processing flow will be explained below.

[0643] Step 1: Send attendance confirmations

[0644] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[0645] Step 2: Receiving attendance responses

[0646] Terminal: "When the user presses the attend button, the response is sent to the server."

[0647] Step 3: Complete attendance confirmation

[0648] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[0649] Step 4: Set a time limit for the agenda

[0650] Server: "Set a time limit for each topic and start the timer."

[0651] Step 5: Record what you say

[0652] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[0653] Step 6: Prompt for silence

[0654] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[0655] Step 7: Natural Language Analysis of Speech

[0656] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[0657] Step 8: Match with past meetings

[0658] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[0659] Step 9: Notification of duplicate posts

[0660] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[0661] Step 10: Generate a meeting summary

[0662] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[0663] Step 11: Submit your summary and action plan

[0664] Server: "Send an email to all participants with a meeting summary and next steps."

[0665] Step 12: Confirm the end of the meeting

[0666] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[0667] The above are the specific processing steps of the system. By executing each step in order, the meeting will proceed smoothly and effective decision-making will be possible.

[0668] Example 1

[0669] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0670] Corporate meetings often do not proceed effectively, resulting in wasted time, delays in moving forward with agenda items, and overlooking important comments. This can result in lower meeting productivity and delays in important decisions. It is also difficult to keep track of past meeting content and prevent new comments from overlapping. A system that solves these problems and improves meeting productivity is needed.

[0671] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0672] In this invention, the server includes means for sending attendance confirmation notices, means for receiving and displaying user attendance responses, means for sharing an agenda list, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants for their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for generating a meeting summary and action items, means for sending a meeting summary and next steps, means for confirming the end of the meeting, means for sending a meeting attendance confirmation notice via email or a notification function of the conference system, means for converting user comments into text using speech recognition technology, means for saving the recorded comments in real time in a memory area, means for notifying users of the comment analysis results, means for searching past meeting records for similar comments, and means for generating an agenda list in HTML format and displaying it on a terminal.This enables efficient meeting progress, prevents important comments from being overlooked, reduces duplicate comments, and clarifies summaries and action points.

[0673] 1. "Attendance confirmation notice" means a means of communication to confirm the attendance of meeting participants.

[0674] 2. "User attendance response" refers to a conference participant's indication of intent to attend in response to an attendance confirmation notice.

[0675] 3. An "agenda list" is a list of items or topics to be discussed at a meeting.

[0676] 4. "Time limit" means the specific time limit assigned to proceed with each agenda item.

[0677] 5. "Real-time recording" refers to recording what is said immediately on the spot.

[0678] 6. "Silent period" refers to a period of time during which no participant speaks.

[0679] 7. "Natural language processing" is a technology that uses computers to analyze human language and understand and process its meaning and syntax.

[0680] 8. "Past Meeting Database" means data storage that accumulates and manages records of previously held meetings.

[0681] 9. "Duplicate comment notification" is a function that notifies participants when a comment is made that is the same as something that has already been discussed.

[0682] 10. A "summary" is a summary of the overall content of the meeting.

[0683] 11. "Action items" are specific next steps or tasks determined as a result of a meeting.

[0684] 12. A "summary" is a concise summary of what was discussed and decided at a meeting.

[0685] 13. “Next steps” are specific actions or plans to be taken after the meeting.

[0686] 14. "Sending attendance confirmation notices by email or via the notification function of the conference system" refers to sending attendance confirmation notices using the notification function of an email service or online conference tool.

[0687] 15. "Speech recognition technology" is technology that converts speech into text.

[0688] 16. "Memory Area" means a computer storage area for storing data.

[0689] 17. "Analyzing the results of speech analysis" refers to analyzing the content of users' speech and evaluating and using the results.

[0690] 18. "Searching for similar statements" refers to searching for similar content or opinions in past meeting records.

[0691] 19. "Generating an agenda list in HTML format and displaying it on a device" means creating an agenda list in web page format and displaying it on a participant's device.

[0692] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments thereof will be described below.

[0693] Starting a meeting

[0694] The server first sends attendance confirmation notifications to all participants. Attendance confirmation notifications are sent by email or using the conference system's notification function. Users who receive this notification press the attendance button to indicate their intention to attend. The terminal detects the user's attendance response and sends it to the server. After receiving all attendance responses, the server generates an agenda list and displays it on each participant's terminal.

[0695] Examples:

[0696] Server: "Everyone is here. Here's the agenda..."

[0697] Terminal: "The user pressed the attend button. A response was sent to the server."

[0698] Progress of the agenda

[0699] The server sets a time limit for each agenda item and monitors the progress of the agenda items. When a comment is made, the device records the content in real time and sends it to the server. If no comment is made for a certain period of time, the server sends a notification to other participants urging them to contribute. This process ensures the smooth flow of the meeting.

[0700] Examples:

[0701] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0702] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0703] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0704] Speech analysis

[0705] The server uses natural language processing to analyze what users say. This analysis extracts important points and keywords from the comments. These points and keywords are then compared with a database of past meetings, and if similar comments are made, the system notifies the user.

[0706] Examples:

[0707] Server: "Analyzing user C's comment."

[0708] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0709] Server: "This comment was raised in a previous meeting. We will notify the users."

[0710] Ending the meeting and sharing

[0711] After all the agenda items have been completed, the server generates a summary of the meeting and action points. This summary and action points are sent to all participants by email. The terminal displays a confirmation screen for the end of the meeting to the user, and the user can close the meeting by pressing the end button.

[0712] Examples:

[0713] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0714] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0715] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0716] Hardware and software used

[0717] High-performance database servers are used as servers. Specifically, general-purpose database servers or cloud-based database services (such as Amazon RDS) are used. Laptops and tablets (such as regular laptops and tablets) are used as devices. Software uses the Google Workspace email API and Microsoft Teams notification function. The Google Cloud NLP API is used for natural language processing.

[0718] Prompt Sentence Examples

[0719] "Please describe in detail a system that improves the productivity of corporate meetings. Explain each step from the start to the end of a meeting from the perspective of the server, device, and user, including specific hardware and software."

[0720] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0721] Step 1:

[0722] The server sends attendance confirmations to all participants at the start of the conference.

[0723] Input: List of conference participants

[0724] Processing: The server uses the Google Workspace email API or the conferencing system's notification function to generate and send attendance confirmation notifications to all participants.

[0725] Output: Notification of completion of sending of attendance confirmation notice

[0726] Step 2:

[0727] The user receives a confirmation of attendance.

[0728] Input: Attendance confirmation notification from the server

[0729] Action: The user checks the notification or email they received and clicks the attend button.

[0730] Output: Click on the attendance button (attendance response)

[0731] Step 3:

[0732] The terminal sends the user's response to the server.

[0733] Input: User's attendance response (clicking the attendance button)

[0734] Processing: The terminal sends the user's response to the server via an HTTP POST request.

[0735] Output: Attendance response data sent to the server

[0736] Step 4:

[0737] The server verifies the attendance of all participants.

[0738] Input: Attendance response data from each participant

[0739] Processing: The server receives responses from all participants and checks whether everyone has confirmed their attendance. Reminder emails are sent again if necessary.

[0740] Output: Attendance confirmation completion notification

[0741] Step 5:

[0742] The server instructs each participant to display the agenda list on their terminal.

[0743] Input: Agenda list data

[0744] Processing: The server generates an agenda list and sends it to each terminal using WebSocket. It also configures it to be displayed in HTML format.

[0745] Output: Send and display agenda list

[0746] Step 6:

[0747] The server sets a time limit for each topic.

[0748] Input: Agenda list data

[0749] Processing: The server sets a timer using Python's time library at the start of each agenda item.

[0750] Output: Timer setting completion notification for each agenda item

[0751] Step 7:

[0752] The server monitors the progress of the agenda.

[0753] Input: Ongoing data for each agenda item

[0754] Processing: The server periodically checks the elapsed time of the agenda and evaluates the progress.

[0755] Output: Progress monitoring data

[0756] Step 8:

[0757] When a user speaks, the content is recorded in real time by the terminal.

[0758] Input: User's voice data

[0759] Processing: The device converts the voice data into text using voice recognition technology and sends it to the server in real time.

[0760] Output: Text data to the server

[0761] Step 9:

[0762] If no one speaks for a certain period of time, the server sends a notification to other participants urging them to speak up.

[0763] Input: Data for silent periods

[0764] Processing: When the silent period exceeds the set time, the server sends a notification to other participants using the Slack API.

[0765] Output: Notice of opinion promotion

[0766] Step 10:

[0767] The server analyzes the content of the user's comments.

[0768] Input: Text data sent to the server

[0769] Processing: The server uses Google Cloud NLP API to parse the text data and extract key points and keywords.

[0770] Output: Analysis results (important points and keywords)

[0771] Step 11:

[0772] The server matches the extracted data with past conference data.

[0773] Input: Analysis results (important points and keywords)

[0774] Processing: The server accesses a database of stored past meetings and uses the Cosine Similarity algorithm to search for similar statements.

[0775] Output: Search results for similar statements

[0776] Step 12:

[0777] The server notifies the user if there is a duplicate message.

[0778] Input: Search results for similar statements

[0779] Processing: When a duplicate message is detected, the server generates a notification and sends it to the relevant user. The device receives the notification and displays it to the user.

[0780] Output: Duplicate message notification

[0781] Step 13:

[0782] The server generates a summary and action points for the meeting.

[0783] Input: All data recorded at the meeting

[0784] Processing: The server summarizes all the agenda items and their conclusions of the meeting, automatically sets action points, saves them in Excel format and generates a summary.

[0785] Output: Meeting summary and action points

[0786] Step 14:

[0787] The server will email the meeting summary and action points to all participants.

[0788] Input: Meeting summary and action points

[0789] Processing: The server uses Google Workspace's email API to send the summary and action points to each participant in PDF format.

[0790] Output: Email with summary and action points

[0791] Step 15:

[0792] The terminal displays a confirmation screen for ending the conference to the user, prompting the user to press the end button.

[0793] Input: Conference end notification from the server

[0794] Processing: The terminal receives the notification of the end of the conference and displays the end confirmation screen. The user clicks the end button to confirm.

[0795] Output: User confirms end of meeting

[0796] By dividing the process into detailed steps in this way, the operation of the entire system becomes clear. These steps efficiently manage the entire flow from the start to the end of the conference.

[0797] (Application example 1)

[0798] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0799] The efficiency of conferences and meetings within factories is extremely important, but it is often difficult to advance productive discussions in an environment where multiple participants simultaneously exchange opinions. Furthermore, revisiting topics and opinions already discussed in previous meetings wastes time and reduces meeting productivity. Furthermore, summarizing and sharing action points after meetings is time-consuming, and there is a risk that important information may be overlooked. There is a need for a system that can solve these issues and improve the productivity and efficiency of meetings.

[0800] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0801] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants to give their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing with a database of past meetings, means for displaying notifications of duplicate comments using a generative AI model, means for generating a meeting summary and action points, means for sending a meeting summary and next steps, means for confirming the end of the meeting, and means for generating prompt sentences to improve the efficiency of meetings within the factory. This makes it possible to improve the efficiency and productivity of factory meetings.

[0802] An "attendance confirmation notice" is a notice sent at the start of a conference or meeting to confirm the attendance of participants.

[0803] "User's attendance response" is response information sent by the user in response to the attendance confirmation notice.

[0804] An "agenda list" is a list of topics to be discussed at a conference or meeting.

[0805] A "time limit" is a time limit set for each agenda item.

[0806] "Real-time recording" means that what a user says is instantly saved in digital form.

[0807] A "silent period" is a period of time during a conference where no one speaks for a certain period of time or more.

[0808] A "notice encouraging opinions" is a notification sent to other participants during a silent period to encourage them to give their opinions.

[0809] "Natural language processing" is a technology that analyzes user statements and extracts important points and keywords.

[0810] "Past Meeting Database" means a database that aggregates and stores records of meetings that have been held in the past.

[0811] A "generative AI model" is a model that uses artificial intelligence in the generation process.

[0812] The "duplicate comment notification" is a notification that notifies the user of content similar to a comment that has already been mentioned in a past conference.

[0813] A "meeting summary" is a summary of what was discussed in the meeting and outlines next steps and action points.

[0814] "Action points" are specific next steps to take as a result of the meeting.

[0815] A "prompt sentence" is a specific instruction sentence for performing a specific task.

[0816] The present invention is a system for improving the efficiency of conferences and meetings in a factory. Specific embodiments of the system will be described below.

[0817] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent to the participant's device (smartphone, smart glasses, head-mounted display, etc.), and the participant responds by pressing the attendance button. This attendance information is immediately sent to the server, and the attendance of all participants is confirmed.

[0818] The server then shares the agenda list with all participants' devices, allowing each participant to prepare for the meeting. A time limit is also set for each agenda item, and the server monitors its progress. When a comment is made, the content is recorded in real time by the device and sent to the server. During periods of silence, the server sends notifications to other participants urging them to contribute, preventing the discussion from stagnating.

[0819] The server analyzes user comments using natural language processing technology. It uses libraries such as Spacy to extract key points and keywords. The extracted information is then compared with a database of past meetings to determine whether there are any overlapping comments. If overlapping comments are detected using a generative AI model, a notification is sent to the user.

[0820] When the meeting ends, the server generates a summary of the meeting and action points, and sends them to all participants. This summary includes the conclusions and next steps for each agenda item. A confirmation screen for the end of the meeting is also displayed on the terminal, and participants can press the confirmation button to end the meeting.

[0821] As a concrete example, the following prompts can be fed into a generative AI model to help guide a meeting:

[0822] Analyze the statements and extract key keywords. Here is an example statement:

[0823] "Do you have any comments on the progress of Project X?"

[0824] In this way, the system of the present invention can improve the efficiency and productivity of conferences and meetings within factories. The hardware used includes factory robots, smartphones, smart glasses, and head-mounted displays, and the software includes a server (Node.js server) and a natural language processing library (Spacy).

[0825] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0826] Step 1:

[0827] The server sends an attendance confirmation notification to all participants at the start of the conference. The inputs are the participant list and the start time of the conference. The attendance notification is sent and displayed on each participant's device.

[0828] Step 2:

[0829] The user responds by pressing the attendance button on the terminal. The input is the user's operation (clicking the attendance button) when the attendance confirmation notification is displayed. When the attendance button is pressed, the response information is sent to the server.

[0830] Step 3:

[0831] The server processes the received attendance responses and confirms the attendance of all participants. The input is the attendance response data sent by each participant. The attendance information is tallied, and once all attendance is confirmed, the server proceeds to the next step.

[0832] Step 4:

[0833] The server shares the agenda list with all participants' devices. The input is the meeting agenda list. The agenda list is displayed on each device and shared with the participants.

[0834] Step 5:

[0835] The server sets a time limit for each agenda item and monitors its progress. The inputs are the time limit and start time for each agenda item. The timer starts and the progress is monitored in real time.

[0836] Step 6:

[0837] Users comment on topics. The input is the content of the user's comments. The content of the comments is recorded in real time on the terminal, and the data is sent to the server.

[0838] Step 7:

[0839] The server sends notifications to other participants during periods of silence, encouraging them to contribute their opinions. An input is a period of no speech. When a period of silence is detected, a notification is sent to other participants encouraging them to contribute their opinions.

[0840] Step 8:

[0841] The server analyzes the received comments using natural language processing technology. The input is the user's comment data. Natural language processing (using the Spacy library) is performed to extract important keywords and key points.

[0842] Step 9:

[0843] The server compares the extracted keywords and points with a database of past conferences. The inputs are the extracted keywords and points and the database of past conferences. Database comparison is performed to detect duplicated comments.

[0844] Step 10:

[0845] The server uses the generative AI model to display a notification of duplicated comments to the user. The inputs are the matching results and the duplicate comment detection information. The detected duplicate comments are notified to the user.

[0846] Step 11:

[0847] After the meeting, the server generates a summary of the meeting and action points. The input is all speech data collected during the meeting and a summary of each agenda item. The summary and action points are automatically generated and sent to all participants by email.

[0848] Step 12:

[0849] The user presses the conference end confirmation button on the terminal. As input, there are buttons displayed on the end confirmation screen. When the end confirmation button is pressed, the server records the end of the conference and the conference is officially ended.

[0850] The above steps will improve the efficiency of meetings within the factory and increase productivity.

[0851] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0852] The present invention is a system for improving the productivity of corporate meetings, which is combined with an emotion engine that recognizes the emotions of users. Specific embodiments of each function of the system including the emotion engine will be described below.

[0853] Starting a meeting

[0854] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[0855] Examples:

[0856] Server: "Attendance confirmation has been sent."

[0857] Terminal: "The user pressed the attend button. A response was sent to the server."

[0858] Server: "Everyone is here. The agenda is as follows..."

[0859] Progress of the agenda

[0860] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[0861] Examples:

[0862] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0863] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0864] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0865] Speech analysis and emotion recognition

[0866] The server uses natural language processing to analyze what users say, extracting important points and keywords. At the same time, an emotion engine analyzes and records the emotions contained in the user's comments. The extracted points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[0867] Examples:

[0868] Server: "Analyzing user C's comment."

[0869] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0870] Server: "This comment was raised in a previous meeting. We will notify the users."

[0871] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0872] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[0873] Emotional feedback

[0874] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. In addition, emotion data is accumulated and compared with past emotion data, allowing changes in emotions during a meeting to be tracked.

[0875] Examples:

[0876] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[0877] Server: "Send feedback on dissatisfaction."

[0878] Device: "The user has notified other participants that they are unhappy."

[0879] Ending the meeting and sharing

[0880] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[0881] Examples:

[0882] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0883] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0884] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0885] As described above, the system of the present invention effectively manages a meeting from start to finish through a series of flows, and by adding an emotion engine, it also takes into account emotional aspects during the meeting, thereby realizing smoother and more constructive meetings.

[0886] The processing flow will be explained below.

[0887] Step 1: Send attendance confirmations

[0888] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[0889] Step 2: Receiving attendance responses

[0890] Terminal: "When the user presses the attend button, the response is sent to the server."

[0891] Step 3: Complete attendance confirmation

[0892] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[0893] Step 4: Set a time limit for the agenda

[0894] Server: "Set a time limit for each topic and start the timer."

[0895] Step 5: Record what you say

[0896] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[0897] Step 6: Prompt for silence

[0898] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[0899] Step 7: Natural Language Analysis of Speech

[0900] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[0901] Step 8: Sentiment Analysis

[0902] Emotion engine: "Based on the content of the user's speech, analyzes the emotions at the time of the speech and generates emotional data."

[0903] Step 9: Provide emotional feedback

[0904] Emotion engine: "Provides appropriate feedback to users in real time based on analyzed emotional data."

[0905] Step 10: Match with past meetings

[0906] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[0907] Step 11: Notification of duplicate posts

[0908] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[0909] Step 12: Generate a meeting summary

[0910] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[0911] Step 13: Submit your summary and action plan

[0912] Server: "Send an email to all participants with a meeting summary and next steps."

[0913] Step 14: Confirm the end of the meeting

[0914] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[0915] Examples:

[0916] The flow from the start to the end of a meeting is shown below.

[0917] Step 1:

[0918] Server: "Attendance confirmation has been sent."

[0919] Step 2:

[0920] Terminal: "The user pressed the attend button. A response was sent to the server."

[0921] Step 3:

[0922] Server: "Everyone is here. The agenda is as follows..."

[0923] Step 4:

[0924] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0925] Step 5:

[0926] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0927] Step 6:

[0928] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0929] Step 7:

[0930] Server: "Analyzing User C's comments. Key points identified."

[0931] Step 8:

[0932] Emotion Engine: "Analyzing emotions from user C's comments. The result is 'Go-heung'."

[0933] Step 9:

[0934] Emotion Engine: "Provide positive feedback because the user is in a 'high-exciting' state."

[0935] Step 10:

[0936] Server: "I checked with past meetings. Similar opinions were expressed in previous meetings."

[0937] Step 11:

[0938] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0939] Step 12:

[0940] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0941] Step 13:

[0942] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[0943] Step 14:

[0944] Terminal: "The meeting has ended. Please press the confirmation button to end."

[0945] These are the specific processing steps of the system that includes the emotion engine. By executing each step in order, the meeting will proceed effectively and participants will be emotionally satisfied.

[0946] Example 2

[0947] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0948] Corporate meetings often have a tendency for certain participants to speak, resulting in meetings that don't proceed smoothly or don't end on time. Furthermore, participants' feelings and intentions are not properly shared during the meeting, which can lead to misunderstandings and frustration. Furthermore, many topics and comments overlap with previously discussed content, making it difficult to hold efficient discussions. There is a need to solve these problems and improve meeting productivity.

[0949] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0950] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording and transmitting user comments in real time, means for encouraging other participants to contribute during periods of silence, means for analyzing comments using natural language processing and extracting important points and keywords, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for analyzing and recording emotions contained in comments using an emotion engine, means for providing feedback based on emotion data, means for generating a meeting summary and action points, means for transmitting a meeting summary and next steps, and means for confirming the end of the meeting. This allows for the equal invitation of all participants' opinions, efficient analysis of the content of comments, and progress of the meeting while linking them with past data. Furthermore, by understanding emotion data in real time and taking appropriate measures, smoother and more productive meeting management can be achieved.

[0951] "Means for sending attendance confirmation notices" refers to a device or system for sending attendance confirmation notices to all participants at the start of a conference.

[0952] The "means for receiving and displaying a user's attendance response" refers to a device or system for receiving information in response to a user's attendance notification and displaying the response.

[0953] The "means for sharing an agenda list" is a device or system for sharing the meeting agenda in list form with all participants.

[0954] The "means for setting a time limit for each agenda item" is a device or system for setting a time limit for the discussion of each agenda item and managing the progress thereof.

[0955] The "means for recording and transmitting user comments in real time" refers to a device or system for recording user comments in real time and transmitting them to a server.

[0956] The "means for encouraging other participants to give their opinions during silent periods" refers to a device or system for encouraging other participants to give their opinions when no one has spoken for a certain period of time.

[0957] "Means for analyzing comments using natural language processing and extracting important points and keywords" refers to a device or system that uses natural language processing technology to analyze the content of user comments and extract important points and keywords.

[0958] The "means for checking against a database of past meetings" refers to a device or system for checking the content of current statements against a database of past meetings to identify duplicate or similar statements.

[0959] The "means for displaying a notification of overlapping comments" is a device or system for displaying a notification to the user when overlapping comments are found as a result of comparison with past conferences.

[0960] The "means for analyzing and recording emotions contained in comments using an emotion engine" refers to a device or system for analyzing a user's emotions based on the content of comments and recording the results.

[0961] The "means for providing feedback based on emotional data" is a device or system for providing appropriate feedback or notifications to a user based on emotional data analyzed in real time.

[0962] The "means for summarizing a meeting and generating action points" is a device or system for summarizing the content of the discussion after the meeting and generating the next action steps.

[0963] The "Means for transmitting meeting summary and next steps" is a device or system for transmitting a meeting summary and future action points to all participants.

[0964] The "means for confirming the end of the conference" is a device or system for allowing participants to confirm the end of the conference.

[0965] This invention is a system for improving the productivity of corporate meetings, comprehensively managing meetings from start to finish, and utilizing an emotion engine to grasp the user's emotional state in real time and provide feedback. Specific embodiments are described below.

[0966] Hardware and Software Configuration

[0967] The server is equipped with a database server, a web server, a natural language processing engine, and a sentiment analysis engine. These engines can also use external APIs. For example, the system uses the Google Cloud Natural Language API for natural language processing and the IBM Watson Tone Analyzer for sentiment analysis. This enables advanced text analysis and sentiment analysis.

[0968] A terminal is a device (such as a PC, tablet, or smartphone) used by a participant, and provides an interface for sending and receiving information through communication with the server and for displaying and inputting information.

[0969] Starting a meeting

[0970] At the start of a meeting, the server sends an attendance confirmation notification to all participants. Specifically, it retrieves participants' email addresses and device information from a database and sends them all emails or push notifications (using SendGrid, for example). When a user receives the notification and presses the "attend" button, the response is sent from the device to the server. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on the device.

[0971] Examples:

[0972] Server: "Attendance confirmation has been sent."

[0973] Terminal: "The user pressed the attend button. A response was sent to the server."

[0974] Server: "Everyone is here. The agenda is as follows..."

[0975] Progress of the agenda

[0976] The server sets a time limit for each topic and monitors its progress. A timer is set at the start and a warning is issued before the time limit is exceeded, ensuring smooth progress in the discussion. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[0977] Examples:

[0978] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[0979] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[0980] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[0981] Speech analysis and emotion recognition

[0982] The server analyzes the user's comments using natural language processing. Specifically, it uses the Google Cloud Natural Language API to analyze the text data and extract important points and keywords. At the same time, the emotion engine uses IBM Watson Tone Analyzer to analyze the sentiment of the comments and record the results. The comments and the extracted points and keywords are compared with a database of past meetings, and if similar comments are found, the user is notified that they are duplicates.

[0983] Examples:

[0984] Server: "Analyzing user C's comment."

[0985] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[0986] Server: "This comment was raised in a previous meeting. We will notify the users."

[0987] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[0988] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[0989] Emotional feedback

[0990] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. Emotion data is accumulated and compared with past emotion data to track changes in emotions during the meeting and consider how to respond if necessary.

[0991] Examples:

[0992] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[0993] Server: "Send feedback on dissatisfaction."

[0994] Device: "The user has notified other participants that they are unhappy."

[0995] Ending the meeting and sharing

[0996] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this to all participants (using SendGrid, for example), and the terminal displays a meeting end confirmation screen to the user. The user presses the end button to officially end the meeting.

[0997] Examples:

[0998] Server: "All meeting agenda items have been completed. Generate summary and action points."

[0999] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1000] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1001] This system efficiently manages the progress of the agenda and analyzes comments, and also grasps the user's emotional state in real time, making for smoother and more constructive meetings.

[1002] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1003] Step 1: Send attendance confirmations

[1004] Specifically, when the meeting starts, the server retrieves the participants' email addresses and device information from the database. Based on the retrieved information, it uses an email sending API (e.g., SendGrid) to send attendance confirmation notifications to all participants.

[1005] Input: Participant's email address and device information obtained from the database.

[1006] Data processing: Convert email addresses and device information into a format suitable for the API.

[1007] Data output: Attendance confirmation notification sending result (success / failure).

[1008] Specific behavior:

[1009] Server: "Attendance confirmation has been sent."

[1010] Step 2: User's attendance response

[1011] When a user presses the attendance button on the terminal, a response is sent from the terminal to the server, which receives it and updates the attendance information in the database.

[1012] Input: User presses the Attend button.

[1013] Data processing: The server receives and analyzes the user's attendance response.

[1014] Data output: Update attendance information to database.

[1015] Specific behavior:

[1016] - Terminal: "The user pressed the attend button. A response was sent to the server."

[1017] - Server: "Attendance information has been updated to the database."

[1018] Step 3: Share the agenda list

[1019] After the server confirms that everyone is present, it shares the meeting agenda list with the terminals. The server retrieves the agenda list from the database and sends it to each terminal.

[1020] Input: A list of topics retrieved by the server from the database.

[1021] Data processing: The acquired agenda list is organized into a format suitable for the device.

[1022] Data output: Send the agenda list to each device.

[1023] Specific behavior:

[1024] - Server: "Agenda list shared."

[1025] Step 4: Set time limits for each agenda item

[1026] During the meeting, the server sets a time limit for each agenda item, runs a timer, and monitors the progress.

[1027] Input: Notifications at the start of each agenda item.

[1028] Data processing: setting time limits and starting timers.

[1029] Data output: Time management data (such as remaining time notification).

[1030] Specific behavior:

[1031] - Server: "Next on the agenda is 'Progress on Project X'. We'll be running it for 20 minutes."

[1032] Step 5: Record and send user comments

[1033] When a user speaks, the content is recorded in real time by the terminal and transmitted to the server.

[1034] Input: What the user says.

[1035] Data processing: Recorded as text data in real time.

[1036] Data output: Sends the statement to the server.

[1037] Specific behavior:

[1038] - Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1039] Step 6: Encourage feedback during silent periods

[1040] If no comments are made for a certain period of time, the server sends a notification to other participants prompting them to comment.

[1041] Input: Whether or not the speech was recorded and the time.

[1042] Data processing: Analyze periods of silence.

[1043] Data output: Notifications encouraging other participants to give their opinions.

[1044] Specific behavior:

[1045] - Server: "There are no new opinions, so I'll ask User B."

[1046] Step 7: Natural Language Processing Analysis of Speech

[1047] The server analyzes the acquired speech using natural language processing (NLP) technology and extracts important points and keywords.

[1048] Input: What the user says.

[1049] Data processing: Analyze using a natural language processing engine (e.g., Google Cloud Natural Language API).

[1050] Data output: Key points and keywords.

[1051] Specific behavior:

[1052] - Server: "Analyzing what you said. Identifying key points."

[1053] Step 8: Sentiment Analysis

[1054] The server uses an emotion engine to analyze the emotions contained in the user's comments and record the results.

[1055] Input: What the user says.

[1056] Data processing: Analyze using a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[1057] Data output: Sentiment analysis results.

[1058] Specific behavior:

[1059] - Server: "Analyzing emotions from user comments."

[1060] Step 9: Notification of duplicate posts

[1061] The server compares the content of comments with a database of past conferences and notifies the user if there are any duplicate comments.

[1062] Input: Analyzed speech content and historical database.

[1063] Data processing: database matching.

[1064] Data output: Duplicate comment notification.

[1065] Specific behavior:

[1066] - Server: "This comment was raised in a previous meeting. We will notify the user."

[1067] - Device: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[1068] Step 10: Provide emotional feedback

[1069] Based on the analysis results, the server provides feedback according to the user's emotional state and also sends notifications to take appropriate action.

[1070] Input: Sentiment analysis results.

[1071] Data processing: Generating feedback content.

[1072] Data output: Sending feedback notifications.

[1073] Specific behavior:

[1074] - Server: "Send feedback on dissatisfaction."

[1075] - Device: "The user has notified other participants that they are unhappy."

[1076] Step 11: Meeting summary and action points generation

[1077] Once all the agenda items have been completed, the server generates a meeting summary and action points and sends this to all participants by email.

[1078] Input: Agenda progress data and discussion content.

[1079] Data processing: summary and generation of action points.

[1080] Data output: sent by email.

[1081] Specific behavior:

[1082] - Server: "All meeting agenda items have been completed. Generate summary and action points."

[1083] - Server: "I've emailed you a summary of the meeting and upcoming actions."

[1084] Step 12: Confirm the end of the meeting

[1085] Finally, the terminal displays a confirmation screen for the user to end the conference and prompts the user to press the end button. When the user presses the end button, the conference is officially ended.

[1086] Input: End of meeting notification.

[1087] Data processing: Display of confirmation screen for completion.

[1088] Data output: Response confirming completion.

[1089] Specific behavior:

[1090] - Device: "The meeting has ended. Please press the confirmation button to end it."

[1091] These detailed steps streamline the overall operation and management of meetings and enable real-time analysis of user comments and emotions.

[1092] (Application example 2)

[1093] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1094] In corporate meetings, attendance confirmation, agenda management, and recording and analysis of comments are often done manually, resulting in problems that reduce meeting efficiency and productivity. It is also difficult to grasp participants' emotions during meetings and reflect their feedback in the progress of the meeting. These problems are particularly pronounced in workplaces such as factories, where effective meeting progress is essential.

[1095] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for sending an attendance confirmation notification, means for receiving and displaying a user's attendance response, means for sharing an agenda list, means for setting a time limit for each agenda, means for recording user comments in real time, means for prompting other participants during periods of silence, means for analyzing comments using natural language processing, means for comparing with a database of past meetings, means for displaying a notification of duplicate comments, means for generating a meeting summary and action points, means for sending a meeting summary and next steps, means for confirming the end of the meeting, means for analyzing emotions from user comments, means for accumulating and comparing emotional data, means for providing feedback according to the emotional state, means for displaying the emotional state, and means for operating a robot to support the progress of the meeting. This improves the efficiency of meetings, allows participants' emotional states to be grasped in real time, and appropriate feedback can be provided, enabling more constructive and effective meeting progress.

[1096] An "attendance confirmation notice" is a notice sent to all participants at the start of a conference to confirm the participants' intention to attend the conference.

[1097] The "user attendance response" is information that the user responds to upon receiving the attendance confirmation notice by pressing the attendance button or other methods, and this response is sent to the server.

[1098] An "agenda list" is a list of topics or themes to be discussed at a meeting that is shared to plan the meeting's progress.

[1099] A "time limit" is a time limit set for each agenda item, and is used to keep the meeting proceeding efficiently.

[1100] "User comments" refer to opinions and comments made by participants during a meeting, which are recorded in real time.

[1101] A "silent period" is a period of time in which no comments are made for a certain period of time or more, during which a means for encouraging other participants to give their opinions is activated.

[1102] "Natural language processing" is a technology that analyzes user statements and extracts important points and keywords, thereby helping to understand the content of the statements.

[1103] A "past meeting database" is a database that stores records of meetings that have taken place in the past and is used to match the content of a current meeting with the content of the previous meeting.

[1104] "Duplicate comment notification" is information that is sent to current speakers when similar comments are identified in past meetings, in order to prevent discussions on the same content.

[1105] A "meeting summary" is a summary of the results of the meeting discussion, including next steps and action points.

[1106] "Action points" are specific action plans or tasks that are derived from the results of the meeting, and clearly indicate the action plan after the meeting.

[1107] "Analyzing emotions" refers to a technique for identifying emotions contained in a user's speech and grasping their emotional state, and is implemented using an emotion engine.

[1108] "Providing feedback" means responding or reacting to the user based on the analyzed emotional state, which contributes to improving the progress of the meeting.

[1109] "Displaying emotional state" refers to visually displaying the analyzed emotional state of the user, allowing other participants to understand the situation.

[1110] "Operating a robot" means using a robot to assist in the progress of a meeting, improving the efficiency of the meeting and managing the emotions of participants.

[1111] The present invention is a system for improving productivity in meetings within a factory, which is combined with an emotion engine that recognizes the emotions of a user. The system of the present invention is configured as follows, and a specific embodiment thereof will be described.

[1112] Components

[1113] 1. Attendance confirmation notice

[1114] At the start of the conference, the server sends an attendance confirmation notification to all participants. Participants receive the notification on their devices and press the attendance button to send the information to the server.

[1115] 2. Share the agenda list

[1116] After confirming everyone's attendance, the server generates an agenda list for the meeting and displays it on each participant's device, allowing participants to understand the progress of the meeting.

[1117] 3. Set a time limit

[1118] A time limit is set for each topic, and the server monitors the progress. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to speak up.

[1119] 4. Recording and Analysis of Speech

[1120] When a user speaks, the content is recorded in real time by the device and sent to the server, which then analyzes the content using natural language processing (e.g., Python's Natural Language Toolkit (NLTK) or spaCy).

[1121] 5. Emotion Analysis

[1122] The server uses an emotion engine to analyze emotions from the content of comments, and the analysis results are accumulated in real time and compared with past emotion data.

[1123] 6. Providing Feedback

[1124] Depending on the analyzed emotional state, the server provides feedback to the participants. For example, if a user is feeling unhappy, the server notifies other participants and adjusts the progress of the meeting.

[1125] 7. Comparison with past meetings

[1126] The server compares the content of comments with a database of past meetings, and if there is any overlap, it notifies participants and makes past meeting records available for reference.

[1127] 8. Summary of the meeting and action points

[1128] Once all the agenda items have been completed, the server generates a meeting summary and action points and shares them with all participants.

[1129] 9. Confirmation of the end of the meeting

[1130] When the conference ends, the server confirms the end and, with the consent of all participants, the conference is officially ended.

[1131] Specific examples

[1132] The server performs the following series of operations:

[1133] Attendance confirmation notification: "Are you attending the meeting? Please press the attend button."

[1134] Sharing the agenda list: "Today's agenda list is as follows. Agenda 1: Improving production efficiency, Agenda 2: Safety measures"

[1135] Setting a time limit: "Next on the agenda is 'Improving Productivity'. We'll have 20 minutes."

[1136] Recording and analysis of speech: A user says, "I want to increase the speed of the production line," and the server records and analyzes this.

[1137] Analyzing emotions and providing feedback: "User A's sentiment was analyzed as 'high excitement'. The situation is good."

[1138] Cross-referencing: "This comment was also raised in a previous meeting. Would you like to review the previous meeting?"

[1139] Summary and action points: "The meeting is over. Summary: Specific action points were generated for 'improving production efficiency'. These have been sent via email."

[1140] Meeting End Confirmation: "The meeting has officially ended. Please press the confirmation button to end it."

[1141] Prompt Sentence Examples

[1142] "We're trying to come up with ideas to speed up the production line. How can we improve it while still maintaining safety?"

[1143] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1144] Step 1:

[1145] The server sends an attendance confirmation notification to all participants at the start of the conference. This notification is necessary to confirm attendance status, and participants who receive the notification press the attendance button on their terminal. The input is the instruction to send the attendance confirmation notification, and the output is the result of the notification being sent. Data processing involves generating a notification based on the participant list and sending it to each participant's terminal.

[1146] Step 2:

[1147] The user receives the attendance confirmation notification and presses the attendance button. This generates an attendance response, and the information is sent to the server. The input is the attendance confirmation notification, and the output is the attendance response. Data processing involves generating the attendance response by pressing the button and sending it to the server.

[1148] Step 3:

[1149] After the server confirms everyone's attendance, it generates an agenda list for the meeting and displays it on each participant's device. The input is the aggregated attendance responses, and the output is the displayed agenda list. Data processing involves creating an agenda list after attendance confirmation and distributing it to each participant.

[1150] Step 4:

[1151] The server sets a time limit for each agenda item and monitors the progress of the meeting while it is in progress. If no one speaks for a certain amount of time, the server sends a notification to other participants encouraging them to comment. The input is the agenda list, and the output is the time limit setting information and a notification encouraging comments. Data processing involves setting a time limit for each agenda item and sending notifications based on the elapsed time.

[1152] Step 5:

[1153] When a user speaks, the content of the speech is recorded in real time by the device and the data is sent to the server. The input is the user's speech and the output is the speech recording data. Data processing involves converting the speech into text data using voice recognition and sending that data to the server.

[1154] Step 6:

[1155] The server analyzes the received speech using natural language processing tools (e.g., Python's Natural Language Toolkit (NLTK) or spaCy). The input is the speech recording data, and the output is the analyzed keywords and key points. Data processing involves analyzing the text data and extracting important points.

[1156] Step 7:

[1157] The server then uses an emotion engine to analyze the analyzed speech content and identify the emotions contained in the speech. The input is the analyzed speech data, and the output is the emotion analysis results. Data processing involves using an emotion estimation algorithm (e.g., EmotionAnalyzer) to identify the emotions.

[1158] Step 8:

[1159] The server provides feedback to participants based on the analyzed emotional state. For example, if a user is feeling dissatisfied, the server notifies the other participants of this information and adjusts the progress of the meeting. The input is the emotion analysis result, and the output is a feedback notification. Data processing involves generating appropriate feedback based on the emotion analysis result and sending it to the necessary participants.

[1160] Step 9:

[1161] The server compares the content of comments with a database of past meetings and notifies participants if there is any overlap. The input is the analyzed comment data and the past meeting database, and the output is a notification of duplicate comments. Data processing involves matching with past meeting records.

[1162] Step 10:

[1163] Once all agenda items have been completed, the server generates a meeting summary and action points and shares them with all participants. The input is the meeting progress data, and the output is the summary and action points. Data processing involves summarizing the discussion and next steps and distributing them to participants.

[1164] Step 11:

[1165] When the meeting ends, the server confirms the end and, with the consent of all participants, the meeting officially ends. The input is the end confirmation instruction, and the output is the end confirmation result. Data processing involves executing the end confirmation process and collecting and displaying the results.

[1166] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1167] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1168] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1169] [Third embodiment]

[1170] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1171] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1172] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1173] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1174] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1175] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1176] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1177] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1178] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1179] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1180] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1181] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1182] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments of each function of the system will be described below.

[1183] Starting a meeting

[1184] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent via email or the notification function of the conference system. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[1185] Examples:

[1186] Server: "Attendance confirmation has been sent."

[1187] Terminal: "The user pressed the attend button. A response was sent to the server."

[1188] Server: "Everyone is here. The agenda is as follows..."

[1189] Progress of the agenda

[1190] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[1191] Examples:

[1192] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1193] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1194] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[1195] Speech analysis

[1196] The server analyzes what users say using natural language processing, extracting important points and keywords. These points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[1197] Examples:

[1198] Server: "Analyzing user C's comment."

[1199] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[1200] Server: "This comment was raised in a previous meeting. We will notify the users."

[1201] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[1202] Ending the meeting and sharing

[1203] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[1204] Examples:

[1205] Server: "All meeting agenda items have been completed. Generate summary and action points."

[1206] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1207] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1208] In this way, the system of the present invention improves the productivity of meetings by efficiently managing the entire flow from the start to the end of the meeting. The server, terminals, and users each fulfill their respective roles, and the system operates smoothly as a whole, allowing employees to effectively understand the content of the meeting and quickly return to their main tasks.

[1209] The processing flow will be explained below.

[1210] Step 1: Send attendance confirmations

[1211] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[1212] Step 2: Receiving attendance responses

[1213] Terminal: "When the user presses the attend button, the response is sent to the server."

[1214] Step 3: Complete attendance confirmation

[1215] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[1216] Step 4: Set a time limit for the agenda

[1217] Server: "Set a time limit for each topic and start the timer."

[1218] Step 5: Record what you say

[1219] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[1220] Step 6: Prompt for silence

[1221] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[1222] Step 7: Natural Language Analysis of Speech

[1223] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[1224] Step 8: Match with past meetings

[1225] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[1226] Step 9: Notification of duplicate posts

[1227] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[1228] Step 10: Generate a meeting summary

[1229] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[1230] Step 11: Submit your summary and action plan

[1231] Server: "Send an email to all participants with a meeting summary and next steps."

[1232] Step 12: Confirm the end of the meeting

[1233] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[1234] The above are the specific processing steps of the system. By executing each step in order, the meeting will proceed smoothly and effective decision-making will be possible.

[1235] Example 1

[1236] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1237] Corporate meetings often do not proceed effectively, resulting in wasted time, delays in moving forward with agenda items, and overlooking important comments. This can result in lower meeting productivity and delays in important decisions. It is also difficult to keep track of past meeting content and prevent new comments from overlapping. A system that solves these problems and improves meeting productivity is needed.

[1238] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1239] In this invention, the server includes means for sending attendance confirmation notices, means for receiving and displaying user attendance responses, means for sharing an agenda list, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants for their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for generating a meeting summary and action items, means for sending a meeting summary and next steps, means for confirming the end of the meeting, means for sending a meeting attendance confirmation notice via email or a notification function of the conference system, means for converting user comments into text using speech recognition technology, means for saving the recorded comments in real time in a memory area, means for notifying users of the comment analysis results, means for searching past meeting records for similar comments, and means for generating an agenda list in HTML format and displaying it on a terminal.This enables efficient meeting progress, prevents important comments from being overlooked, reduces duplicate comments, and clarifies summaries and action points.

[1240] 1. "Attendance confirmation notice" means a means of communication to confirm the attendance of meeting participants.

[1241] 2. "User attendance response" refers to a conference participant's indication of intent to attend in response to an attendance confirmation notice.

[1242] 3. An "agenda list" is a list of items or topics to be discussed at a meeting.

[1243] 4. "Time limit" means the specific time limit assigned to proceed with each agenda item.

[1244] 5. "Real-time recording" refers to recording what is said immediately on the spot.

[1245] 6. "Silent period" refers to a period of time during which no participant speaks.

[1246] 7. "Natural language processing" is a technology that uses computers to analyze human language and understand and process its meaning and syntax.

[1247] 8. "Past Meeting Database" means data storage that accumulates and manages records of previously held meetings.

[1248] 9. "Duplicate comment notification" is a function that notifies participants when a comment is made that is the same as something that has already been discussed.

[1249] 10. A "summary" is a summary of the overall content of the meeting.

[1250] 11. "Action items" are specific next steps or tasks determined as a result of a meeting.

[1251] 12. A "summary" is a concise summary of what was discussed and decided at a meeting.

[1252] 13. “Next steps” are specific actions or plans to be taken after the meeting.

[1253] 14. "Sending attendance confirmation notices by email or via the notification function of the conference system" refers to sending attendance confirmation notices using the notification function of an email service or online conference tool.

[1254] 15. "Speech recognition technology" is technology that converts speech into text.

[1255] 16. "Memory Area" means a computer storage area for storing data.

[1256] 17. "Analyzing the results of speech analysis" refers to analyzing the content of users' speech and evaluating and using the results.

[1257] 18. "Searching for similar statements" refers to searching for similar content or opinions in past meeting records.

[1258] 19. "Generating an agenda list in HTML format and displaying it on a device" means creating an agenda list in web page format and displaying it on a participant's device.

[1259] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments thereof will be described below.

[1260] Starting a meeting

[1261] The server first sends attendance confirmation notifications to all participants. Attendance confirmation notifications are sent by email or using the conference system's notification function. Users who receive this notification press the attendance button to indicate their intention to attend. The terminal detects the user's attendance response and sends it to the server. After receiving all attendance responses, the server generates an agenda list and displays it on each participant's terminal.

[1262] Examples:

[1263] Server: "Everyone is here. Here's the agenda..."

[1264] Terminal: "The user pressed the attend button. A response was sent to the server."

[1265] Progress of the agenda

[1266] The server sets a time limit for each agenda item and monitors the progress of the agenda items. When a comment is made, the device records the content in real time and sends it to the server. If no comment is made for a certain period of time, the server sends a notification to other participants urging them to contribute. This process ensures the smooth flow of the meeting.

[1267] Examples:

[1268] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1269] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1270] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[1271] Speech analysis

[1272] The server uses natural language processing to analyze what users say. This analysis extracts important points and keywords from the comments. These points and keywords are then compared with a database of past meetings, and if similar comments are made, the system notifies the user.

[1273] Examples:

[1274] Server: "Analyzing user C's comment."

[1275] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[1276] Server: "This comment was raised in a previous meeting. We will notify the users."

[1277] Ending the meeting and sharing

[1278] After all the agenda items have been completed, the server generates a summary of the meeting and action points. This summary and action points are sent to all participants by email. The terminal displays a confirmation screen for the end of the meeting to the user, and the user can close the meeting by pressing the end button.

[1279] Examples:

[1280] Server: "All meeting agenda items have been completed. Generate summary and action points."

[1281] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1282] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1283] Hardware and software used

[1284] High-performance database servers are used as servers. Specifically, general-purpose database servers or cloud-based database services (such as Amazon RDS) are used. Laptops and tablets (such as regular laptops and tablets) are used as devices. Software uses the Google Workspace email API and Microsoft Teams notification function. The Google Cloud NLP API is used for natural language processing.

[1285] Prompt Sentence Examples

[1286] "Please describe in detail a system that improves the productivity of corporate meetings. Explain each step from the start to the end of a meeting from the perspective of the server, device, and user, including specific hardware and software."

[1287] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1288] Step 1:

[1289] The server sends attendance confirmations to all participants at the start of the conference.

[1290] Input: List of conference participants

[1291] Processing: The server uses the Google Workspace email API or the conferencing system's notification function to generate and send attendance confirmation notifications to all participants.

[1292] Output: Notification of completion of sending of attendance confirmation notice

[1293] Step 2:

[1294] The user receives a confirmation of attendance.

[1295] Input: Attendance confirmation notification from the server

[1296] Action: The user checks the notification or email they received and clicks the attend button.

[1297] Output: Click on the attendance button (attendance response)

[1298] Step 3:

[1299] The terminal sends the user's response to the server.

[1300] Input: User's attendance response (clicking the attendance button)

[1301] Processing: The terminal sends the user's response to the server via an HTTP POST request.

[1302] Output: Attendance response data sent to the server

[1303] Step 4:

[1304] The server verifies the attendance of all participants.

[1305] Input: Attendance response data from each participant

[1306] Processing: The server receives responses from all participants and checks whether everyone has confirmed their attendance. Reminder emails are sent again if necessary.

[1307] Output: Attendance confirmation completion notification

[1308] Step 5:

[1309] The server instructs each participant to display the agenda list on their terminal.

[1310] Input: Agenda list data

[1311] Processing: The server generates an agenda list and sends it to each terminal using WebSocket. It also configures it to be displayed in HTML format.

[1312] Output: Send and display agenda list

[1313] Step 6:

[1314] The server sets a time limit for each topic.

[1315] Input: Agenda list data

[1316] Processing: The server sets a timer using Python's time library at the start of each agenda item.

[1317] Output: Timer setting completion notification for each agenda item

[1318] Step 7:

[1319] The server monitors the progress of the agenda.

[1320] Input: Ongoing data for each agenda item

[1321] Processing: The server periodically checks the elapsed time of the agenda and evaluates the progress.

[1322] Output: Progress monitoring data

[1323] Step 8:

[1324] When a user speaks, the content is recorded in real time by the terminal.

[1325] Input: User's voice data

[1326] Processing: The device converts the voice data into text using voice recognition technology and sends it to the server in real time.

[1327] Output: Text data to the server

[1328] Step 9:

[1329] If no one speaks for a certain period of time, the server sends a notification to other participants urging them to speak up.

[1330] Input: Data for silent periods

[1331] Processing: When the silent period exceeds the set time, the server sends a notification to other participants using the Slack API.

[1332] Output: Notice of opinion promotion

[1333] Step 10:

[1334] The server analyzes the content of the user's comments.

[1335] Input: Text data sent to the server

[1336] Processing: The server uses Google Cloud NLP API to parse the text data and extract key points and keywords.

[1337] Output: Analysis results (important points and keywords)

[1338] Step 11:

[1339] The server matches the extracted data with past conference data.

[1340] Input: Analysis results (important points and keywords)

[1341] Processing: The server accesses a database of stored past meetings and uses the Cosine Similarity algorithm to search for similar statements.

[1342] Output: Search results for similar statements

[1343] Step 12:

[1344] The server notifies the user if there is a duplicate message.

[1345] Input: Search results for similar statements

[1346] Processing: When a duplicate message is detected, the server generates a notification and sends it to the relevant user. The device receives the notification and displays it to the user.

[1347] Output: Duplicate message notification

[1348] Step 13:

[1349] The server generates a summary and action points for the meeting.

[1350] Input: All data recorded at the meeting

[1351] Processing: The server summarizes all the agenda items and their conclusions of the meeting, automatically sets action points, saves them in Excel format and generates a summary.

[1352] Output: Meeting summary and action points

[1353] Step 14:

[1354] The server will email the meeting summary and action points to all participants.

[1355] Input: Meeting summary and action points

[1356] Processing: The server uses Google Workspace's email API to send the summary and action points to each participant in PDF format.

[1357] Output: Email with summary and action points

[1358] Step 15:

[1359] The terminal displays a confirmation screen for ending the conference to the user, prompting the user to press the end button.

[1360] Input: Conference end notification from the server

[1361] Processing: The terminal receives the notification of the end of the conference and displays the end confirmation screen. The user clicks the end button to confirm.

[1362] Output: User confirms end of meeting

[1363] By dividing the process into detailed steps in this way, the operation of the entire system becomes clear. These steps efficiently manage the entire flow from the start to the end of the conference.

[1364] (Application example 1)

[1365] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1366] The efficiency of conferences and meetings within factories is extremely important, but it is often difficult to advance productive discussions in an environment where multiple participants simultaneously exchange opinions. Furthermore, revisiting topics and opinions already discussed in previous meetings wastes time and reduces meeting productivity. Furthermore, summarizing and sharing action points after meetings is time-consuming, and there is a risk that important information may be overlooked. There is a need for a system that can solve these issues and improve the productivity and efficiency of meetings.

[1367] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1368] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants to give their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing with a database of past meetings, means for displaying notifications of duplicate comments using a generative AI model, means for generating a meeting summary and action points, means for sending a meeting summary and next steps, means for confirming the end of the meeting, and means for generating prompt sentences to improve the efficiency of meetings within the factory. This makes it possible to improve the efficiency and productivity of factory meetings.

[1369] An "attendance confirmation notice" is a notice sent at the start of a conference or meeting to confirm the attendance of participants.

[1370] "User's attendance response" is response information sent by the user in response to the attendance confirmation notice.

[1371] An "agenda list" is a list of topics to be discussed at a conference or meeting.

[1372] A "time limit" is a time limit set for each agenda item.

[1373] "Real-time recording" means that what a user says is instantly saved in digital form.

[1374] A "silent period" is a period of time during a conference where no one speaks for a certain period of time or more.

[1375] A "notice encouraging opinions" is a notification sent to other participants during a silent period to encourage them to give their opinions.

[1376] "Natural language processing" is a technology that analyzes user statements and extracts important points and keywords.

[1377] "Past Meeting Database" means a database that aggregates and stores records of meetings that have been held in the past.

[1378] A "generative AI model" is a model that uses artificial intelligence in the generation process.

[1379] The "duplicate comment notification" is a notification that notifies the user of content similar to a comment that has already been mentioned in a past conference.

[1380] A "meeting summary" is a summary of what was discussed in the meeting and outlines next steps and action points.

[1381] "Action points" are specific next steps to take as a result of the meeting.

[1382] A "prompt sentence" is a specific instruction sentence for performing a specific task.

[1383] The present invention is a system for improving the efficiency of conferences and meetings in a factory. Specific embodiments of the system will be described below.

[1384] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent to the participant's device (smartphone, smart glasses, head-mounted display, etc.), and the participant responds by pressing the attendance button. This attendance information is immediately sent to the server, and the attendance of all participants is confirmed.

[1385] The server then shares the agenda list with all participants' devices, allowing each participant to prepare for the meeting. A time limit is also set for each agenda item, and the server monitors its progress. When a comment is made, the content is recorded in real time by the device and sent to the server. During periods of silence, the server sends notifications to other participants urging them to contribute, preventing the discussion from stagnating.

[1386] The server analyzes user comments using natural language processing technology. It uses libraries such as Spacy to extract key points and keywords. The extracted information is then compared with a database of past meetings to determine whether there are any overlapping comments. If overlapping comments are detected using a generative AI model, a notification is sent to the user.

[1387] When the meeting ends, the server generates a summary of the meeting and action points, and sends them to all participants. This summary includes the conclusions and next steps for each agenda item. A confirmation screen for the end of the meeting is also displayed on the terminal, and participants can press the confirmation button to end the meeting.

[1388] As a concrete example, the following prompts can be fed into a generative AI model to help guide a meeting:

[1389] Analyze the statements and extract key keywords. Here is an example statement:

[1390] "Do you have any comments on the progress of Project X?"

[1391] In this way, the system of the present invention can improve the efficiency and productivity of conferences and meetings within factories. The hardware used includes factory robots, smartphones, smart glasses, and head-mounted displays, and the software includes a server (Node.js server) and a natural language processing library (Spacy).

[1392] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1393] Step 1:

[1394] The server sends an attendance confirmation notification to all participants at the start of the conference. The inputs are the participant list and the start time of the conference. The attendance notification is sent and displayed on each participant's device.

[1395] Step 2:

[1396] The user responds by pressing the attendance button on the terminal. The input is the user's operation (clicking the attendance button) when the attendance confirmation notification is displayed. When the attendance button is pressed, the response information is sent to the server.

[1397] Step 3:

[1398] The server processes the received attendance responses and confirms the attendance of all participants. The input is the attendance response data sent by each participant. The attendance information is tallied, and once all attendance is confirmed, the server proceeds to the next step.

[1399] Step 4:

[1400] The server shares the agenda list with all participants' devices. The input is the meeting agenda list. The agenda list is displayed on each device and shared with the participants.

[1401] Step 5:

[1402] The server sets a time limit for each agenda item and monitors its progress. The inputs are the time limit and start time for each agenda item. The timer starts and the progress is monitored in real time.

[1403] Step 6:

[1404] Users comment on topics. The input is the content of the user's comments. The content of the comments is recorded in real time on the terminal, and the data is sent to the server.

[1405] Step 7:

[1406] The server sends notifications to other participants during periods of silence, encouraging them to contribute their opinions. An input is a period of no speech. When a period of silence is detected, a notification is sent to other participants encouraging them to contribute their opinions.

[1407] Step 8:

[1408] The server analyzes the received comments using natural language processing technology. The input is the user's comment data. Natural language processing (using the Spacy library) is performed to extract important keywords and key points.

[1409] Step 9:

[1410] The server compares the extracted keywords and points with a database of past conferences. The inputs are the extracted keywords and points and the database of past conferences. Database comparison is performed to detect duplicated comments.

[1411] Step 10:

[1412] The server uses the generative AI model to display a notification of duplicated comments to the user. The inputs are the matching results and the duplicate comment detection information. The detected duplicate comments are notified to the user.

[1413] Step 11:

[1414] After the meeting, the server generates a summary of the meeting and action points. The input is all speech data collected during the meeting and a summary of each agenda item. The summary and action points are automatically generated and sent to all participants by email.

[1415] Step 12:

[1416] The user presses the conference end confirmation button on the terminal. As input, there are buttons displayed on the end confirmation screen. When the end confirmation button is pressed, the server records the end of the conference and the conference is officially ended.

[1417] The above steps will improve the efficiency of meetings within the factory and increase productivity.

[1418] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1419] The present invention is a system for improving the productivity of corporate meetings, which is combined with an emotion engine that recognizes the emotions of users. Specific embodiments of each function of the system including the emotion engine will be described below.

[1420] Starting a meeting

[1421] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[1422] Examples:

[1423] Server: "Attendance confirmation has been sent."

[1424] Terminal: "The user pressed the attend button. A response was sent to the server."

[1425] Server: "Everyone is here. The agenda is as follows..."

[1426] Progress of the agenda

[1427] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[1428] Examples:

[1429] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1430] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1431] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[1432] Speech analysis and emotion recognition

[1433] The server uses natural language processing to analyze what users say, extracting important points and keywords. At the same time, an emotion engine analyzes and records the emotions contained in the user's comments. The extracted points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[1434] Examples:

[1435] Server: "Analyzing user C's comment."

[1436] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[1437] Server: "This comment was raised in a previous meeting. We will notify the users."

[1438] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[1439] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[1440] Emotional feedback

[1441] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. In addition, emotion data is accumulated and compared with past emotion data, allowing changes in emotions during a meeting to be tracked.

[1442] Examples:

[1443] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[1444] Server: "Send feedback on dissatisfaction."

[1445] Device: "The user has notified other participants that they are unhappy."

[1446] Ending the meeting and sharing

[1447] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[1448] Examples:

[1449] Server: "All meeting agenda items have been completed. Generate summary and action points."

[1450] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1451] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1452] As described above, the system of the present invention effectively manages a meeting from start to finish through a series of flows, and by adding an emotion engine, it also takes into account emotional aspects during the meeting, thereby realizing smoother and more constructive meetings.

[1453] The processing flow will be explained below.

[1454] Step 1: Send attendance confirmations

[1455] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[1456] Step 2: Receiving attendance responses

[1457] Terminal: "When the user presses the attend button, the response is sent to the server."

[1458] Step 3: Complete attendance confirmation

[1459] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[1460] Step 4: Set a time limit for the agenda

[1461] Server: "Set a time limit for each topic and start the timer."

[1462] Step 5: Record what you say

[1463] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[1464] Step 6: Prompt for silence

[1465] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[1466] Step 7: Natural Language Analysis of Speech

[1467] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[1468] Step 8: Sentiment Analysis

[1469] Emotion engine: "Based on the content of the user's speech, analyzes the emotions at the time of the speech and generates emotional data."

[1470] Step 9: Provide emotional feedback

[1471] Emotion engine: "Provides appropriate feedback to users in real time based on analyzed emotional data."

[1472] Step 10: Match with past meetings

[1473] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[1474] Step 11: Notification of duplicate posts

[1475] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[1476] Step 12: Generate a meeting summary

[1477] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[1478] Step 13: Submit your summary and action plan

[1479] Server: "Send an email to all participants with a meeting summary and next steps."

[1480] Step 14: Confirm the end of the meeting

[1481] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[1482] Examples:

[1483] The flow from the start to the end of a meeting is shown below.

[1484] Step 1:

[1485] Server: "Attendance confirmation has been sent."

[1486] Step 2:

[1487] Terminal: "The user pressed the attend button. A response was sent to the server."

[1488] Step 3:

[1489] Server: "Everyone is here. The agenda is as follows..."

[1490] Step 4:

[1491] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1492] Step 5:

[1493] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1494] Step 6:

[1495] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[1496] Step 7:

[1497] Server: "Analyzing User C's comments. Key points identified."

[1498] Step 8:

[1499] Emotion Engine: "Analyzing emotions from user C's comments. The result is 'Go-heung'."

[1500] Step 9:

[1501] Emotion Engine: "Provide positive feedback because the user is in a 'high-exciting' state."

[1502] Step 10:

[1503] Server: "I checked with past meetings. Similar opinions were expressed in previous meetings."

[1504] Step 11:

[1505] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[1506] Step 12:

[1507] Server: "All meeting agenda items have been completed. Generate summary and action points."

[1508] Step 13:

[1509] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1510] Step 14:

[1511] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1512] These are the specific processing steps of the system that includes the emotion engine. By executing each step in order, the meeting will proceed effectively and participants will be emotionally satisfied.

[1513] Example 2

[1514] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1515] Corporate meetings often have a tendency for certain participants to speak, resulting in meetings that don't proceed smoothly or don't end on time. Furthermore, participants' feelings and intentions are not properly shared during the meeting, which can lead to misunderstandings and frustration. Furthermore, many topics and comments overlap with previously discussed content, making it difficult to hold efficient discussions. There is a need to solve these problems and improve meeting productivity.

[1516] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1517] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording and transmitting user comments in real time, means for encouraging other participants to contribute during periods of silence, means for analyzing comments using natural language processing and extracting important points and keywords, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for analyzing and recording emotions contained in comments using an emotion engine, means for providing feedback based on emotion data, means for generating a meeting summary and action points, means for transmitting a meeting summary and next steps, and means for confirming the end of the meeting. This allows for the equal invitation of all participants' opinions, efficient analysis of the content of comments, and progress of the meeting while linking them with past data. Furthermore, by understanding emotion data in real time and taking appropriate measures, smoother and more productive meeting management can be achieved.

[1518] "Means for sending attendance confirmation notices" refers to a device or system for sending attendance confirmation notices to all participants at the start of a conference.

[1519] The "means for receiving and displaying a user's attendance response" refers to a device or system for receiving information in response to a user's attendance notification and displaying the response.

[1520] The "means for sharing an agenda list" is a device or system for sharing the meeting agenda in list form with all participants.

[1521] The "means for setting a time limit for each agenda item" is a device or system for setting a time limit for the discussion of each agenda item and managing the progress thereof.

[1522] The "means for recording and transmitting user comments in real time" refers to a device or system for recording user comments in real time and transmitting them to a server.

[1523] The "means for encouraging other participants to give their opinions during silent periods" refers to a device or system for encouraging other participants to give their opinions when no one has spoken for a certain period of time.

[1524] "Means for analyzing comments using natural language processing and extracting important points and keywords" refers to a device or system that uses natural language processing technology to analyze the content of user comments and extract important points and keywords.

[1525] The "means for checking against a database of past meetings" refers to a device or system for checking the content of current statements against a database of past meetings to identify duplicate or similar statements.

[1526] The "means for displaying a notification of overlapping comments" is a device or system for displaying a notification to the user when overlapping comments are found as a result of comparison with past conferences.

[1527] The "means for analyzing and recording emotions contained in comments using an emotion engine" refers to a device or system for analyzing a user's emotions based on the content of comments and recording the results.

[1528] The "means for providing feedback based on emotional data" is a device or system for providing appropriate feedback or notifications to a user based on emotional data analyzed in real time.

[1529] The "means for summarizing a meeting and generating action points" is a device or system for summarizing the content of the discussion after the meeting and generating the next action steps.

[1530] The "Means for transmitting meeting summary and next steps" is a device or system for transmitting a meeting summary and future action points to all participants.

[1531] The "means for confirming the end of the conference" is a device or system for allowing participants to confirm the end of the conference.

[1532] This invention is a system for improving the productivity of corporate meetings, comprehensively managing meetings from start to finish, and utilizing an emotion engine to grasp the user's emotional state in real time and provide feedback. Specific embodiments are described below.

[1533] Hardware and Software Configuration

[1534] The server is equipped with a database server, a web server, a natural language processing engine, and a sentiment analysis engine. These engines can also use external APIs. For example, the system uses the Google Cloud Natural Language API for natural language processing and the IBM Watson Tone Analyzer for sentiment analysis. This enables advanced text analysis and sentiment analysis.

[1535] A terminal is a device (such as a PC, tablet, or smartphone) used by a participant, and provides an interface for sending and receiving information through communication with the server and for displaying and inputting information.

[1536] Starting a meeting

[1537] At the start of a meeting, the server sends an attendance confirmation notification to all participants. Specifically, it retrieves participants' email addresses and device information from a database and sends them all emails or push notifications (using SendGrid, for example). When a user receives the notification and presses the "attend" button, the response is sent from the device to the server. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on the device.

[1538] Examples:

[1539] Server: "Attendance confirmation has been sent."

[1540] Terminal: "The user pressed the attend button. A response was sent to the server."

[1541] Server: "Everyone is here. The agenda is as follows..."

[1542] Progress of the agenda

[1543] The server sets a time limit for each topic and monitors its progress. A timer is set at the start and a warning is issued before the time limit is exceeded, ensuring smooth progress in the discussion. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[1544] Examples:

[1545] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1546] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1547] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[1548] Speech analysis and emotion recognition

[1549] The server analyzes the user's comments using natural language processing. Specifically, it uses the Google Cloud Natural Language API to analyze the text data and extract important points and keywords. At the same time, the emotion engine uses IBM Watson Tone Analyzer to analyze the sentiment of the comments and record the results. The comments and the extracted points and keywords are compared with a database of past meetings, and if similar comments are found, the user is notified that they are duplicates.

[1550] Examples:

[1551] Server: "Analyzing user C's comment."

[1552] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[1553] Server: "This comment was raised in a previous meeting. We will notify the users."

[1554] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[1555] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[1556] Emotional feedback

[1557] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. Emotion data is accumulated and compared with past emotion data to track changes in emotions during the meeting and consider how to respond if necessary.

[1558] Examples:

[1559] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[1560] Server: "Send feedback on dissatisfaction."

[1561] Device: "The user has notified other participants that they are unhappy."

[1562] Ending the meeting and sharing

[1563] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this to all participants (using SendGrid, for example), and the terminal displays a meeting end confirmation screen to the user. The user presses the end button to officially end the meeting.

[1564] Examples:

[1565] Server: "All meeting agenda items have been completed. Generate summary and action points."

[1566] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1567] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1568] This system efficiently manages the progress of the agenda and analyzes comments, and also grasps the user's emotional state in real time, making for smoother and more constructive meetings.

[1569] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1570] Step 1: Send attendance confirmations

[1571] Specifically, when the meeting starts, the server retrieves the participants' email addresses and device information from the database. Based on the retrieved information, it uses an email sending API (e.g., SendGrid) to send attendance confirmation notifications to all participants.

[1572] Input: Participant's email address and device information obtained from the database.

[1573] Data processing: Convert email addresses and device information into a format suitable for the API.

[1574] Data output: Attendance confirmation notification sending result (success / failure).

[1575] Specific behavior:

[1576] Server: "Attendance confirmation has been sent."

[1577] Step 2: User's attendance response

[1578] When a user presses the attendance button on the terminal, a response is sent from the terminal to the server, which receives it and updates the attendance information in the database.

[1579] Input: User presses the Attend button.

[1580] Data processing: The server receives and analyzes the user's attendance response.

[1581] Data output: Update attendance information to database.

[1582] Specific behavior:

[1583] - Terminal: "The user pressed the attend button. A response was sent to the server."

[1584] - Server: "Attendance information has been updated to the database."

[1585] Step 3: Share the agenda list

[1586] After the server confirms that everyone is present, it shares the meeting agenda list with the terminals. The server retrieves the agenda list from the database and sends it to each terminal.

[1587] Input: A list of topics retrieved by the server from the database.

[1588] Data processing: The acquired agenda list is organized into a format suitable for the device.

[1589] Data output: Send the agenda list to each device.

[1590] Specific behavior:

[1591] - Server: "Agenda list shared."

[1592] Step 4: Set time limits for each agenda item

[1593] During the meeting, the server sets a time limit for each agenda item, runs a timer, and monitors the progress.

[1594] Input: Notifications at the start of each agenda item.

[1595] Data processing: setting time limits and starting timers.

[1596] Data output: Time management data (such as remaining time notification).

[1597] Specific behavior:

[1598] - Server: "Next on the agenda is 'Progress on Project X'. We'll be running it for 20 minutes."

[1599] Step 5: Record and send user comments

[1600] When a user speaks, the content is recorded in real time by the terminal and transmitted to the server.

[1601] Input: What the user says.

[1602] Data processing: Recorded as text data in real time.

[1603] Data output: Sends the statement to the server.

[1604] Specific behavior:

[1605] - Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1606] Step 6: Encourage feedback during silent periods

[1607] If no comments are made for a certain period of time, the server sends a notification to other participants prompting them to comment.

[1608] Input: Whether or not the speech was recorded and the time.

[1609] Data processing: Analyze periods of silence.

[1610] Data output: Notifications encouraging other participants to give their opinions.

[1611] Specific behavior:

[1612] - Server: "There are no new opinions, so I'll ask User B."

[1613] Step 7: Natural Language Processing Analysis of Speech

[1614] The server analyzes the acquired speech using natural language processing (NLP) technology and extracts important points and keywords.

[1615] Input: What the user says.

[1616] Data processing: Analyze using a natural language processing engine (e.g., Google Cloud Natural Language API).

[1617] Data output: Key points and keywords.

[1618] Specific behavior:

[1619] - Server: "Analyzing what you said. Identifying key points."

[1620] Step 8: Sentiment Analysis

[1621] The server uses an emotion engine to analyze the emotions contained in the user's comments and record the results.

[1622] Input: What the user says.

[1623] Data processing: Analyze using a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[1624] Data output: Sentiment analysis results.

[1625] Specific behavior:

[1626] - Server: "Analyzing emotions from user comments."

[1627] Step 9: Notification of duplicate posts

[1628] The server compares the content of comments with a database of past conferences and notifies the user if there are any duplicate comments.

[1629] Input: Analyzed speech content and historical database.

[1630] Data processing: database matching.

[1631] Data output: Duplicate comment notification.

[1632] Specific behavior:

[1633] - Server: "This comment was raised in a previous meeting. We will notify the user."

[1634] - Device: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[1635] Step 10: Provide emotional feedback

[1636] Based on the analysis results, the server provides feedback according to the user's emotional state and also sends notifications to take appropriate action.

[1637] Input: Sentiment analysis results.

[1638] Data processing: Generating feedback content.

[1639] Data output: Sending feedback notifications.

[1640] Specific behavior:

[1641] - Server: "Send feedback on dissatisfaction."

[1642] - Device: "The user has notified other participants that they are unhappy."

[1643] Step 11: Meeting summary and action points generation

[1644] Once all the agenda items have been completed, the server generates a meeting summary and action points and sends this to all participants by email.

[1645] Input: Agenda progress data and discussion content.

[1646] Data processing: summary and generation of action points.

[1647] Data output: sent by email.

[1648] Specific behavior:

[1649] - Server: "All meeting agenda items have been completed. Generate summary and action points."

[1650] - Server: "I've emailed you a summary of the meeting and upcoming actions."

[1651] Step 12: Confirm the end of the meeting

[1652] Finally, the terminal displays a confirmation screen for the user to end the conference and prompts the user to press the end button. When the user presses the end button, the conference is officially ended.

[1653] Input: End of meeting notification.

[1654] Data processing: Display of confirmation screen for completion.

[1655] Data output: Response confirming completion.

[1656] Specific behavior:

[1657] - Device: "The meeting has ended. Please press the confirmation button to end it."

[1658] These detailed steps streamline the overall operation and management of meetings and enable real-time analysis of user comments and emotions.

[1659] (Application example 2)

[1660] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1661] In corporate meetings, attendance confirmation, agenda management, and recording and analysis of comments are often done manually, resulting in problems that reduce meeting efficiency and productivity. It is also difficult to grasp participants' emotions during meetings and reflect their feedback in the progress of the meeting. These problems are particularly pronounced in workplaces such as factories, where effective meeting progress is essential.

[1662] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for sending an attendance confirmation notification, means for receiving and displaying a user's attendance response, means for sharing an agenda list, means for setting a time limit for each agenda, means for recording user comments in real time, means for prompting other participants during periods of silence, means for analyzing comments using natural language processing, means for comparing with a database of past meetings, means for displaying a notification of duplicate comments, means for generating a meeting summary and action points, means for sending a meeting summary and next steps, means for confirming the end of the meeting, means for analyzing emotions from user comments, means for accumulating and comparing emotional data, means for providing feedback according to the emotional state, means for displaying the emotional state, and means for operating a robot to support the progress of the meeting. This improves the efficiency of meetings, allows participants' emotional states to be grasped in real time, and appropriate feedback can be provided, enabling more constructive and effective meeting progress.

[1663] An "attendance confirmation notice" is a notice sent to all participants at the start of a conference to confirm the participants' intention to attend the conference.

[1664] The "user attendance response" is information that the user responds to upon receiving the attendance confirmation notice by pressing the attendance button or other methods, and this response is sent to the server.

[1665] An "agenda list" is a list of topics or themes to be discussed at a meeting that is shared to plan the meeting's progress.

[1666] A "time limit" is a time limit set for each agenda item, and is used to keep the meeting proceeding efficiently.

[1667] "User comments" refer to opinions and comments made by participants during a meeting, which are recorded in real time.

[1668] A "silent period" is a period of time in which no comments are made for a certain period of time or more, during which a means for encouraging other participants to give their opinions is activated.

[1669] "Natural language processing" is a technology that analyzes user statements and extracts important points and keywords, thereby helping to understand the content of the statements.

[1670] A "past meeting database" is a database that stores records of meetings that have taken place in the past and is used to match the content of a current meeting with the content of the previous meeting.

[1671] "Duplicate comment notification" is information that is sent to current speakers when similar comments are identified in past meetings, in order to prevent discussions on the same content.

[1672] A "meeting summary" is a summary of the results of the meeting discussion, including next steps and action points.

[1673] "Action points" are specific action plans or tasks that are derived from the results of the meeting, and clearly indicate the action plan after the meeting.

[1674] "Analyzing emotions" refers to a technique for identifying emotions contained in a user's speech and grasping their emotional state, and is implemented using an emotion engine.

[1675] "Providing feedback" means responding or reacting to the user based on the analyzed emotional state, which contributes to improving the progress of the meeting.

[1676] "Displaying emotional state" refers to visually displaying the analyzed emotional state of the user, allowing other participants to understand the situation.

[1677] "Operating a robot" means using a robot to assist in the progress of a meeting, improving the efficiency of the meeting and managing the emotions of participants.

[1678] The present invention is a system for improving productivity in meetings within a factory, which is combined with an emotion engine that recognizes the emotions of a user. The system of the present invention is configured as follows, and a specific embodiment thereof will be described.

[1679] Components

[1680] 1. Attendance confirmation notice

[1681] At the start of the conference, the server sends an attendance confirmation notification to all participants. Participants receive the notification on their devices and press the attendance button to send the information to the server.

[1682] 2. Share the agenda list

[1683] After confirming everyone's attendance, the server generates an agenda list for the meeting and displays it on each participant's device, allowing participants to understand the progress of the meeting.

[1684] 3. Set a time limit

[1685] A time limit is set for each topic, and the server monitors the progress. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to speak up.

[1686] 4. Recording and Analysis of Speech

[1687] When a user speaks, the content is recorded in real time by the device and sent to the server, which then analyzes the content using natural language processing (e.g., Python's Natural Language Toolkit (NLTK) or spaCy).

[1688] 5. Emotion Analysis

[1689] The server uses an emotion engine to analyze emotions from the content of comments, and the analysis results are accumulated in real time and compared with past emotion data.

[1690] 6. Providing Feedback

[1691] Depending on the analyzed emotional state, the server provides feedback to the participants. For example, if a user is feeling unhappy, the server notifies other participants and adjusts the progress of the meeting.

[1692] 7. Comparison with past meetings

[1693] The server compares the content of comments with a database of past meetings, and if there is any overlap, it notifies participants and makes past meeting records available for reference.

[1694] 8. Summary of the meeting and action points

[1695] Once all the agenda items have been completed, the server generates a meeting summary and action points and shares them with all participants.

[1696] 9. Confirmation of the end of the meeting

[1697] When the conference ends, the server confirms the end and, with the consent of all participants, the conference is officially ended.

[1698] Specific examples

[1699] The server performs the following series of operations:

[1700] Attendance confirmation notification: "Are you attending the meeting? Please press the attend button."

[1701] Sharing the agenda list: "Today's agenda list is as follows. Agenda 1: Improving production efficiency, Agenda 2: Safety measures"

[1702] Setting a time limit: "Next on the agenda is 'Improving Productivity'. We'll have 20 minutes."

[1703] Recording and analysis of speech: A user says, "I want to increase the speed of the production line," and the server records and analyzes this.

[1704] Analyzing emotions and providing feedback: "User A's sentiment was analyzed as 'high excitement'. The situation is good."

[1705] Cross-referencing: "This comment was also raised in a previous meeting. Would you like to review the previous meeting?"

[1706] Summary and action points: "The meeting is over. Summary: Specific action points were generated for 'improving production efficiency'. These have been sent via email."

[1707] Meeting End Confirmation: "The meeting has officially ended. Please press the confirmation button to end it."

[1708] Prompt Sentence Examples

[1709] "We're trying to come up with ideas to speed up the production line. How can we improve it while still maintaining safety?"

[1710] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1711] Step 1:

[1712] The server sends an attendance confirmation notification to all participants at the start of the conference. This notification is necessary to confirm attendance status, and participants who receive the notification press the attendance button on their terminal. The input is the instruction to send the attendance confirmation notification, and the output is the result of the notification being sent. Data processing involves generating a notification based on the participant list and sending it to each participant's terminal.

[1713] Step 2:

[1714] The user receives the attendance confirmation notification and presses the attendance button. This generates an attendance response, and the information is sent to the server. The input is the attendance confirmation notification, and the output is the attendance response. Data processing involves generating the attendance response by pressing the button and sending it to the server.

[1715] Step 3:

[1716] After the server confirms everyone's attendance, it generates an agenda list for the meeting and displays it on each participant's device. The input is the aggregated attendance responses, and the output is the displayed agenda list. Data processing involves creating an agenda list after attendance confirmation and distributing it to each participant.

[1717] Step 4:

[1718] The server sets a time limit for each agenda item and monitors the progress of the meeting while it is in progress. If no one speaks for a certain amount of time, the server sends a notification to other participants encouraging them to comment. The input is the agenda list, and the output is the time limit setting information and a notification encouraging comments. Data processing involves setting a time limit for each agenda item and sending notifications based on the elapsed time.

[1719] Step 5:

[1720] When a user speaks, the content of the speech is recorded in real time by the device and the data is sent to the server. The input is the user's speech and the output is the speech recording data. Data processing involves converting the speech into text data using voice recognition and sending that data to the server.

[1721] Step 6:

[1722] The server analyzes the received speech using natural language processing tools (e.g., Python's Natural Language Toolkit (NLTK) or spaCy). The input is the speech recording data, and the output is the analyzed keywords and key points. Data processing involves analyzing the text data and extracting important points.

[1723] Step 7:

[1724] The server then uses an emotion engine to analyze the analyzed speech content and identify the emotions contained in the speech. The input is the analyzed speech data, and the output is the emotion analysis results. Data processing involves using an emotion estimation algorithm (e.g., EmotionAnalyzer) to identify the emotions.

[1725] Step 8:

[1726] The server provides feedback to participants based on the analyzed emotional state. For example, if a user is feeling dissatisfied, the server notifies the other participants of this information and adjusts the progress of the meeting. The input is the emotion analysis result, and the output is a feedback notification. Data processing involves generating appropriate feedback based on the emotion analysis result and sending it to the necessary participants.

[1727] Step 9:

[1728] The server compares the content of comments with a database of past meetings and notifies participants if there is any overlap. The input is the analyzed comment data and the past meeting database, and the output is a notification of duplicate comments. Data processing involves matching with past meeting records.

[1729] Step 10:

[1730] Once all agenda items have been completed, the server generates a meeting summary and action points and shares them with all participants. The input is the meeting progress data, and the output is the summary and action points. Data processing involves summarizing the discussion and next steps and distributing them to participants.

[1731] Step 11:

[1732] When the meeting ends, the server confirms the end and, with the consent of all participants, the meeting officially ends. The input is the end confirmation instruction, and the output is the end confirmation result. Data processing involves executing the end confirmation process and collecting and displaying the results.

[1733] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1734] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1735] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1736] [Fourth embodiment]

[1737] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1738] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1739] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1740] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1741] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1742] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1743] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1744] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1745] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1746] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1747] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1748] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1749] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1750] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments of each function of the system will be described below.

[1751] Starting a meeting

[1752] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent via email or the notification function of the conference system. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[1753] Examples:

[1754] Server: "Attendance confirmation has been sent."

[1755] Terminal: "The user pressed the attend button. A response was sent to the server."

[1756] Server: "Everyone is here. The agenda is as follows..."

[1757] Progress of the agenda

[1758] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[1759] Examples:

[1760] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1761] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1762] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[1763] Speech analysis

[1764] The server analyzes what users say using natural language processing, extracting important points and keywords. These points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[1765] Examples:

[1766] Server: "Analyzing user C's comment."

[1767] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[1768] Server: "This comment was raised in a previous meeting. We will notify the users."

[1769] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[1770] Ending the meeting and sharing

[1771] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[1772] Examples:

[1773] Server: "All meeting agenda items have been completed. Generate summary and action points."

[1774] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1775] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1776] In this way, the system of the present invention improves the productivity of meetings by efficiently managing the entire flow from the start to the end of the meeting. The server, terminals, and users each fulfill their respective roles, and the system operates smoothly as a whole, allowing employees to effectively understand the content of the meeting and quickly return to their main tasks.

[1777] The processing flow will be explained below.

[1778] Step 1: Send attendance confirmations

[1779] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[1780] Step 2: Receiving attendance responses

[1781] Terminal: "When the user presses the attend button, the response is sent to the server."

[1782] Step 3: Complete attendance confirmation

[1783] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[1784] Step 4: Set a time limit for the agenda

[1785] Server: "Set a time limit for each topic and start the timer."

[1786] Step 5: Record what you say

[1787] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[1788] Step 6: Prompt for silence

[1789] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[1790] Step 7: Natural Language Analysis of Speech

[1791] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[1792] Step 8: Match with past meetings

[1793] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[1794] Step 9: Notification of duplicate posts

[1795] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[1796] Step 10: Generate a meeting summary

[1797] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[1798] Step 11: Submit your summary and action plan

[1799] Server: "Send an email to all participants with a meeting summary and next steps."

[1800] Step 12: Confirm the end of the meeting

[1801] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[1802] The above are the specific processing steps of the system. By executing each step in order, the meeting will proceed smoothly and effective decision-making will be possible.

[1803] Example 1

[1804] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1805] Corporate meetings often do not proceed effectively, resulting in wasted time, delays in moving forward with agenda items, and overlooking important comments. This can result in lower meeting productivity and delays in important decisions. It is also difficult to keep track of past meeting content and prevent new comments from overlapping. A system that solves these problems and improves meeting productivity is needed.

[1806] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1807] In this invention, the server includes means for sending attendance confirmation notices, means for receiving and displaying user attendance responses, means for sharing an agenda list, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants for their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for generating a meeting summary and action items, means for sending a meeting summary and next steps, means for confirming the end of the meeting, means for sending a meeting attendance confirmation notice via email or a notification function of the conference system, means for converting user comments into text using speech recognition technology, means for saving the recorded comments in real time in a memory area, means for notifying users of the comment analysis results, means for searching past meeting records for similar comments, and means for generating an agenda list in HTML format and displaying it on a terminal.This enables efficient meeting progress, prevents important comments from being overlooked, reduces duplicate comments, and clarifies summaries and action points.

[1808] 1. "Attendance confirmation notice" means a means of communication to confirm the attendance of meeting participants.

[1809] 2. "User attendance response" refers to a conference participant's indication of intent to attend in response to an attendance confirmation notice.

[1810] 3. An "agenda list" is a list of items or topics to be discussed at a meeting.

[1811] 4. "Time limit" means the specific time limit assigned to proceed with each agenda item.

[1812] 5. "Real-time recording" refers to recording what is said immediately on the spot.

[1813] 6. "Silent period" refers to a period of time during which no participant speaks.

[1814] 7. "Natural language processing" is a technology that uses computers to analyze human language and understand and process its meaning and syntax.

[1815] 8. "Past Meeting Database" means data storage that accumulates and manages records of previously held meetings.

[1816] 9. "Duplicate comment notification" is a function that notifies participants when a comment is made that is the same as something that has already been discussed.

[1817] 10. A "summary" is a summary of the overall content of the meeting.

[1818] 11. "Action items" are specific next steps or tasks determined as a result of a meeting.

[1819] 12. A "summary" is a concise summary of what was discussed and decided at a meeting.

[1820] 13. “Next steps” are specific actions or plans to be taken after the meeting.

[1821] 14. "Sending attendance confirmation notices by email or via the notification function of the conference system" refers to sending attendance confirmation notices using the notification function of an email service or online conference tool.

[1822] 15. "Speech recognition technology" is technology that converts speech into text.

[1823] 16. "Memory Area" means a computer storage area for storing data.

[1824] 17. "Analyzing the results of speech analysis" refers to analyzing the content of users' speech and evaluating and using the results.

[1825] 18. "Searching for similar statements" refers to searching for similar content or opinions in past meeting records.

[1826] 19. "Generating an agenda list in HTML format and displaying it on a device" means creating an agenda list in web page format and displaying it on a participant's device.

[1827] The present invention is a system for improving the productivity of corporate meetings, and specific embodiments thereof will be described below.

[1828] Starting a meeting

[1829] The server first sends attendance confirmation notifications to all participants. Attendance confirmation notifications are sent by email or using the conference system's notification function. Users who receive this notification press the attendance button to indicate their intention to attend. The terminal detects the user's attendance response and sends it to the server. After receiving all attendance responses, the server generates an agenda list and displays it on each participant's terminal.

[1830] Examples:

[1831] Server: "Everyone is here. Here's the agenda..."

[1832] Terminal: "The user pressed the attend button. A response was sent to the server."

[1833] Progress of the agenda

[1834] The server sets a time limit for each agenda item and monitors the progress of the agenda items. When a comment is made, the device records the content in real time and sends it to the server. If no comment is made for a certain period of time, the server sends a notification to other participants urging them to contribute. This process ensures the smooth flow of the meeting.

[1835] Examples:

[1836] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1837] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1838] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[1839] Speech analysis

[1840] The server uses natural language processing to analyze what users say. This analysis extracts important points and keywords from the comments. These points and keywords are then compared with a database of past meetings, and if similar comments are made, the system notifies the user.

[1841] Examples:

[1842] Server: "Analyzing user C's comment."

[1843] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[1844] Server: "This comment was raised in a previous meeting. We will notify the users."

[1845] Ending the meeting and sharing

[1846] After all the agenda items have been completed, the server generates a summary of the meeting and action points. This summary and action points are sent to all participants by email. The terminal displays a confirmation screen for the end of the meeting to the user, and the user can close the meeting by pressing the end button.

[1847] Examples:

[1848] Server: "All meeting agenda items have been completed. Generate summary and action points."

[1849] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[1850] Terminal: "The meeting has ended. Please press the confirmation button to end."

[1851] Hardware and software used

[1852] High-performance database servers are used as servers. Specifically, general-purpose database servers or cloud-based database services (such as Amazon RDS) are used. Laptops and tablets (such as regular laptops and tablets) are used as devices. Software uses the Google Workspace email API and Microsoft Teams notification function. The Google Cloud NLP API is used for natural language processing.

[1853] Prompt Sentence Examples

[1854] "Please describe in detail a system that improves the productivity of corporate meetings. Explain each step from the start to the end of a meeting from the perspective of the server, device, and user, including specific hardware and software."

[1855] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1856] Step 1:

[1857] The server sends attendance confirmations to all participants at the start of the conference.

[1858] Input: List of conference participants

[1859] Processing: The server uses the Google Workspace email API or the conferencing system's notification function to generate and send attendance confirmation notifications to all participants.

[1860] Output: Notification of completion of sending of attendance confirmation notice

[1861] Step 2:

[1862] The user receives a confirmation of attendance.

[1863] Input: Attendance confirmation notification from the server

[1864] Action: The user checks the notification or email they received and clicks the attend button.

[1865] Output: Click on the attendance button (attendance response)

[1866] Step 3:

[1867] The terminal sends the user's response to the server.

[1868] Input: User's attendance response (clicking the attendance button)

[1869] Processing: The terminal sends the user's response to the server via an HTTP POST request.

[1870] Output: Attendance response data sent to the server

[1871] Step 4:

[1872] The server verifies the attendance of all participants.

[1873] Input: Attendance response data from each participant

[1874] Processing: The server receives responses from all participants and checks whether everyone has confirmed their attendance. Reminder emails are sent again if necessary.

[1875] Output: Attendance confirmation completion notification

[1876] Step 5:

[1877] The server instructs each participant to display the agenda list on their terminal.

[1878] Input: Agenda list data

[1879] Processing: The server generates an agenda list and sends it to each terminal using WebSocket. It also configures it to be displayed in HTML format.

[1880] Output: Send and display agenda list

[1881] Step 6:

[1882] The server sets a time limit for each topic.

[1883] Input: Agenda list data

[1884] Processing: The server sets a timer using Python's time library at the start of each agenda item.

[1885] Output: Timer setting completion notification for each agenda item

[1886] Step 7:

[1887] The server monitors the progress of the agenda.

[1888] Input: Ongoing data for each agenda item

[1889] Processing: The server periodically checks the elapsed time of the agenda and evaluates the progress.

[1890] Output: Progress monitoring data

[1891] Step 8:

[1892] When a user speaks, the content is recorded in real time by the terminal.

[1893] Input: User's voice data

[1894] Processing: The device converts the voice data into text using voice recognition technology and sends it to the server in real time.

[1895] Output: Text data to the server

[1896] Step 9:

[1897] If no one speaks for a certain period of time, the server sends a notification to other participants urging them to speak up.

[1898] Input: Data for silent periods

[1899] Processing: When the silent period exceeds the set time, the server sends a notification to other participants using the Slack API.

[1900] Output: Notice of opinion promotion

[1901] Step 10:

[1902] The server analyzes the content of the user's comments.

[1903] Input: Text data sent to the server

[1904] Processing: The server uses Google Cloud NLP API to parse the text data and extract key points and keywords.

[1905] Output: Analysis results (important points and keywords)

[1906] Step 11:

[1907] The server matches the extracted data with past conference data.

[1908] Input: Analysis results (important points and keywords)

[1909] Processing: The server accesses a database of stored past meetings and uses the Cosine Similarity algorithm to search for similar statements.

[1910] Output: Search results for similar statements

[1911] Step 12:

[1912] The server notifies the user if there is a duplicate message.

[1913] Input: Search results for similar statements

[1914] Processing: When a duplicate message is detected, the server generates a notification and sends it to the relevant user. The device receives the notification and displays it to the user.

[1915] Output: Duplicate message notification

[1916] Step 13:

[1917] The server generates a summary and action points for the meeting.

[1918] Input: All data recorded at the meeting

[1919] Processing: The server summarizes all the agenda items and their conclusions of the meeting, automatically sets action points, saves them in Excel format and generates a summary.

[1920] Output: Meeting summary and action points

[1921] Step 14:

[1922] The server will email the meeting summary and action points to all participants.

[1923] Input: Meeting summary and action points

[1924] Processing: The server uses Google Workspace's email API to send the summary and action points to each participant in PDF format.

[1925] Output: Email with summary and action points

[1926] Step 15:

[1927] The terminal displays a confirmation screen for ending the conference to the user, prompting the user to press the end button.

[1928] Input: Conference end notification from the server

[1929] Processing: The terminal receives the notification of the end of the conference and displays the end confirmation screen. The user clicks the end button to confirm.

[1930] Output: User confirms end of meeting

[1931] By dividing the process into detailed steps in this way, the operation of the entire system becomes clear. These steps efficiently manage the entire flow from the start to the end of the conference.

[1932] (Application example 1)

[1933] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1934] The efficiency of conferences and meetings within factories is extremely important, but it is often difficult to advance productive discussions in an environment where multiple participants simultaneously exchange opinions. Furthermore, revisiting topics and opinions already discussed in previous meetings wastes time and reduces meeting productivity. Furthermore, summarizing and sharing action points after meetings is time-consuming, and there is a risk that important information may be overlooked. There is a need for a system that can solve these issues and improve the productivity and efficiency of meetings.

[1935] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1936] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording user comments in real time, means for prompting other participants to give their opinions during periods of silence, means for analyzing comments using natural language processing, means for comparing with a database of past meetings, means for displaying notifications of duplicate comments using a generative AI model, means for generating a meeting summary and action points, means for sending a meeting summary and next steps, means for confirming the end of the meeting, and means for generating prompt sentences to improve the efficiency of meetings within the factory. This makes it possible to improve the efficiency and productivity of factory meetings.

[1937] An "attendance confirmation notice" is a notice sent at the start of a conference or meeting to confirm the attendance of participants.

[1938] "User's attendance response" is response information sent by the user in response to the attendance confirmation notice.

[1939] An "agenda list" is a list of topics to be discussed at a conference or meeting.

[1940] A "time limit" is a time limit set for each agenda item.

[1941] "Real-time recording" means that what a user says is instantly saved in digital form.

[1942] A "silent period" is a period of time during a conference where no one speaks for a certain period of time or more.

[1943] A "notice encouraging opinions" is a notification sent to other participants during a silent period to encourage them to give their opinions.

[1944] "Natural language processing" is a technology that analyzes user statements and extracts important points and keywords.

[1945] "Past Meeting Database" means a database that aggregates and stores records of meetings that have been held in the past.

[1946] A "generative AI model" is a model that uses artificial intelligence in the generation process.

[1947] The "duplicate comment notification" is a notification that notifies the user of content similar to a comment that has already been mentioned in a past conference.

[1948] A "meeting summary" is a summary of what was discussed in the meeting and outlines next steps and action points.

[1949] "Action points" are specific next steps to take as a result of the meeting.

[1950] A "prompt sentence" is a specific instruction sentence for performing a specific task.

[1951] The present invention is a system for improving the efficiency of conferences and meetings in a factory. Specific embodiments of the system will be described below.

[1952] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. This notification is sent to the participant's device (smartphone, smart glasses, head-mounted display, etc.), and the participant responds by pressing the attendance button. This attendance information is immediately sent to the server, and the attendance of all participants is confirmed.

[1953] The server then shares the agenda list with all participants' devices, allowing each participant to prepare for the meeting. A time limit is also set for each agenda item, and the server monitors its progress. When a comment is made, the content is recorded in real time by the device and sent to the server. During periods of silence, the server sends notifications to other participants urging them to contribute, preventing the discussion from stagnating.

[1954] The server analyzes user comments using natural language processing technology. It uses libraries such as Spacy to extract key points and keywords. The extracted information is then compared with a database of past meetings to determine whether there are any overlapping comments. If overlapping comments are detected using a generative AI model, a notification is sent to the user.

[1955] When the meeting ends, the server generates a summary of the meeting and action points, and sends them to all participants. This summary includes the conclusions and next steps for each agenda item. A confirmation screen for the end of the meeting is also displayed on the terminal, and participants can press the confirmation button to end the meeting.

[1956] As a concrete example, the following prompts can be fed into a generative AI model to help guide a meeting:

[1957] Analyze the statements and extract key keywords. Here is an example statement:

[1958] "Do you have any comments on the progress of Project X?"

[1959] In this way, the system of the present invention can improve the efficiency and productivity of conferences and meetings within factories. The hardware used includes factory robots, smartphones, smart glasses, and head-mounted displays, and the software includes a server (Node.js server) and a natural language processing library (Spacy).

[1960] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1961] Step 1:

[1962] The server sends an attendance confirmation notification to all participants at the start of the conference. The inputs are the participant list and the start time of the conference. The attendance notification is sent and displayed on each participant's device.

[1963] Step 2:

[1964] The user responds by pressing the attendance button on the terminal. The input is the user's operation (clicking the attendance button) when the attendance confirmation notification is displayed. When the attendance button is pressed, the response information is sent to the server.

[1965] Step 3:

[1966] The server processes the received attendance responses and confirms the attendance of all participants. The input is the attendance response data sent by each participant. The attendance information is tallied, and once all attendance is confirmed, the server proceeds to the next step.

[1967] Step 4:

[1968] The server shares the agenda list with all participants' devices. The input is the meeting agenda list. The agenda list is displayed on each device and shared with the participants.

[1969] Step 5:

[1970] The server sets a time limit for each agenda item and monitors its progress. The inputs are the time limit and start time for each agenda item. The timer starts and the progress is monitored in real time.

[1971] Step 6:

[1972] Users comment on topics. The input is the content of the user's comments. The content of the comments is recorded in real time on the terminal, and the data is sent to the server.

[1973] Step 7:

[1974] The server sends notifications to other participants during periods of silence, encouraging them to contribute their opinions. An input is a period of no speech. When a period of silence is detected, a notification is sent to other participants encouraging them to contribute their opinions.

[1975] Step 8:

[1976] The server analyzes the received comments using natural language processing technology. The input is the user's comment data. Natural language processing (using the Spacy library) is performed to extract important keywords and key points.

[1977] Step 9:

[1978] The server compares the extracted keywords and points with a database of past conferences. The inputs are the extracted keywords and points and the database of past conferences. Database comparison is performed to detect duplicated comments.

[1979] Step 10:

[1980] The server uses the generative AI model to display a notification of duplicated comments to the user. The inputs are the matching results and the duplicate comment detection information. The detected duplicate comments are notified to the user.

[1981] Step 11:

[1982] After the meeting, the server generates a summary of the meeting and action points. The input is all speech data collected during the meeting and a summary of each agenda item. The summary and action points are automatically generated and sent to all participants by email.

[1983] Step 12:

[1984] The user presses the conference end confirmation button on the terminal. As input, there are buttons displayed on the end confirmation screen. When the end confirmation button is pressed, the server records the end of the conference and the conference is officially ended.

[1985] The above steps will improve the efficiency of meetings within the factory and increase productivity.

[1986] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1987] The present invention is a system for improving the productivity of corporate meetings, which is combined with an emotion engine that recognizes the emotions of users. Specific embodiments of each function of the system including the emotion engine will be described below.

[1988] Starting a meeting

[1989] First, at the start of the meeting, the server sends an attendance confirmation notification to all participants. When a user receives the notification and presses the attendance button, the response is sent to the server by the terminal. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on each participant's terminal.

[1990] Examples:

[1991] Server: "Attendance confirmation has been sent."

[1992] Terminal: "The user pressed the attend button. A response was sent to the server."

[1993] Server: "Everyone is here. The agenda is as follows..."

[1994] Progress of the agenda

[1995] Next, a time limit is set for each agenda item. The server sets a timer at the start of each agenda item and monitors its progress. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[1996] Examples:

[1997] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[1998] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[1999] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[2000] Speech analysis and emotion recognition

[2001] The server uses natural language processing to analyze what users say, extracting important points and keywords. At the same time, an emotion engine analyzes and records the emotions contained in the user's comments. The extracted points and keywords are then compared with a database of past meetings to identify similar comments. If any duplicate comments are found, the server notifies the user.

[2002] Examples:

[2003] Server: "Analyzing user C's comment."

[2004] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[2005] Server: "This comment was raised in a previous meeting. We will notify the users."

[2006] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[2007] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[2008] Emotional feedback

[2009] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. In addition, emotion data is accumulated and compared with past emotion data, allowing changes in emotions during a meeting to be tracked.

[2010] Examples:

[2011] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[2012] Server: "Send feedback on dissatisfaction."

[2013] Device: "The user has notified other participants that they are unhappy."

[2014] Ending the meeting and sharing

[2015] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this summary and action points to all participants. At the same time, the terminal displays a confirmation screen for the end of the meeting to the user, prompting them to press the end button.

[2016] Examples:

[2017] Server: "All meeting agenda items have been completed. Generate summary and action points."

[2018] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[2019] Terminal: "The meeting has ended. Please press the confirmation button to end."

[2020] As described above, the system of the present invention effectively manages a meeting from start to finish through a series of flows, and by adding an emotion engine, it also takes into account emotional aspects during the meeting, thereby realizing smoother and more constructive meetings.

[2021] The processing flow will be explained below.

[2022] Step 1: Send attendance confirmations

[2023] Server: "When the meeting starts, send attendance confirmation notifications to all participants."

[2024] Step 2: Receiving attendance responses

[2025] Terminal: "When the user presses the attend button, the response is sent to the server."

[2026] Step 3: Complete attendance confirmation

[2027] Server: "Once everyone is present, share the meeting agenda and topic list to all devices."

[2028] Step 4: Set a time limit for the agenda

[2029] Server: "Set a time limit for each topic and start the timer."

[2030] Step 5: Record what you say

[2031] Terminal: "When a user speaks, the content is recorded as text and sent to the server."

[2032] Step 6: Prompt for silence

[2033] Server: "If no one speaks for a certain amount of time, send a notification to randomly selected other participants asking them to speak up."

[2034] Step 7: Natural Language Analysis of Speech

[2035] Server: "Analyzes each user's comments in real time and extracts important points and keywords."

[2036] Step 8: Sentiment Analysis

[2037] Emotion engine: "Based on the content of the user's speech, analyzes the emotions at the time of the speech and generates emotional data."

[2038] Step 9: Provide emotional feedback

[2039] Emotion engine: "Provides appropriate feedback to users in real time based on analyzed emotional data."

[2040] Step 10: Match with past meetings

[2041] Server: "Compare the extracted points and keywords with past meeting data to identify overlapping and similar points."

[2042] Step 11: Notification of duplicate posts

[2043] Terminal: "If duplicate or similar comments are identified, notify the speaker."

[2044] Step 12: Generate a meeting summary

[2045] Server: "Once all agenda items are completed, generate a meeting summary and action points."

[2046] Step 13: Submit your summary and action plan

[2047] Server: "Send an email to all participants with a meeting summary and next steps."

[2048] Step 14: Confirm the end of the meeting

[2049] Terminal: "Display a confirmation screen to the user to end the conference and prompt them to press the end button."

[2050] Examples:

[2051] The flow from the start to the end of a meeting is shown below.

[2052] Step 1:

[2053] Server: "Attendance confirmation has been sent."

[2054] Step 2:

[2055] Terminal: "The user pressed the attend button. A response was sent to the server."

[2056] Step 3:

[2057] Server: "Everyone is here. The agenda is as follows..."

[2058] Step 4:

[2059] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[2060] Step 5:

[2061] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[2062] Step 6:

[2063] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[2064] Step 7:

[2065] Server: "Analyzing User C's comments. Key points identified."

[2066] Step 8:

[2067] Emotion Engine: "Analyzing emotions from user C's comments. The result is 'Go-heung'."

[2068] Step 9:

[2069] Emotion Engine: "Provide positive feedback because the user is in a 'high-exciting' state."

[2070] Step 10:

[2071] Server: "I checked with past meetings. Similar opinions were expressed in previous meetings."

[2072] Step 11:

[2073] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[2074] Step 12:

[2075] Server: "All meeting agenda items have been completed. Generate summary and action points."

[2076] Step 13:

[2077] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[2078] Step 14:

[2079] Terminal: "The meeting has ended. Please press the confirmation button to end."

[2080] These are the specific processing steps of the system that includes the emotion engine. By executing each step in order, the meeting will proceed effectively and participants will be emotionally satisfied.

[2081] Example 2

[2082] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2083] Corporate meetings often have a tendency for certain participants to speak, resulting in meetings that don't proceed smoothly or don't end on time. Furthermore, participants' feelings and intentions are not properly shared during the meeting, which can lead to misunderstandings and frustration. Furthermore, many topics and comments overlap with previously discussed content, making it difficult to hold efficient discussions. There is a need to solve these problems and improve meeting productivity.

[2084] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2085] In this invention, the server includes means for sending attendance confirmation notifications, means for receiving and displaying user attendance responses, means for sharing a list of agenda items, means for setting a time limit for each agenda item, means for recording and transmitting user comments in real time, means for encouraging other participants to contribute during periods of silence, means for analyzing comments using natural language processing and extracting important points and keywords, means for comparing comments with a database of past meetings, means for displaying notifications of duplicate comments, means for analyzing and recording emotions contained in comments using an emotion engine, means for providing feedback based on emotion data, means for generating a meeting summary and action points, means for transmitting a meeting summary and next steps, and means for confirming the end of the meeting. This allows for the equal invitation of all participants' opinions, efficient analysis of the content of comments, and progress of the meeting while linking them with past data. Furthermore, by understanding emotion data in real time and taking appropriate measures, smoother and more productive meeting management can be achieved.

[2086] "Means for sending attendance confirmation notices" refers to a device or system for sending attendance confirmation notices to all participants at the start of a conference.

[2087] The "means for receiving and displaying a user's attendance response" refers to a device or system for receiving information in response to a user's attendance notification and displaying the response.

[2088] The "means for sharing an agenda list" is a device or system for sharing the meeting agenda in list form with all participants.

[2089] The "means for setting a time limit for each agenda item" is a device or system for setting a time limit for the discussion of each agenda item and managing the progress thereof.

[2090] The "means for recording and transmitting user comments in real time" refers to a device or system for recording user comments in real time and transmitting them to a server.

[2091] The "means for encouraging other participants to give their opinions during silent periods" refers to a device or system for encouraging other participants to give their opinions when no one has spoken for a certain period of time.

[2092] "Means for analyzing comments using natural language processing and extracting important points and keywords" refers to a device or system that uses natural language processing technology to analyze the content of user comments and extract important points and keywords.

[2093] The "means for checking against a database of past meetings" refers to a device or system for checking the content of current statements against a database of past meetings to identify duplicate or similar statements.

[2094] The "means for displaying a notification of overlapping comments" is a device or system for displaying a notification to the user when overlapping comments are found as a result of comparison with past conferences.

[2095] The "means for analyzing and recording emotions contained in comments using an emotion engine" refers to a device or system for analyzing a user's emotions based on the content of comments and recording the results.

[2096] The "means for providing feedback based on emotional data" is a device or system for providing appropriate feedback or notifications to a user based on emotional data analyzed in real time.

[2097] The "means for summarizing a meeting and generating action points" is a device or system for summarizing the content of the discussion after the meeting and generating the next action steps.

[2098] The "Means for transmitting meeting summary and next steps" is a device or system for transmitting a meeting summary and future action points to all participants.

[2099] The "means for confirming the end of the conference" is a device or system for allowing participants to confirm the end of the conference.

[2100] This invention is a system for improving the productivity of corporate meetings, comprehensively managing meetings from start to finish, and utilizing an emotion engine to grasp the user's emotional state in real time and provide feedback. Specific embodiments are described below.

[2101] Hardware and Software Configuration

[2102] The server is equipped with a database server, a web server, a natural language processing engine, and a sentiment analysis engine. These engines can also use external APIs. For example, the system uses the Google Cloud Natural Language API for natural language processing and the IBM Watson Tone Analyzer for sentiment analysis. This enables advanced text analysis and sentiment analysis.

[2103] A terminal is a device (such as a PC, tablet, or smartphone) used by a participant, and provides an interface for sending and receiving information through communication with the server and for displaying and inputting information.

[2104] Starting a meeting

[2105] At the start of a meeting, the server sends an attendance confirmation notification to all participants. Specifically, it retrieves participants' email addresses and device information from a database and sends them all emails or push notifications (using SendGrid, for example). When a user receives the notification and presses the "attend" button, the response is sent from the device to the server. After the server confirms everyone's attendance, it shares the meeting agenda list and displays it on the device.

[2106] Examples:

[2107] Server: "Attendance confirmation has been sent."

[2108] Terminal: "The user pressed the attend button. A response was sent to the server."

[2109] Server: "Everyone is here. The agenda is as follows..."

[2110] Progress of the agenda

[2111] The server sets a time limit for each topic and monitors its progress. A timer is set at the start and a warning is issued before the time limit is exceeded, ensuring smooth progress in the discussion. When a user speaks, the content is recorded in real time by the device and sent to the server. If no one speaks for a certain period of time, the server sends a notification to other participants urging them to contribute.

[2112] Examples:

[2113] Server: "Next on the agenda is 'Project X Progress'. We'll be running it for 20 minutes."

[2114] Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[2115] Server: "10 minutes left. No new suggestions, so I'll ask User B."

[2116] Speech analysis and emotion recognition

[2117] The server analyzes the user's comments using natural language processing. Specifically, it uses the Google Cloud Natural Language API to analyze the text data and extract important points and keywords. At the same time, the emotion engine uses IBM Watson Tone Analyzer to analyze the sentiment of the comments and record the results. The comments and the extracted points and keywords are compared with a database of past meetings, and if similar comments are found, the user is notified that they are duplicates.

[2118] Examples:

[2119] Server: "Analyzing user C's comment."

[2120] Server: "We've identified the key points you mentioned. We'll compare them with past meetings."

[2121] Server: "This comment was raised in a previous meeting. We will notify the users."

[2122] Terminal: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[2123] Server: "Analyzing sentiment from user's statement. Result is 'High Hing'."

[2124] Emotional feedback

[2125] The emotion engine grasps the user's emotional state in real time and provides feedback according to the situation. For example, if the user is feeling dissatisfied, the server will take appropriate action based on that information. Emotion data is accumulated and compared with past emotion data to track changes in emotions during the meeting and consider how to respond if necessary.

[2126] Examples:

[2127] Server: "The emotion analyzed from User D's comment is 'dissatisfied'."

[2128] Server: "Send feedback on dissatisfaction."

[2129] Device: "The user has notified other participants that they are unhappy."

[2130] Ending the meeting and sharing

[2131] Once all agenda items have been completed, the server generates a meeting summary and action points. This summary includes the conclusions and next steps for each agenda item. The server then emails this to all participants (using SendGrid, for example), and the terminal displays a meeting end confirmation screen to the user. The user presses the end button to officially end the meeting.

[2132] Examples:

[2133] Server: "All meeting agenda items have been completed. Generate summary and action points."

[2134] Server: "I've sent you an email with a summary of the meeting and upcoming actions."

[2135] Terminal: "The meeting has ended. Please press the confirmation button to end."

[2136] This system efficiently manages the progress of the agenda and analyzes comments, and also grasps the user's emotional state in real time, making for smoother and more constructive meetings.

[2137] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2138] Step 1: Send attendance confirmations

[2139] Specifically, when the meeting starts, the server retrieves the participants' email addresses and device information from the database. Based on the retrieved information, it uses an email sending API (e.g., SendGrid) to send attendance confirmation notifications to all participants.

[2140] Input: Participant's email address and device information obtained from the database.

[2141] Data processing: Convert email addresses and device information into a format suitable for the API.

[2142] Data output: Attendance confirmation notification sending result (success / failure).

[2143] Specific behavior:

[2144] Server: "Attendance confirmation has been sent."

[2145] Step 2: User's attendance response

[2146] When a user presses the attendance button on the terminal, a response is sent from the terminal to the server, which receives it and updates the attendance information in the database.

[2147] Input: User presses the Attend button.

[2148] Data processing: The server receives and analyzes the user's attendance response.

[2149] Data output: Update attendance information to database.

[2150] Specific behavior:

[2151] - Terminal: "The user pressed the attend button. A response was sent to the server."

[2152] - Server: "Attendance information has been updated to the database."

[2153] Step 3: Share the agenda list

[2154] After the server confirms that everyone is present, it shares the meeting agenda list with the terminals. The server retrieves the agenda list from the database and sends it to each terminal.

[2155] Input: A list of topics retrieved by the server from the database.

[2156] Data processing: The acquired agenda list is organized into a format suitable for the device.

[2157] Data output: Send the agenda list to each device.

[2158] Specific behavior:

[2159] - Server: "Agenda list shared."

[2160] Step 4: Set time limits for each agenda item

[2161] During the meeting, the server sets a time limit for each agenda item, runs a timer, and monitors the progress.

[2162] Input: Notifications at the start of each agenda item.

[2163] Data processing: setting time limits and starting timers.

[2164] Data output: Time management data (such as remaining time notification).

[2165] Specific behavior:

[2166] - Server: "Next on the agenda is 'Progress on Project X'. We'll be running it for 20 minutes."

[2167] Step 5: Record and send user comments

[2168] When a user speaks, the content is recorded in real time by the terminal and transmitted to the server.

[2169] Input: What the user says.

[2170] Data processing: Recorded as text data in real time.

[2171] Data output: Sends the statement to the server.

[2172] Specific behavior:

[2173] - Terminal: "User A has expressed his opinion. The comment has been sent to the server."

[2174] Step 6: Encourage feedback during silent periods

[2175] If no comments are made for a certain period of time, the server sends a notification to other participants prompting them to comment.

[2176] Input: Whether or not the speech was recorded and the time.

[2177] Data processing: Analyze periods of silence.

[2178] Data output: Notifications encouraging other participants to give their opinions.

[2179] Specific behavior:

[2180] - Server: "There are no new opinions, so I'll ask User B."

[2181] Step 7: Natural Language Processing Analysis of Speech

[2182] The server analyzes the acquired speech using natural language processing (NLP) technology and extracts important points and keywords.

[2183] Input: What the user says.

[2184] Data processing: Analyze using a natural language processing engine (e.g., Google Cloud Natural Language API).

[2185] Data output: Key points and keywords.

[2186] Specific behavior:

[2187] - Server: "Analyzing what you said. Identifying key points."

[2188] Step 8: Sentiment Analysis

[2189] The server uses an emotion engine to analyze the emotions contained in the user's comments and record the results.

[2190] Input: What the user says.

[2191] Data processing: Analyze using a sentiment analysis engine (e.g., IBM Watson Tone Analyzer).

[2192] Data output: Sentiment analysis results.

[2193] Specific behavior:

[2194] - Server: "Analyzing emotions from user comments."

[2195] Step 9: Notification of duplicate posts

[2196] The server compares the content of comments with a database of past conferences and notifies the user if there are any duplicate comments.

[2197] Input: Analyzed speech content and historical database.

[2198] Data processing: database matching.

[2199] Data output: Duplicate comment notification.

[2200] Specific behavior:

[2201] - Server: "This comment was raised in a previous meeting. We will notify the user."

[2202] - Device: "This topic was discussed in a previous meeting. Would you like to review the previous meeting content?"

[2203] Step 10: Provide emotional feedback

[2204] Based on the analysis results, the server provides feedback according to the user's emotional state and also sends notifications to take appropriate action.

[2205] Input: Sentiment analysis results.

[2206] Data processing: Generating feedback content.

[2207] Data output: Sending feedback notifications.

[2208] Specific behavior:

[2209] - Server: "Send feedback on dissatisfaction."

[2210] - Device: "The user has notified other participants that they are unhappy."

[2211] Step 11: Meeting summary and action points generation

[2212] Once all the agenda items have been completed, the server generates a meeting summary and action points and sends this to all participants by email.

[2213] Input: Agenda progress data and discussion content.

[2214] Data processing: summary and generation of action points.

[2215] Data output: sent by email.

[2216] Specific behavior:

[2217] - Server: "All meeting agenda items have been completed. Generate summary and action points."

[2218] - Server: "I've emailed you a summary of the meeting and upcoming actions."

[2219] Step 12: Confirm the end of the meeting

[2220] Finally, the terminal displays a confirmation screen for the user to end the conference and prompts the user to press the end button. When the user presses the end button, the conference is officially ended.

[2221] Input: End of meeting notification.

[2222] Data processing: Display of confirmation screen for completion.

[2223] Data output: Response confirming completion.

[2224] Specific behavior:

[2225] - Device: "The meeting has ended. Please press the confirmation button to end it."

[2226] These detailed steps streamline the overall operation and management of meetings and enable real-time analysis of user comments and emotions.

[2227] (Application example 2)

[2228] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the ro...

Claims

1. means for sending attendance confirmation notices; means for receiving and displaying a user's attendance response; A means of sharing agenda lists; A means to set time limits for each agenda item, a means for recording user utterances in real time; A means of encouraging other participants to give their opinions during silent periods; A means of analyzing statements using natural language processing, a means of checking against a database of past meetings; a means for displaying a notification of duplicate comments; a means of generating meeting summaries and action points; a means of sending meeting summaries and next steps; A system including a means for confirming the end of a meeting.

2. 2. The system according to claim 1, wherein the means for analyzing comments using natural language processing has a function for extracting important points for each topic.

3. 2. The system according to claim 1, wherein the means for checking against a database of past meetings has a function of identifying similar statements and making it possible to refer to corresponding records of past meetings.

Images