System

Abstract

A system is provided.SOLUTION: The system includes a generation means for receiving information on the theme and purpose of a conference and automatically generating a draft agenda of the conference based on the information, a generation means for collecting schedule information of prospective participants in the conference and generating an optimal meeting holding schedule candidate, a generation means for generating a speech comment draft for a facilitator in real time during the conference and proposing the draft, a generation means for collecting speech data of the conference and automatically generating minutes, and a generation means for generating a mail delivery draft to conference participants based on the generated minutes.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Preparing and running a meeting requires a lot of time and effort, and there is a need to improve its efficiency. In particular, creating a meeting agenda, coordinating participants' schedules, managing comments made during the meeting, and creating and distributing meeting minutes consumes a lot of resources. There is a need for a method to solve these issues and improve the efficiency and quality of meeting management. [Means for solving the problem]

[0005] The present invention solves the above-mentioned problems by providing a system that uses generative AI to streamline meeting management. Specifically, the system includes a generator that receives information about the meeting's theme and purpose and automatically generates a meeting agenda based on that information, a generator that collects schedule information from potential meeting participants and generates optimal meeting date options, a generator that generates and proposes speech comments to the facilitator in real time during the meeting, a generator that collects speech data from the meeting and automatically generates minutes, and a generator that generates email proposals to be sent to meeting participants based on the generated minutes. This improves the efficiency of pre-meeting preparations, invigorating discussion during the meeting, and creating and distributing meeting minutes after the meeting.

[0006] A "generator" is a process or device used to generate a result based on particular inputs.

[0007] The "conference theme" is the central topic or subject of discussion at the conference.

[0008] "Purpose" is the goal or intention that you want to achieve through the meeting.

[0009] An "agenda" is a list of topics or issues to be discussed at a meeting and is used to plan the progress of the meeting.

[0010] "Schedule information" is data on other plans already scheduled by the conference participants and free time.

[0011] "Meeting date candidates" is a list of possible dates and times for holding a meeting.

[0012] "Suggested comments" are suggestions or advice for what the facilitator or participants should say during the meeting.

[0013] "Speech data" is a record of what participants said and discussed during a meeting.

[0014] Minutes are a document that organizes and records what was said and decided during a meeting.

[0015] The "email distribution proposal" is a draft of an email to be sent to the meeting participants based on the generated minutes. [Brief explanation of the drawings]

[0016] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0017] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0018] First, the terms used in the following description will be explained.

[0019] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0020] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0021] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0022] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0024] [First embodiment]

[0025] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0026] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0029] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0032] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0036] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0037] The present invention is a system that utilizes generative AI to improve the efficiency of meeting management, and its program and processing will be explained based on specific procedures.

[0038] 1. Automatic generation of agenda proposals

[0039] User:

[0040] The user inputs the meeting topic, purpose, relevant information, etc. into the system.

[0041] server:

[0042] The server receives this information and sends it to an internal generation AI, which generates a draft agenda based on this data.

[0043] Terminal (user):

[0044] The generated agenda plan is provided to the user, who can then modify and confirm it as necessary.

[0045] Examples:

[0046] If a user types in "kickoff meeting for a new project," the server uses generation AI to generate a draft agenda, including "1. Project overview," "2. Setting main goals and timeline," and "3. Confirming the roles and responsibilities of each person involved," and displays it to the user.

[0047] 2. Providing possible meeting dates

[0048] User:

[0049] The user inputs a list of members to attend the meeting.

[0050] server:

[0051] The server collects each participant's schedule information from the company's calendar system and uses generation AI to create optimal meeting date options.

[0052] Terminal (user):

[0053] The generated schedule candidates are presented to the user, who then selects and confirms the final schedule.

[0054] Examples:

[0055] Based on the schedules of participants A, B, and C, the generation AI proposes possible dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00" and displays them to the user.

[0056] 3. Generate comments for the facilitator and a summary of the meeting content

[0057] server:

[0058] Once the conference begins, the server analyzes the conference's voice input and chat log in real time.

[0059] Terminal (user):

[0060] The server provides the facilitator with real-time suggestions for comments and discussions, and also summarizes the meeting content and displays it to the facilitator.

[0061] Examples:

[0062] During the meeting, the server displays suggested comments on the device, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[0063] 4. Automatic minutes creation

[0064] server:

[0065] After the meeting ends, the server collects all audio data and chat logs, and automatically generates minutes using a generation AI.

[0066] Terminal (user):

[0067] The generated minutes are provided to the user, who can then review and edit the contents.

[0068] Examples:

[0069] After the meeting, the generative AI generates minutes summarizing the main points and decisions of the meeting, and presents the user with a document stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later."

[0070] 5. Create a draft of email distribution of meeting minutes

[0071] server:

[0072] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[0073] Terminal (user):

[0074] The user checks and edits the email content, and then sends the final email.

[0075] Examples:

[0076] The server creates a draft email containing the minutes of the meeting with the subject "We will share the minutes of the meeting as follows" and displays it to the user. After the user confirms it, the generation AI sends the email to all participants.

[0077] As described above, this invention is a system that utilizes generative AI to provide integrated support for meeting preparation, progress, discussion stimulation, and the creation and distribution of meeting minutes. This system significantly improves the efficiency of meeting management, saves resources, and improves meeting quality.

[0078] The processing flow will be explained below.

[0079] Automatic generation of agenda proposals

[0080] Step 1:

[0081] The user enters the meeting topic, purpose, and related information.

[0082] Step 2:

[0083] The server receives input information from the user and sends it to the generation AI module.

[0084] Step 3:

[0085] The generation AI on the server generates a draft agenda based on the input information.

[0086] Step 4:

[0087] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[0088] Providing possible meeting dates

[0089] Step 1:

[0090] The user enters a list of conference participants.

[0091] Step 2:

[0092] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[0093] Step 3:

[0094] The server uses a generation AI to generate optimal meeting date options based on available time.

[0095] Step 4:

[0096] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[0097] Generates comments for the facilitator and a summary of the meeting content

[0098] Step 1:

[0099] When the meeting starts, the server collects the meeting's voice input and chat log in real time.

[0100] Step 2:

[0101] The server uses generative AI to create suggested comments for the facilitator and suggestions for how to proceed with the discussion.

[0102] Step 3:

[0103] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[0104] Automatic minutes creation

[0105] Step 1:

[0106] After the meeting ends, the server collects all audio data and chat logs.

[0107] Step 2:

[0108] The server sends the collected data to the generation AI, which automatically generates minutes.

[0109] Step 3:

[0110] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[0111] Creating a draft for email distribution of meeting minutes

[0112] Step 1:

[0113] The server creates a proposal for sending emails to the meeting participants based on the generated minutes.

[0114] Step 2:

[0115] The server generates a draft email and provides it to the user.

[0116] Step 3:

[0117] The user checks the email content and corrects it if necessary.

[0118] Step 4:

[0119] After the user's confirmation, the server delivers the final email to the conference participants.

[0120] Example 1

[0121] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0122] Preparing for, conducting, and post-conference administrative tasks require time and effort, and efficiency is especially important for large-scale meetings. Furthermore, manual tasks such as scheduling and creating minutes can easily lead to errors, potentially lowering the quality of the meeting. Given this current situation, it is necessary to utilize generative AI models to centrally manage meeting operations and improve efficiency.

[0123] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0124] In this invention, the server includes means for receiving information about the theme and purpose of the meeting and automatically generating a meeting agenda based on this information, means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, and means for generating and proposing speech and comment proposals for the facilitator in real time during the meeting. This allows for centralized management of meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving efficiency.

[0125] "Information regarding the theme and purpose of the meeting" is information necessary to identify the content of the meeting, and mainly includes the theme, purpose, and related information of the meeting.

[0126] "Schedule information" is information indicating the schedule of people who plan to attend a conference, and refers to data acquired from a calendar system or the like.

[0127] A "generative AI model" is an artificial intelligence model that generates draft agendas, draft remarks, minutes, etc. based on information about the theme and purpose of a meeting.

[0128] "Minutes" are documents that record the contents of a meeting, statements made, decisions made, etc., and are automatically generated.

[0129] The "email distribution proposal" is a proposal for the content of emails to be sent to the meeting participants, which is created based on the generated minutes.

[0130] "Biometric authentication" is a technology that uses biological characteristics such as fingerprint authentication and facial authentication to verify a user's identity.

[0131] A "user" refers to a person who operates the system and whose role is to input the theme and purpose of the meeting, a list of participants, and so on.

[0132] A "facilitator" is a person whose role is to proceed and manage the meeting, and who guides the discussion by referring to suggestions and summaries from the system.

[0133] A "prompt sentence" is an input sentence given to a generative AI model, which contains instructions for the model to generate a specific output.

[0134] This invention is a system that utilizes generative AI models to improve the efficiency of meeting management. The program of this system is mainly composed of a server, terminals, and users, each of which plays a specific role.

[0135] Automatic generation of agenda proposals

[0136] The user inputs the meeting theme, purpose, and related information into the device. The server receives this information and sends it to a generation AI (e.g., OpenAI's GPT-4 (registered trademark)). The generation AI generates a draft agenda based on this data. The generated draft agenda is again sent to the device via the server and provided to the user. For example, if the user inputs "kickoff meeting for a new project," the generation AI will generate a draft agenda including "1. Overview of the project," "2. Setting of main goals and timeline," and "3. Confirmation of roles and responsibilities of each person involved," and display it to the user.

[0137] Example prompt sentence:

[0138] Create an agenda for the kickoff meeting for a new project.

[0139] Providing possible meeting dates

[0140] The user inputs a list of members participating in the meeting into the terminal. The server collects schedule information for each participant from the company's calendar system (e.g., Google® Calendar, Microsoft® Outlook). The collected schedule information is sent to the generation AI, which generates optimal meeting date candidates. These date candidates are presented to the user via the server, and the user selects and confirms the final date. For example, based on the schedules of participants A, B, and C, the generation AI proposes candidate dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00."

[0141] Example prompt sentence:

[0142] Please suggest the best possible meeting date based on the schedules of participants A, B, and C.

[0143] Generates comments for the facilitator and a summary of the meeting content

[0144] When a meeting begins, the device collects the meeting's voice input and chat log in real time and sends it to the server. The server analyzes this data in real time and sends it to a generation AI to generate suggested comments and a summary of the meeting content. The generated suggested comments and summaries are displayed on the device via the server and used by the facilitator to guide the meeting. For example, during a meeting, the server displays suggested comments on the device, such as "It's time to move on to the next agenda item" or "We should ask for other opinions on this agenda item."

[0145] Example prompt sentence:

[0146] Analyze meeting audio data and chat logs in real time and provide suggested comments for the facilitator.

[0147] Automatic minutes creation

[0148] When the meeting ends, the server collects all audio data and chat logs and sends them to the generation AI, which automatically generates minutes. The generated minutes are then sent to the device via the server, where the user can review and edit the contents. For example, after a meeting, the generation AI generates minutes stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later," and presents them to the user.

[0149] Example prompt sentence:

[0150] Automatically generate meeting minutes based on meeting audio data and chat logs.

[0151] Creating a draft for email distribution of meeting minutes

[0152] Based on the generated minutes, the server sends a prompt to the generation AI to generate a proposed email to send to the meeting participants. This proposed email is sent to the terminal via the server, where the user can confirm and modify it, and then finally send the email. For example, the server creates a proposed email with the subject "We will share the minutes of the meeting as follows" and displays it to the user.

[0153] Example prompt sentence:

[0154] Please create a proposal for emailing meeting minutes in the following format.

[0155] This system utilizes generative AI models to centrally manage and efficiently support meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving the overall efficiency and quality of meeting management.

[0156] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0157] Automatic generation of agenda proposals

[0158] Step 1:

[0159] The user inputs the theme, purpose, and related information of the meeting into an input form on the terminal. In this example, the user inputs information on the theme "kick-off meeting for a new project."

[0160] Input: Meeting topic, objectives, relevant information

[0161] Output: Initial input data sent from the device to the server

[0162] Step 2:

[0163] The server receives the input data and sends it to the generation AI, which then formats the input data appropriately as a prompt and provides it to the generation AI.

[0164] Input: Initial input data from the user

[0165] Output: Formatted prompt to send to the generation AI

[0166] Step 3:

[0167] The AI ​​generates a draft agenda based on the prompts, such as "1. Overview of the project" and "2. Setting main goals and timelines."

[0168] Input: Server formatted prompt

[0169] Output: The generated agenda proposal

[0170] Step 4:

[0171] The server receives the output from the generation AI and sends the proposed agenda to the terminal.

[0172] Input: Agenda proposal from the generative AI

[0173] Output: Draft agenda displayed on terminal

[0174] Step 5:

[0175] The user checks the agenda proposal displayed on the device and makes corrections or additions as necessary, for example, adding new items or modifying existing items.

[0176] Input: Generated agenda proposal

[0177] Output: Revised and confirmed agenda proposal

[0178] Providing possible meeting dates

[0179] Step 1:

[0180] The user inputs the list of participants in the conference into the terminal. For example, the user inputs the names and email addresses of "Participants A, B, and C."

[0181] Input: Participant list

[0182] Output: Participant data sent from the device to the server

[0183] Step 2:

[0184] The server accesses the company's calendar system (e.g., Google Calendar, Microsoft Outlook) and collects schedule information for the specified participants.

[0185] Input: Participant data

[0186] Output: Collected schedule information

[0187] Step 3:

[0188] The server sends the schedule information as a prompt to the generation AI, which then generates optimal meeting date candidates. For example, it uses the prompt "Tell me a date when everyone is free."

[0189] Input: Collected schedule information

[0190] Output: Generated itinerary candidates

[0191] Step 4:

[0192] The server receives the output of the generated AI and sends the proposed dates to the terminal.

[0193] Input: Generated date candidates

[0194] Output: Date options displayed on the terminal

[0195] Step 5:

[0196] The user checks the schedule options displayed on the device and selects and confirms the final date.

[0197] Input: Generated date candidates

[0198] Output: Confirmed meeting dates

[0199] Generates comments for the facilitator and a summary of the meeting content

[0200] Step 1:

[0201] When a meeting starts, the device collects the meeting's voice input and chat log in real time and sends them to the server, where the voice data is converted into text using automatic speech recognition (ASR) technology.

[0202] Input: Conference audio input, chat log

[0203] Output: Text data sent to the server

[0204] Step 2:

[0205] The server analyzes the received data in real time and sends it to the generating AI.

[0206] Input: Voice-to-text data, chat logs

[0207] Output: The prompt sent to the generation AI

[0208] Step 3:

[0209] The generative AI generates suggested comments and summaries of meeting content, such as "It's time to move on to the next agenda item" or "We should seek other opinions on this topic."

[0210] Input: The prompt sent to the generation AI

[0211] Output: Generated comment proposals and a summary of the meeting content

[0212] Step 4:

[0213] The server receives the output of the generated AI and sends proposed comments and meeting summaries to the device.

[0214] Input: Generated speech and comment proposals, summary of the meeting content

[0215] Output: Proposed comments and summary displayed on terminal

[0216] Step 5:

[0217] The facilitator checks the proposed comments and meeting summary displayed on the terminal and leads the discussion.

[0218] Input: Generated speech and comment proposals, summary of the meeting content

[0219] Output: Moderated meeting

[0220] Automatic minutes creation

[0221] Step 1:

[0222] When the meeting ends, the server collects all audio data and chat logs.

[0223] Input: Audio data of completed meetings, chat logs

[0224] Output: Data stored on the server

[0225] Step 2:

[0226] The server sends the collected data to the generation AI and requests it to generate minutes.

[0227] Input: Accumulated meeting data

[0228] Output: The prompt sent to the generation AI

[0229] Step 3:

[0230] The AI ​​automatically generates minutes based on the provided data, including key points and decisions.

[0231] Input: The prompt sent to the generation AI

[0232] Output: Auto-generated meeting transcript

[0233] Step 4:

[0234] The server receives the generated minutes and sends them to the terminal.

[0235] Input: Auto-generated meeting minutes

[0236] Output: Transcript displayed on terminal

[0237] Step 5:

[0238] The user checks the minutes on the terminal and corrects them as necessary.

[0239] Input: Auto-generated meeting minutes

[0240] Output: Corrected and confirmed minutes

[0241] Creating a draft for email distribution of meeting minutes

[0242] Step 1:

[0243] Based on the generated minutes, the server creates a prompt to request the AI ​​to generate email delivery suggestions.

[0244] Input: Amended and confirmed minutes

[0245] Output: prompt statement

[0246] Step 2:

[0247] The server sends the prompt text to the generation AI, which generates an email delivery proposal.

[0248] Input: prompt statement

[0249] Output: Generated email delivery proposal

[0250] Step 3:

[0251] The server receives the generated email delivery proposal and sends it to the terminal.

[0252] Input: Generated email delivery proposal

[0253] Output: Email delivery proposal displayed on the terminal

[0254] Step 4:

[0255] The user checks the email delivery proposal on the device and modifies and sends it as necessary.

[0256] Input: Generated email delivery proposal

[0257] Output: Confirmed email delivery proposal

[0258] (Application example 1)

[0259] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0260] Meeting management involves a lot of time-consuming and labor-intensive tasks, such as creating agendas, scheduling meetings, and creating and distributing minutes. Similarly, robots and workers working together in factories also require a lot of time and effort, such as creating work plans, managing progress, and creating and sharing work records. There is a need for a system that can streamline these tasks, save resources, and improve work quality.

[0261] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0262] In this invention, the server includes a generating means for receiving information on the theme and purpose of the meeting and automatically generating a proposed meeting agenda based on this information, a generating means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, a generating means for generating and proposing proposed comments for the facilitator in real time during the meeting, a generating means for receiving information on the theme and purpose of the work and automatically generating a proposed work agenda based on this information, a generating means for collecting schedule information of the workers and generating an optimal work schedule, a generating means for generating and displaying feedback on the progress of the work in real time while the work is in progress, a generating means for collecting work comment data and automatically generating work records, and a generating means for generating proposed email distribution to work participants based on the generated work records. This makes it possible to improve the efficiency of meetings and work in the factory.

[0263] A "theme" refers to the central subject of discussion or activity in a meeting or task.

[0264] "Purpose" refers to the goal or outcome you are trying to achieve when conducting a meeting or task.

[0265] "Information" refers to data and knowledge related to the progress of meetings and work.

[0266] "Generator" refers to a technical device or software that produces a particular output based on specified input.

[0267] "Schedule information" refers to data that includes the schedules of conference participants and workers.

[0268] A "facilitator" is a person whose role is to help the meeting proceed and facilitate smooth discussion.

[0269] "Suggested remarks and comments" are sentences that provide advice or instructions to encourage appropriate remarks during meetings or work.

[0270] "Real-time" refers to immediate processing and feedback on the spot.

[0271] Minutes are a document that records the progress and decisions made at a meeting.

[0272] "Email distribution proposal" refers to the proposed content of the email to be sent to participants in a meeting or task.

[0273] An "agenda proposal" is a plan that shows the topics and order of proceedings in a meeting or task.

[0274] "Worker" refers to the person or robot who actually performs the work in a factory or work site.

[0275] A "schedule" is a plan that allocates events by time.

[0276] "Progress" refers to the progress of work or meetings.

[0277] "Feedback" is information that provides immediate evaluation and advice regarding the progress of a task or meeting.

[0278] "Work records" refer to documents or data that record the work content and progress results.

[0279] In order to put this invention into practice, it is necessary to build a system that combines the above-described generating means. The specific program and processing method of this system will be described below.

[0280] The server first receives information about the meeting's theme and purpose. This information is entered by the user through their device. The server then uses a generative AI model to automatically generate a meeting agenda proposal based on this information. OpenAI's model is a suitable model for this purpose. This model generates an appropriate agenda proposal based on the input information.

[0281] Next, the server collects schedule information for meeting participants. In this step, it obtains data from corporate calendar systems and individual scheduling systems and uses a generative AI model to propose optimal meeting dates. Again, using OpenAI's generative model enables highly accurate schedule adjustments.

[0282] During the meeting, the server generates and suggests comments for the facilitator in real time. This is done in combination with speech analysis technology, and the generative AI model suggests appropriate comments. For example, it provides specific comments in real time, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[0283] For work within the factory, the server receives information about the work theme and objectives and automatically generates a proposed work agenda. This is also entered by the user via a terminal, and the proposed agenda is created by the generative AI model. Similarly, worker schedule information is collected and an optimal work schedule is generated.

[0284] As work progresses, the server generates real-time feedback on the progress and displays it to the worker via their device. This allows them to make appropriate improvements and adjustments to the work in progress immediately. For example, feedback such as "when to proceed to the next step" or "points to note to improve work efficiency" could be considered.

[0285] After the meeting ends, the server collects all voice data and chat logs, and automatically generates minutes and work records using a generative AI model. Based on the generated minutes and work records, the server then creates email distribution proposals for meeting participants and workers, which are then sent after the user has confirmed and revised them.

[0286] For example, to coordinate robots that inspect items and assemble parts in a factory, you could enter:

[0287] "RobotA's task: inspecting items"

[0288] "RobotB's task: Assembling parts"

[0289] "Robot availability:

[0290] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[0291] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[0292] Use this data to generate a proposed work agenda and schedule.

[0293] By inputting these prompts into a generative AI model, optimal agendas and schedules are proposed, improving work efficiency. This system enables the efficiency and quality of meetings and factory work to be improved.

[0294] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0295] Step 1:

[0296] Users enter information about the topic and purpose of meetings and tasks

[0297] Users input information about the topic and purpose of the meeting or task through their terminal. This information becomes the initial input data for the system. For example, they can input content such as "kickoff meeting for a new project" or "inspection and assembly of parts."

[0298] Input: Theme and purpose of the meeting or task

[0299] Output: Initial information sent to the server

[0300] Step 2:

[0301] The server creates the agenda proposal using a generative AI model

[0302] Based on the received information, the server sends prompts to the generative AI model to generate a proposed agenda. OpenAI's model is used as the generative AI model. For example, for the topic "Kickoff meeting for a new project," the server generates agenda proposals such as "Project overview" and "Setting main goals and timeline."

[0303] Input: The topic and purpose of the meeting or task entered by the user

[0304] Output: Agenda proposal obtained from the generative AI model

[0305] Step 3:

[0306] The server collects schedule information for meeting participants and workers.

[0307] The server collects schedule information for meeting participants and workers from the company's calendar system and uses this information to create prompts for generating optimal meeting dates and work schedules.

[0308] Input: List of meeting participants and workers

[0309] Output: Collected schedule information

[0310] Step 4:

[0311] The server creates prompts for schedule adjustment and creates a schedule proposal using a generative AI model.

[0312] Based on the collected schedule information, the server creates prompts to send to the generative AI model, which then generates optimal meeting date candidates and work schedule proposals. For example, it might present suggestions such as "next Monday 14:00-15:00" or "next Wednesday 16:00-17:00."

[0313] Input: Collected schedule information

[0314] Output: Optimal schedule proposal obtained from the generative AI model

[0315] Step 5:

[0316] The user reviews the generated agenda and schedule and makes any necessary revisions.

[0317] The user checks the agenda and schedule proposals provided by the server on their device and makes any necessary corrections. This process of checking and correcting determines the final agenda and schedule.

[0318] Input: Generated agenda and schedule proposals

[0319] Output: Confirmed agenda and schedule as reviewed and modified by the user

[0320] Step 6:

[0321] The server analyzes the progress of meetings and tasks in real time and provides suggestions and feedback for the facilitator.

[0322] As meetings and tasks progress, the server collects voice input and real-time data and analyzes it with a generative AI model. This provides real-time suggestions for comments to the facilitator and feedback to participants. For example, it provides specific suggestions such as "It's time to move on to the next agenda item" or "It's time to move on to the next step."

[0323] Input: Real-time voice input and real-time data collected in real time

[0324] Output: Proposed comments and feedback obtained from the generative AI model

[0325] Step 7:

[0326] Once a meeting or work session is over, the server automatically creates minutes and work records using a generation AI model.

[0327] Once a meeting or task is completed, the server collects all audio data and text logs, analyzes them with a generative AI model, and automatically creates meeting minutes and task logs, generating documents that summarize key points and decisions.

[0328] Input: Audio data and text logs of completed meetings and tasks

[0329] Output: Meeting minutes and work records obtained from generative AI models

[0330] Step 8:

[0331] The server creates an email delivery plan based on the generated minutes and work records, and the email is sent after the user checks and modifies it.

[0332] The server creates a proposal for sending emails to meeting participants and workers based on the generated minutes and work records. Users can check and modify this proposal on their devices, and the emails are finally sent.

[0333] Input: Generated minutes and work logs

[0334] Output: The email delivered after being reviewed and corrected by the user

[0335] Specifically, for coordinating robots that inspect items and assemble parts in a factory, the prompt might look like this:

[0336] "RobotA's task: inspecting items"

[0337] "RobotB's task: Assembling parts"

[0338] "Robot availability:

[0339] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[0340] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[0341] Use this data to generate a proposed work agenda and schedule.

[0342] This system will enable more efficient meetings and work within the factory, as well as improve quality.

[0343] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0344] This invention is a system that not only improves the efficiency of meeting management but also recognizes the emotions of meeting participants. This system uses generative AI and an emotion engine to support the entire meeting. The specific program processing and operation are explained below.

[0345] Automatic generation of agenda proposals

[0346] User:

[0347] The user enters the meeting topic, purpose, and related information into the system.

[0348] server:

[0349] The server receives this information and sends it to the generation AI module, which then generates a draft agenda based on this information.

[0350] Terminal (user):

[0351] The generated agenda plan is provided to the user, who then checks the plan and modifies it as necessary.

[0352] Examples:

[0353] If a user enters "new product launch meeting," the server uses generative AI to generate agenda proposals such as "1. Product overview explanation," "2. Marketing strategy," and "3. Sales plan," and displays them to the user.

[0354] Providing possible meeting dates

[0355] User:

[0356] The user enters a list of conference participants.

[0357] server:

[0358] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[0359] server:

[0360] The server uses a generation AI to generate optimal meeting date options based on available time.

[0361] Terminal (user):

[0362] The generated schedule candidates are presented to the user, who then selects the final schedule.

[0363] Examples:

[0364] Based on the schedules of participants A, B, and C, the generation AI presents candidate dates such as "next Tuesday 14:00-15:00" and "next Thursday 16:00-17:00," which the user can confirm.

[0365] Generates comments for the facilitator and a summary of the meeting content

[0366] server:

[0367] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[0368] server:

[0369] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[0370] Terminal (user):

[0371] The generative AI displays suggested comments and a summary of the discussion to the facilitator in real time.

[0372] Examples:

[0373] During the meeting, the server displays suggested comments on the device, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[0374] Automatic minutes creation

[0375] server:

[0376] Once the meeting is over, the server collects all the voice data and chat logs.

[0377] server:

[0378] The collected data is sent to a generation AI, which automatically generates minutes.

[0379] Terminal (user):

[0380] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[0381] Examples:

[0382] After the meeting, the AI ​​generates minutes summarizing the meeting, such as "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy," which the user can then review.

[0383] Creating a draft for email distribution of meeting minutes

[0384] server:

[0385] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[0386] Terminal (user):

[0387] The user checks and edits the email content, and the final email is sent.

[0388] Examples:

[0389] The server generates a draft email with the subject "The minutes of the meeting will be shared as follows," and after the user confirms it, it is sent to all participants.

[0390] Collaboration with emotion engine

[0391] server:

[0392] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[0393] server:

[0394] The emotion engine recognizes participants' emotions in real time and provides that information to the generative AI.

[0395] Terminal (user):

[0396] During the meeting, participants' emotional information is displayed on the facilitator's device, allowing the facilitator to take appropriate action.

[0397] Examples:

[0398] If the emotion engine recognizes that Participant A is feeling stressed, the server will display a suggestion to the facilitator such as, "Participant A seems tense. Please add a comment to help them relax before asking for other opinions."

[0399] After the meeting, the generated minutes also include emotional data of participants recognized during the meeting, providing a detailed record of the meeting atmosphere and participants' reactions, which can be used for future improvements.

[0400] As described above, this invention is a system that combines generative AI and an emotion engine to not only improve the efficiency of meeting management but also improve the quality of discussions based on the emotions of participants.

[0401] The processing flow will be explained below.

[0402] Automatic generation of agenda proposals

[0403] Step 1:

[0404] The user enters the meeting topic, purpose, and related information.

[0405] Step 2:

[0406] The server receives this information and sends it to the generation AI module.

[0407] Step 3:

[0408] The generation AI on the server generates a draft agenda based on the input information.

[0409] Step 4:

[0410] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[0411] Providing possible meeting dates

[0412] Step 1:

[0413] The user enters a list of conference participants.

[0414] Step 2:

[0415] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[0416] Step 3:

[0417] The server uses a generation AI to generate optimal meeting date options based on available time.

[0418] Step 4:

[0419] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[0420] Generates comments for the facilitator and a summary of the meeting content

[0421] Step 1:

[0422] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[0423] Step 2:

[0424] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[0425] Step 3:

[0426] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[0427] Automatic minutes creation

[0428] Step 1:

[0429] Once the meeting is over, the server collects all the voice data and chat logs.

[0430] Step 2:

[0431] The server sends the collected data to the generation AI, which automatically generates minutes.

[0432] Step 3:

[0433] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[0434] Creating a draft for email distribution of meeting minutes

[0435] Step 1:

[0436] The server generates a mail distribution plan for the meeting participants based on the generated minutes.

[0437] Step 2:

[0438] The server generates a draft email and provides it to the user.

[0439] Step 3:

[0440] The user checks the email content and corrects it if necessary.

[0441] Step 4:

[0442] After the user's confirmation, the server delivers the final email to the conference participants.

[0443] Collaboration with emotion engine

[0444] Step 1:

[0445] Before the meeting starts, the server starts the emotion engine, which analyzes the facial expressions and voices of the meeting participants in real time.

[0446] Step 2:

[0447] During the meeting, the server collects emotion data obtained from the emotion engine and provides it to the generation AI.

[0448] Step 3:

[0449] The server's generating AI generates appropriate response suggestions for the facilitator based on the emotional state of the participants.

[0450] Step 4:

[0451] The server displays participants' emotional states and suggested responses in real time on the facilitator's terminal.

[0452] Examples:

[0453] If the emotion engine recognizes that Participant A is nervous, the server will use the generation AI to generate a suggestion such as, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions," and display this on the facilitator's device. Emotional data will also be reflected in the proposed comments and discussion summaries displayed during the meeting.

[0454] Recording and utilizing emotional data

[0455] Step 1:

[0456] After the meeting ends, the server collects all emotional data and provides it to the minutes generation AI.

[0457] Step 2:

[0458] The server uses generation AI to add emotional data to meeting minutes and generate detailed records.

[0459] Step 3:

[0460] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[0461] Examples:

[0462] The generated minutes include not only a summary of each topic, but also the emotional state of the participants at the time of each comment (e.g., "tension" or "excitement"), which allows you to review the atmosphere of the meeting and the reactions of the participants later.

[0463] In this way, the present invention is a system that combines generative AI and an emotion engine to not only make meeting management more efficient, but also improve the quality of discussions based on the emotions of participants.

[0464] Example 2

[0465] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0466] Conventional meeting management systems have the problem that it takes a lot of time and effort to prepare for the meeting, coordinate participants' schedules, create and distribute minutes, and manage the progress of the meeting. Furthermore, it is difficult to grasp the emotional state of meeting participants in real time and respond appropriately, making it difficult to improve the quality of the meeting. There is a need to solve these problems and improve the efficiency and quality of meeting management.

[0467] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0468] In this invention, the server includes a generating means for receiving information about the theme and purpose of the conference and automatically generating a proposed conference content plan based on this information, a generating means for collecting schedule information of prospective conference participants and generating optimal conference date candidates, a generating means for generating and proposing speech and comment proposals for the facilitator in real time during the conference, a generating means for collecting conference speech data and written information and automatically generating a conference record, a generating means for generating a proposal for email distribution to conference participants based on the generated conference record, and a generating means for analyzing participants' facial expressions and voices and proposing the progress of the conference based on emotion analysis.This enables efficient meeting preparation, schedule adjustment, minutes creation and distribution, and meeting progress, as well as high-quality conference management through emotion analysis.

[0469] "Information about the theme and purpose of the meeting" is data indicating the content, purpose, target, etc. of the meeting.

[0470] The "means for automatically generating a meeting agenda" refers to a device or system that has the function of generating a meeting agenda based on input information.

[0471] "Schedule information of participants in the conference" is data related to the calendars and schedules of the participants.

[0472] The "means for generating optimal conference date candidates" refers to a device or system that has the function of automatically proposing optimal conference dates based on the schedule information of the participants.

[0473] The "means for generating and proposing speech comment proposals for the facilitator in real time" is a device or system that generates and displays speech proposals in real time so that the facilitator can appropriately conduct the meeting during the meeting.

[0474] "Speech data and written information" refers to document data such as voice data and chat logs recorded during a conference.

[0475] The "means for automatically generating meeting records" refers to a device or system that has the function of automatically creating minutes of a meeting based on speech data and written information.

[0476] The "means for generating a proposal for email distribution to conference participants" is a device or system that creates a template email for distributing the contents of the conference to participants by email based on the generated minutes.

[0477] The "generation means for analyzing facial expressions and voice and proposing meeting progress based on emotional analysis" is a device or system that has the function of analyzing the facial expressions and voice data of participants and proposing meeting progress based on their emotional state.

[0478] This invention is a system that uses a generative AI model and an emotion analysis engine to improve the efficiency and quality of meetings. This system exchanges data between a server, a terminal, and a user, and supports the management of meetings using various generation methods.

[0479] Automatic generation of agenda proposals

[0480] The user inputs the topic of the meeting (e.g., "launch meeting for a new product"), the purpose (e.g., "explanation of the product overview"), and related information into the terminal.

[0481] The server sends the information received from the user as a prompt to a generative AI model (e.g., GPT-3 (registered trademark)). The generative AI model generates a draft agenda based on the input information and sends it back to the server.

[0482] The terminal (user) provides the generated agenda proposal to the user, who then modifies it as necessary.

[0483] Example: If a user inputs "launch meeting for a new product," the server requests and receives agenda proposals such as "1. Product overview explanation," "2. Marketing strategy," and "3. Sales plan" from the generative AI model.

[0484] Providing possible meeting dates

[0485] The user inputs a list of conference participants into the terminal.

[0486] The server uses APIs to collect participants' schedule information from corporate calendars and personal calendar systems (e.g., Google Calendar). Based on the collected data, a generative AI model generates optimal meeting date candidates.

[0487] The terminal (user) presents the generated schedule candidates to the user, who then selects the final schedule.

[0488] Example: Based on the schedules of participants A, B, and C, suggest possible dates such as "next Tuesday 14:00-15:00" and "next Thursday 16:00-17:00."

[0489] Generates comments for the facilitator and a summary of the meeting content

[0490] The server collects audio data and chat logs (e.g., Zoom transcripts) in real time during the meeting.

[0491] The generative AI model generates proposed comments and discussion progression suggestions in real time based on the collected data and sends them back to the server.

[0492] The terminal (user) displays the generated proposed remarks and summaries to the facilitator.

[0493] Example: Displaying suggested comments during a meeting, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[0494] Automatic minutes creation

[0495] The server collects all voice data and chat logs once the conference is over.

[0496] The generative AI model automatically generates minutes based on the collected data and sends them back to the server.

[0497] The terminal (user) provides the generated minutes to the user, who checks the contents and corrects them as necessary.

[0498] Example: After a meeting, a generative AI model generates a summary such as, "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy."

[0499] Creating a draft for email distribution of meeting minutes

[0500] The server uses a generative AI model based on the generated minutes to create email suggestions for sending to meeting participants.

[0501] The terminal (user) provides the email content to the user, who then checks the content and corrects it if necessary.

[0502] The user finally clicks the "Send" button to deliver the email.

[0503] Example: Generate a draft email with the subject "We will share the meeting minutes as follows," and send it to all participants after the user has confirmed it.

[0504] Collaboration with emotion engine

[0505] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[0506] The emotion engine provides analyzed emotion information to the generative AI model.

[0507] The generative AI model proposes meeting progress based on emotional information and sends it back to the server.

[0508] The terminal (user) displays emotional information on the facilitator's terminal in real time.

[0509] Example: If the emotion engine recognizes that participant A is nervous, it will display a suggestion such as "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions." After the meeting, emotion data is added to the generated minutes.

[0510] The above is an embodiment of the present invention. This system makes it possible to prepare for a meeting, adjust schedules, create and distribute minutes, improve the efficiency of the meeting, and manage a high-quality meeting through emotion analysis.

[0511] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0512] Step 1:

[0513] The user inputs the topic, purpose and related information of the meeting into the terminal.

[0514] Input: Meeting topic (e.g., "New product launch meeting"), purpose (e.g., "Product overview"), and related information.

[0515] Specific action: The user types the topic, purpose, and related information into an input form.

[0516] Output: The meeting topic, purpose, and related information are saved as data on the device.

[0517] Step 2:

[0518] The server sends the information received from the user to the generative AI model as a prompt sentence.

[0519] Input: User-entered meeting topic, purpose, and related information.

[0520] Data processing: The server generates a prompt based on this information.

[0521] Specific behavior: The server creates a prompt sentence, "New product launch meeting," and sends it to the generative AI model.

[0522] Output: A prompt to the generative AI model.

[0523] Step 3:

[0524] The generative AI model generates a draft agenda based on the prompt text and sends it back to the server.

[0525] Input: The prompt text sent by the server.

[0526] Data computation: A generative AI model analyzes the prompt and generates a draft agenda.

[0527] Specific operation: The generative AI model creates agenda proposals such as "1. Product overview description," "2. Marketing strategy," and "3. Sales plan."

[0528] Output: The generated draft agenda.

[0529] Step 4:

[0530] The server transmits the generated agenda plan to the terminal.

[0531] Input: Agenda proposal returned by the generative AI model.

[0532] Specific operation: The server receives the agenda proposal and sends it to the user's device.

[0533] Output: The proposed agenda displayed on the user's device.

[0534] Step 5:

[0535] The terminal (user) checks the generated agenda plan and corrects it if necessary.

[0536] Input: Proposed agenda displayed on terminal.

[0537] What happens: User reviews the draft agenda and makes revisions using the editor.

[0538] Output: Revised proposed agenda.

[0539] Step 6:

[0540] A user inputs a list of conference participants into a terminal.

[0541] Input: Participant information such as name and email address.

[0542] Specific behavior: The user types participant information into an input form.

[0543] Output: The participant list is saved as data on the device.

[0544] Step 7:

[0545] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[0546] Input: Participant list.

[0547] Data processing: The server uses the API to collect participants' schedule information.

[0548] Specific operation: The server calls the Google Calendar API or similar to obtain the schedule information of participants.

[0549] Output: Collected schedule information.

[0550] Step 8:

[0551] Based on the schedule information collected by the server, the generative AI model generates optimal candidate meeting dates.

[0552] Input: Collected schedule information.

[0553] Data calculation: A generative AI model analyzes schedule information and generates optimal schedule candidates.

[0554] Specific operation: The generative AI model creates candidate dates such as "Next Tuesday 14:00-15:00" or "Next Thursday 16:00-17:00."

[0555] Output: The generated candidate dates.

[0556] Step 9:

[0557] The terminal (user) presents the generated schedule candidates to the user, who then selects the final schedule.

[0558] Input: Generated candidate dates.

[0559] Specific operation: Candidate dates are displayed on the user's screen, and the user selects the desired date.

[0560] Output: The final selected dates.

[0561] Step 10:

[0562] The server collects the audio data and chat logs of the conference.

[0563] Input: Audio data and chat logs generated during the conference.

[0564] Data processing: The server collects these data in real time.

[0565] Specific operation: The server uses a speech recognition API to convert the voice data into text and saves it together with the chat log.

[0566] Output: Collected meeting data.

[0567] Step 11:

[0568] The generative AI model generates suggested comments and summaries in real time based on the meeting's audio data and chat logs.

[0569] Input: Audio data and chat logs.

[0570] Data computation: Generative AI models analyze data and generate suggested comments and discussion summaries.

[0571] What it does: The generative AI model makes suggestions such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[0572] Output: Generated comment proposals and summaries.

[0573] Step 12:

[0574] The terminal (user) displays the generated speech comment proposals and summaries to the facilitator.

[0575] Input: Generated comment proposals and summaries.

[0576] Specific operation: Proposed comments and summaries are displayed in real time on the facilitator's screen.

[0577] Output: Displayed information for the facilitator.

[0578] Step 13:

[0579] The server collects all voice data and chat logs after the conference ends.

[0580] Input: Audio data and chat logs generated during the meeting.

[0581] Data processing: The server collects these data again at the end of the meeting and manages them centrally.

[0582] What happens: The server uses the speech recognition API again to retrieve the voice and chat data.

[0583] Output: All collected meeting data.

[0584] Step 14:

[0585] The generative AI model automatically generates meeting minutes based on the collected data.

[0586] Input: Audio data and chat logs.

[0587] Data computation: A generative AI model analyzes data and generates meeting minutes.

[0588] What it does: The generative AI model creates a summary such as, "Today's agenda item 1 explained the details of the new product, and agenda item 2 discussed the marketing strategy."

[0589] Output: The generated transcript.

[0590] Step 15:

[0591] The terminal (user) provides the generated minutes to the user, who then checks and modifies the contents.

[0592] Input: The generated minutes.

[0593] What happens: The minutes are displayed on the user's screen and the user can make edits using the editor.

[0594] Output: Revised minutes.

[0595] Step 16:

[0596] The server uses a generative AI model based on the generated minutes to create email suggestions for participants.

[0597] Input: The generated minutes.

[0598] Data calculation: A generative AI model generates email delivery suggestions based on the meeting minutes.

[0599] What happens: The generative AI model creates an email with the subject line "We're sharing the meeting minutes as follows."

[0600] Output: The generated email delivery proposal.

[0601] Step 17:

[0602] The terminal (user) provides the email content to the user, who then checks, modifies, and distributes the content.

[0603] Input: The generated email proposal.

[0604] Specific behavior: The draft text is displayed in the user's email client and can be edited. The user clicks the send button and the email is sent to all participants.

[0605] Output: The email delivered.

[0606] Step 18:

[0607] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[0608] Input: Facial expression and voice data of meeting participants.

[0609] Data calculation: The emotion engine analyzes the data and generates emotion information.

[0610] Specific operation: The server calls the emotion engine API and starts analysis.

[0611] Output: The generated emotion information.

[0612] Step 19:

[0613] A generative AI model suggests meeting progress based on emotional information.

[0614] Input: Generated emotion information.

[0615] Data computation: A generative AI model analyzes emotional information and generates suggestions for how to proceed with the meeting.

[0616] What happens: The generative AI model makes suggestions like, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions."

[0617] Output: The generated meeting progress proposal.

[0618] Step 20:

[0619] The terminal (user) displays the generated conference progress proposal to the user.

[0620] Input: The generated meeting progress proposal.

[0621] What happens: The suggestion will appear on the facilitator's screen.

[0622] Output: Displayed suggestion information for the facilitator.

[0623] Step 21:

[0624] The generative AI model analyzes emotional data after the meeting and adds it to the minutes.

[0625] Input: Parsed emotion information and meeting data.

[0626] Data computation: Generative AI models analyze emotional data and incorporate it into meeting minutes.

[0627] Specific operation: The generative AI model adds information such as "Participant A seemed nervous during the meeting" to the minutes.

[0628] Output: Meeting transcripts with sentiment data added.

[0629] In this way, the system streamlines conference management and supports high-quality conference progress through each processing step.

[0630] (Application example 2)

[0631] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0632] Managing a meeting requires a great deal of time and effort, including creating an agenda, arranging meeting dates, providing real-time support, and creating minutes. Furthermore, it is necessary to understand participants' emotions in real time and devise ways to improve the quality of the meeting. In particular, in busy environments such as factories, there is a demand for systems that can efficiently manage these elements.

[0633] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0634] In this invention, the server includes a generating means for receiving information on the theme and purpose of the meeting and automatically generating a proposed meeting agenda based on this information, a generating means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, a generating means for generating and proposing proposed remarks for the facilitator in real time during the meeting, a generating means for collecting meeting remark data and automatically generating minutes, a generating means for generating email distribution proposals to the meeting participants based on the generated minutes, and a generating means for analyzing the facial expressions and voices of the participants during the meeting to recognize their emotions and supporting the progress of the meeting based on this information. This makes it possible to run the meeting more efficiently and further improve the quality of the meeting through real-time emotion recognition.

[0635] "Receiving information about the subject and purpose of the meeting" refers to obtaining data input from the user regarding the subject and purpose of the meeting.

[0636] An "agenda proposal" refers to a plan that summarizes the items and topics necessary for the meeting to proceed.

[0637] "Generation means" refers to a means that allows new information or data to be created using a generation method or device.

[0638] "Schedule information" refers to data related to the schedules and time management of each person scheduled to attend the meeting.

[0639] "Potential Meeting Dates" refers to possible options for dates and times for holding a conference.

[0640] "Suggested comments" refers to AI suggestions for comments or instructions that the facilitator should make during a meeting.

[0641] "Real-time" refers to processing or operations occurring continuously in actual time without delay.

[0642] "Speech data" is data indicating the content of oral statements made during a conference.

[0643] "Minutes" refers to a document that records what was discussed and what decisions were made at a meeting.

[0644] "Email distribution proposal" refers to a format that proposes the content of the email to be sent to participants after the meeting.

[0645] "Recognizing emotions by analyzing participants' facial expressions and voices" means analyzing the facial expressions and tone of voice of meeting participants to identify the emotions they are feeling.

[0646] This invention is a system aimed at streamlining meeting management, providing various functions using generative AI models and emotion engines, particularly for efficient meeting management in factory environments.

[0647] First, the user inputs information about the theme and purpose of the meeting into the system. The server receives this information and uses a generative AI model to generate a proposed agenda for the meeting. The generated agenda is provided to the user on their device, who can make any necessary revisions. For example, if the user inputs "New product development meeting," the server will generate a proposed agenda such as "1. Introduction of the new product," "2. Sharing of market research results," and "3. Confirmation of the development schedule."

[0648] Next, the server collects schedule information from the people planning to attend the meeting. The server obtains this information from corporate and personal calendar systems and generates optimal meeting date candidates. For example, based on the schedules of participants A, B, and C, the AI ​​generator might present candidate dates such as "next Tuesday, 2:00 PM - 3:00 PM" or "next Thursday, 4:00 PM - 5:00 PM."

[0649] During the meeting, the server collects the voice input and chat logs of the meeting in real time and generates suggested comments and discussion progress suggestions for the facilitator. This information is displayed on the device in real time. For example, the server generates suggested comments in real time, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[0650] After the meeting ends, the server collects all audio data and chat logs and automatically generates meeting minutes using a generative AI model. The generated minutes are provided to the user on their device and can be revised as needed. For example, the server generates minutes summarizing the meeting, such as "Today's agenda item 1 explained the details of the new product, and agenda item 2 discussed marketing strategies," and the user can review them.

[0651] Furthermore, the server generates a proposal for sending emails to the meeting participants based on the generated minutes. The user can then check and modify the email content, and the final email is sent to the participants.

[0652] It also works in conjunction with an emotion engine. During a meeting, the server analyzes participants' facial expressions and voices to recognize their emotions in real time. This information is displayed on the facilitator's device and helps support the progress of the meeting. For example, if the emotion engine recognizes that one participant is feeling stressed, it will make a suggestion such as, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions."

[0653] The hardware used includes computers, smart devices, and devices with cameras. The software includes the OpenAI API, a calendar integration module, and a proprietary emotion engine. The generative AI model uses prompts as input and generates the necessary information.

[0654] For example, by inputting a prompt such as "New product development meeting," the system automatically generates a draft agenda and meeting minutes. It also suggests optimal meeting dates based on the participants' schedules, provides support during the meeting, and analyzes participants' sentiment.

[0655] In this way, the present invention provides a system for improving the efficiency and quality of conference management even in busy environments such as factories.

[0656] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0657] Step 1: Enter the meeting theme and purpose and automatically generate a draft agenda

[0658] The user inputs information about the meeting's theme and purpose into the device. The server receives this information and sends prompts to the generative AI model. The generative AI model generates a draft agenda and returns the results to the server. The server displays the draft agenda on the device, allowing the user to review and modify it.

[0659] Input: Meeting topic and purpose (e.g., "New product development meeting")

[0660] Output: Agenda proposal (e.g., "1. Introduce new product," "2. Share market research results," "3. Confirm development schedule")

[0661] How it works: The user enters information into the device, the server sends it to the generative AI model, which then generates a draft agenda and displays the results on the device.

[0662] Step 2: Collect schedules of meeting participants and generate candidate meeting dates

[0663] The server collects schedule information of meeting participants from corporate and personal calendar systems. Based on the collected data, it uses a generative AI model to propose optimal meeting date candidates. The server then displays the generated date candidates on the device for the user to confirm.

[0664] Input: List of expected attendees (e.g., Participants A, B, C) and their schedule information

[0665] Output: Best possible dates (e.g. "Next Tuesday 14:00-15:00" or "Next Thursday 16:00-17:00")

[0666] Specific operation: The server collects schedule information from the calendar system, analyzes it, and generates schedule suggestions. The results are displayed on the device for the user to confirm.

[0667] Step 3: Real-time comment proposals and discussion direction suggestions

[0668] During the meeting, the server collects the meeting's voice input and chat logs in real time. The generative AI model uses this data to generate suggested comments and discussion progress suggestions for the facilitator. The generated suggestions are displayed on the device in real time.

[0669] Input: Meeting audio data and chat logs

[0670] Output: Proposals for comments and discussion progression (e.g., "Let's move on to the next topic" or "Let's hear the opinions of other participants")

[0671] Specific operation: The server collects and analyzes voice data and chat logs, and the generative AI model generates suggested comments and displays them on the device.

[0672] Step 4: Automatic meeting transcript generation

[0673] When the meeting ends, the server collects all audio data and chat logs and sends them to the generative AI model. The generative AI model then summarizes the meeting content and automatically generates minutes. The generated minutes are displayed on the device, allowing the user to review and edit the content.

[0674] Input: Collected meeting audio data and chat logs

[0675] Output: Generated meeting minutes (e.g., "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy.")

[0676] Specific operation: After the server collects voice data and chat logs, it uses a generative AI model to generate minutes, which are then displayed on the device for the user to review and edit.

[0677] Step 5: Generate email suggestions based on the meeting minutes

[0678] The server uses a generative AI model to create email proposals to send to meeting participants based on the generated minutes. The generated email proposals are displayed on the device, where the user can check and edit the content before finally sending them.

[0679] Input: Generated minutes

[0680] Output: Proposal for email distribution (e.g. "The minutes of the meeting will be shared as follows")

[0681] Specific operation: The server generates an email distribution proposal based on the minutes and displays it on the terminal. After the user confirms and corrects it, the proposal is distributed.

[0682] Step 6: Recognizing participants' emotions and supporting the progress of the meeting using an emotion engine

[0683] The server collects participants' facial expressions and voices through cameras and microphones during the meeting, and analyzes them with an emotion engine to recognize their emotions. Based on the recognized emotional information, a generative AI model generates support suggestions for the progress of the meeting and displays them on the device in real time.

[0684] Input: Participant's facial expression and voice data

[0685] Output: Emotion recognition results and meeting facilitation suggestions (e.g., "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions.")

[0686] Specific operation: The server collects data from the camera and microphone, analyzes it with an emotion engine, and generates progress support measures using a generative AI model based on the recognized emotions and displays them on the device.

[0687] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0688] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ), Gemini (registered trademark) (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0689] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0690] [Second embodiment]

[0691] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0692] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0693] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0694] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0695] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0696] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0697] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0698] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0699] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0700] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0701] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0702] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0703] The present invention is a system that utilizes generative AI to improve the efficiency of meeting management, and its program and processing will be explained based on specific procedures.

[0704] 1. Automatic generation of agenda proposals

[0705] User:

[0706] The user inputs the meeting topic, purpose, relevant information, etc. into the system.

[0707] server:

[0708] The server receives this information and sends it to an internal generation AI, which generates a draft agenda based on this data.

[0709] Terminal (user):

[0710] The generated agenda plan is provided to the user, who can then modify and confirm it as necessary.

[0711] Examples:

[0712] If a user types in "kickoff meeting for a new project," the server uses generation AI to generate a draft agenda, including "1. Project overview," "2. Setting main goals and timeline," and "3. Confirming the roles and responsibilities of each person involved," and displays it to the user.

[0713] 2. Providing possible meeting dates

[0714] User:

[0715] The user inputs a list of members to attend the meeting.

[0716] server:

[0717] The server collects each participant's schedule information from the company's calendar system and uses generation AI to create optimal meeting date options.

[0718] Terminal (user):

[0719] The generated schedule candidates are presented to the user, who then selects and confirms the final schedule.

[0720] Examples:

[0721] Based on the schedules of participants A, B, and C, the generation AI proposes possible dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00" and displays them to the user.

[0722] 3. Generate comments for the facilitator and a summary of the meeting content

[0723] server:

[0724] Once the conference begins, the server analyzes the conference's voice input and chat log in real time.

[0725] Terminal (user):

[0726] The server provides the facilitator with real-time suggestions for comments and discussions, and also summarizes the meeting content and displays it to the facilitator.

[0727] Examples:

[0728] During the meeting, the server displays suggested comments on the device, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[0729] 4. Automatic minutes creation

[0730] server:

[0731] After the meeting ends, the server collects all audio data and chat logs, and automatically generates minutes using a generation AI.

[0732] Terminal (user):

[0733] The generated minutes are provided to the user, who can then review and edit the contents.

[0734] Examples:

[0735] After the meeting, the generative AI generates minutes summarizing the main points and decisions of the meeting, and presents the user with a document stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later."

[0736] 5. Create a draft of email distribution of meeting minutes

[0737] server:

[0738] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[0739] Terminal (user):

[0740] The user checks and edits the email content, and then sends the final email.

[0741] Examples:

[0742] The server creates a draft email containing the minutes of the meeting with the subject "We will share the minutes of the meeting as follows" and displays it to the user. After the user confirms it, the generation AI sends the email to all participants.

[0743] As described above, this invention is a system that utilizes generative AI to provide integrated support for meeting preparation, progress, discussion stimulation, and the creation and distribution of meeting minutes. This system significantly improves the efficiency of meeting management, saves resources, and improves meeting quality.

[0744] The processing flow will be explained below.

[0745] Automatic generation of agenda proposals

[0746] Step 1:

[0747] The user enters the meeting topic, purpose, and related information.

[0748] Step 2:

[0749] The server receives input information from the user and sends it to the generation AI module.

[0750] Step 3:

[0751] The generation AI on the server generates a draft agenda based on the input information.

[0752] Step 4:

[0753] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[0754] Providing possible meeting dates

[0755] Step 1:

[0756] The user enters a list of conference participants.

[0757] Step 2:

[0758] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[0759] Step 3:

[0760] The server uses a generation AI to generate optimal meeting date options based on available time.

[0761] Step 4:

[0762] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[0763] Generates comments for the facilitator and a summary of the meeting content

[0764] Step 1:

[0765] When the meeting starts, the server collects the meeting's voice input and chat log in real time.

[0766] Step 2:

[0767] The server uses generative AI to create suggested comments for the facilitator and suggestions for how to proceed with the discussion.

[0768] Step 3:

[0769] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[0770] Automatic minutes creation

[0771] Step 1:

[0772] After the meeting ends, the server collects all audio data and chat logs.

[0773] Step 2:

[0774] The server sends the collected data to the generation AI, which automatically generates minutes.

[0775] Step 3:

[0776] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[0777] Creating a draft for email distribution of meeting minutes

[0778] Step 1:

[0779] The server creates a proposal for sending emails to the meeting participants based on the generated minutes.

[0780] Step 2:

[0781] The server generates a draft email and provides it to the user.

[0782] Step 3:

[0783] The user checks the email content and corrects it if necessary.

[0784] Step 4:

[0785] After the user's confirmation, the server delivers the final email to the conference participants.

[0786] Example 1

[0787] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0788] Preparing for, conducting, and post-conference administrative tasks require time and effort, and efficiency is especially important for large-scale meetings. Furthermore, manual tasks such as scheduling and creating minutes can easily lead to errors, potentially lowering the quality of the meeting. Given this current situation, it is necessary to utilize generative AI models to centrally manage meeting operations and improve efficiency.

[0789] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0790] In this invention, the server includes means for receiving information about the theme and purpose of the meeting and automatically generating a meeting agenda based on this information, means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, and means for generating and proposing speech and comment proposals for the facilitator in real time during the meeting. This allows for centralized management of meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving efficiency.

[0791] "Information regarding the theme and purpose of the meeting" is information necessary to identify the content of the meeting, and mainly includes the theme, purpose, and related information of the meeting.

[0792] "Schedule information" is information indicating the schedule of people who plan to attend a conference, and refers to data acquired from a calendar system or the like.

[0793] A "generative AI model" is an artificial intelligence model that generates draft agendas, draft remarks, minutes, etc. based on information about the theme and purpose of a meeting.

[0794] "Minutes" are documents that record the contents of a meeting, statements made, decisions made, etc., and are automatically generated.

[0795] The "email distribution proposal" is a proposal for the content of emails to be sent to the meeting participants, which is created based on the generated minutes.

[0796] "Biometric authentication" is a technology that uses biological characteristics such as fingerprint authentication and facial authentication to verify a user's identity.

[0797] A "user" refers to a person who operates the system and whose role is to input the theme and purpose of the meeting, a list of participants, and so on.

[0798] A "facilitator" is a person whose role is to proceed and manage the meeting, and who guides the discussion by referring to suggestions and summaries from the system.

[0799] A "prompt sentence" is an input sentence given to a generative AI model, which contains instructions for the model to generate a specific output.

[0800] This invention is a system that utilizes generative AI models to improve the efficiency of meeting management. The program of this system is mainly composed of a server, terminals, and users, each of which plays a specific role.

[0801] Automatic generation of agenda proposals

[0802] The user inputs the meeting theme, purpose, and related information into the device. The server receives this information and sends it to a generation AI (e.g., OpenAI's GPT-4). The generation AI generates a draft agenda based on this data. The generated draft agenda is again sent to the device via the server and provided to the user. For example, if the user inputs "kickoff meeting for a new project," the generation AI will generate a draft agenda including "1. Overview of the project," "2. Setting of main goals and timeline," and "3. Confirmation of roles and responsibilities of each person involved," and display it to the user.

[0803] Example prompt sentence:

[0804] Create an agenda for the kickoff meeting for a new project.

[0805] Providing possible meeting dates

[0806] The user inputs a list of members participating in the meeting into the device. The server collects schedule information for each participant from the company's calendar system (e.g., Google Calendar, Microsoft Outlook). The collected schedule information is sent to the generation AI, which generates optimal meeting date candidates. These date candidates are presented to the user via the server, who selects and confirms the final date. For example, based on the schedules of participants A, B, and C, the generation AI proposes candidate dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00."

[0807] Example prompt sentence:

[0808] Please suggest the best possible meeting date based on the schedules of participants A, B, and C.

[0809] Generates comments for the facilitator and a summary of the meeting content

[0810] When a meeting begins, the device collects the meeting's voice input and chat log in real time and sends it to the server. The server analyzes this data in real time and sends it to a generation AI to generate suggested comments and a summary of the meeting content. The generated suggested comments and summaries are displayed on the device via the server and used by the facilitator to guide the meeting. For example, during a meeting, the server displays suggested comments on the device, such as "It's time to move on to the next agenda item" or "We should ask for other opinions on this agenda item."

[0811] Example prompt sentence:

[0812] Analyze meeting audio data and chat logs in real time and provide suggested comments for the facilitator.

[0813] Automatic minutes creation

[0814] When the meeting ends, the server collects all audio data and chat logs and sends them to the generation AI, which automatically generates minutes. The generated minutes are then sent to the device via the server, where the user can review and edit the contents. For example, after a meeting, the generation AI generates minutes stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later," and presents them to the user.

[0815] Example prompt sentence:

[0816] Automatically generate meeting minutes based on meeting audio data and chat logs.

[0817] Creating a draft for email distribution of meeting minutes

[0818] Based on the generated minutes, the server sends a prompt to the generation AI to generate a proposed email to send to the meeting participants. This proposed email is sent to the terminal via the server, where the user can confirm and modify it, and then finally send the email. For example, the server creates a proposed email with the subject "We will share the minutes of the meeting as follows" and displays it to the user.

[0819] Example prompt sentence:

[0820] Please create a proposal for emailing meeting minutes in the following format.

[0821] This system utilizes generative AI models to centrally manage and efficiently support meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving the overall efficiency and quality of meeting management.

[0822] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0823] Automatic generation of agenda proposals

[0824] Step 1:

[0825] The user inputs the theme, purpose, and related information of the meeting into an input form on the terminal. In this example, the user inputs information on the theme "kick-off meeting for a new project."

[0826] Input: Meeting topic, objectives, relevant information

[0827] Output: Initial input data sent from the device to the server

[0828] Step 2:

[0829] The server receives the input data and sends it to the generation AI, which then formats the input data appropriately as a prompt and provides it to the generation AI.

[0830] Input: Initial input data from the user

[0831] Output: Formatted prompt to send to the generation AI

[0832] Step 3:

[0833] The AI ​​generates a draft agenda based on the prompts, such as "1. Overview of the project" and "2. Setting main goals and timelines."

[0834] Input: Server formatted prompt

[0835] Output: The generated agenda proposal

[0836] Step 4:

[0837] The server receives the output from the generation AI and sends the proposed agenda to the terminal.

[0838] Input: Agenda proposal from the generative AI

[0839] Output: Draft agenda displayed on terminal

[0840] Step 5:

[0841] The user checks the agenda proposal displayed on the device and makes corrections or additions as necessary, for example, adding new items or modifying existing items.

[0842] Input: Generated agenda proposal

[0843] Output: Revised and confirmed agenda proposal

[0844] Providing possible meeting dates

[0845] Step 1:

[0846] The user inputs the list of participants in the conference into the terminal. For example, the user inputs the names and email addresses of "Participants A, B, and C."

[0847] Input: Participant list

[0848] Output: Participant data sent from the device to the server

[0849] Step 2:

[0850] The server accesses the company's calendar system (e.g., Google Calendar, Microsoft Outlook) and collects schedule information for the specified participants.

[0851] Input: Participant data

[0852] Output: Collected schedule information

[0853] Step 3:

[0854] The server sends the schedule information as a prompt to the generation AI, which then generates optimal meeting date candidates. For example, it uses the prompt "Tell me a date when everyone is free."

[0855] Input: Collected schedule information

[0856] Output: Generated itinerary candidates

[0857] Step 4:

[0858] The server receives the output of the generated AI and sends the proposed dates to the terminal.

[0859] Input: Generated date candidates

[0860] Output: Date options displayed on the terminal

[0861] Step 5:

[0862] The user checks the schedule options displayed on the device and selects and confirms the final date.

[0863] Input: Generated date candidates

[0864] Output: Confirmed meeting dates

[0865] Generates comments for the facilitator and a summary of the meeting content

[0866] Step 1:

[0867] When a meeting starts, the device collects the meeting's voice input and chat log in real time and sends them to the server, where the voice data is converted into text using automatic speech recognition (ASR) technology.

[0868] Input: Conference audio input, chat log

[0869] Output: Text data sent to the server

[0870] Step 2:

[0871] The server analyzes the received data in real time and sends it to the generating AI.

[0872] Input: Voice-to-text data, chat logs

[0873] Output: The prompt sent to the generation AI

[0874] Step 3:

[0875] The generative AI generates suggested comments and summaries of meeting content, such as "It's time to move on to the next agenda item" or "We should seek other opinions on this topic."

[0876] Input: The prompt sent to the generation AI

[0877] Output: Generated comment proposals and a summary of the meeting content

[0878] Step 4:

[0879] The server receives the output of the generated AI and sends proposed comments and meeting summaries to the device.

[0880] Input: Generated speech and comment proposals, summary of the meeting content

[0881] Output: Proposed comments and summary displayed on terminal

[0882] Step 5:

[0883] The facilitator checks the proposed comments and meeting summary displayed on the terminal and leads the discussion.

[0884] Input: Generated speech and comment proposals, summary of the meeting content

[0885] Output: Moderated meeting

[0886] Automatic minutes creation

[0887] Step 1:

[0888] When the meeting ends, the server collects all audio data and chat logs.

[0889] Input: Audio data of completed meetings, chat logs

[0890] Output: Data stored on the server

[0891] Step 2:

[0892] The server sends the collected data to the generation AI and requests it to generate minutes.

[0893] Input: Accumulated meeting data

[0894] Output: The prompt sent to the generation AI

[0895] Step 3:

[0896] The AI ​​automatically generates minutes based on the provided data, including key points and decisions.

[0897] Input: The prompt sent to the generation AI

[0898] Output: Auto-generated meeting transcript

[0899] Step 4:

[0900] The server receives the generated minutes and sends them to the terminal.

[0901] Input: Auto-generated meeting minutes

[0902] Output: Transcript displayed on terminal

[0903] Step 5:

[0904] The user checks the minutes on the terminal and corrects them as necessary.

[0905] Input: Auto-generated meeting minutes

[0906] Output: Corrected and confirmed minutes

[0907] Creating a draft for email distribution of meeting minutes

[0908] Step 1:

[0909] Based on the generated minutes, the server creates a prompt to request the AI ​​to generate email delivery suggestions.

[0910] Input: Amended and confirmed minutes

[0911] Output: prompt statement

[0912] Step 2:

[0913] The server sends the prompt text to the generation AI, which generates an email delivery proposal.

[0914] Input: prompt statement

[0915] Output: Generated email delivery proposal

[0916] Step 3:

[0917] The server receives the generated email delivery proposal and sends it to the terminal.

[0918] Input: Generated email delivery proposal

[0919] Output: Email delivery proposal displayed on the terminal

[0920] Step 4:

[0921] The user checks the email delivery proposal on the device and modifies and sends it as necessary.

[0922] Input: Generated email delivery proposal

[0923] Output: Confirmed email delivery proposal

[0924] (Application example 1)

[0925] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0926] Meeting management involves a lot of time-consuming and labor-intensive tasks, such as creating agendas, scheduling meetings, and creating and distributing minutes. Similarly, robots and workers working together in factories also require a lot of time and effort, such as creating work plans, managing progress, and creating and sharing work records. There is a need for a system that can streamline these tasks, save resources, and improve work quality.

[0927] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0928] In this invention, the server includes a generating means for receiving information on the theme and purpose of the meeting and automatically generating a proposed meeting agenda based on this information, a generating means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, a generating means for generating and proposing proposed comments for the facilitator in real time during the meeting, a generating means for receiving information on the theme and purpose of the work and automatically generating a proposed work agenda based on this information, a generating means for collecting schedule information of the workers and generating an optimal work schedule, a generating means for generating and displaying feedback on the progress of the work in real time while the work is in progress, a generating means for collecting work comment data and automatically generating work records, and a generating means for generating proposed email distribution to work participants based on the generated work records. This makes it possible to improve the efficiency of meetings and work in the factory.

[0929] A "theme" refers to the central subject of discussion or activity in a meeting or task.

[0930] "Purpose" refers to the goal or outcome you are trying to achieve when conducting a meeting or task.

[0931] "Information" refers to data and knowledge related to the progress of meetings and work.

[0932] "Generator" refers to a technical device or software that produces a particular output based on specified input.

[0933] "Schedule information" refers to data that includes the schedules of conference participants and workers.

[0934] A "facilitator" is a person whose role is to help the meeting proceed and facilitate smooth discussion.

[0935] "Suggested remarks and comments" are sentences that provide advice or instructions to encourage appropriate remarks during meetings or work.

[0936] "Real-time" refers to immediate processing and feedback on the spot.

[0937] Minutes are a document that records the progress and decisions made at a meeting.

[0938] "Email distribution proposal" refers to the proposed content of the email to be sent to participants in a meeting or task.

[0939] An "agenda proposal" is a plan that shows the topics and order of proceedings in a meeting or task.

[0940] "Worker" refers to the person or robot who actually performs the work in a factory or work site.

[0941] A "schedule" is a plan that allocates events by time.

[0942] "Progress" refers to the progress of work or meetings.

[0943] "Feedback" is information that provides immediate evaluation and advice regarding the progress of a task or meeting.

[0944] "Work records" refer to documents or data that record the work content and progress results.

[0945] In order to put this invention into practice, it is necessary to build a system that combines the above-described generating means. The specific program and processing method of this system will be described below.

[0946] The server first receives information about the meeting's theme and purpose. This information is entered by the user through their device. The server then uses a generative AI model to automatically generate a meeting agenda proposal based on this information. OpenAI's model is a suitable model for this purpose. This model generates an appropriate agenda proposal based on the input information.

[0947] Next, the server collects schedule information for meeting participants. In this step, it obtains data from corporate calendar systems and individual scheduling systems and uses a generative AI model to propose optimal meeting dates. Again, using OpenAI's generative model enables highly accurate schedule adjustments.

[0948] During the meeting, the server generates and suggests comments for the facilitator in real time. This is done in combination with speech analysis technology, and the generative AI model suggests appropriate comments. For example, it provides specific comments in real time, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[0949] For work within the factory, the server receives information about the work theme and objectives and automatically generates a proposed work agenda. This is also entered by the user via a terminal, and the proposed agenda is created by the generative AI model. Similarly, worker schedule information is collected and an optimal work schedule is generated.

[0950] As work progresses, the server generates real-time feedback on the progress and displays it to the worker via their device. This allows them to make appropriate improvements and adjustments to the work in progress immediately. For example, feedback such as "when to proceed to the next step" or "points to note to improve work efficiency" could be considered.

[0951] After the meeting ends, the server collects all voice data and chat logs, and automatically generates minutes and work records using a generative AI model. Based on the generated minutes and work records, the server then creates email distribution proposals for meeting participants and workers, which are then sent after the user has confirmed and revised them.

[0952] For example, to coordinate robots that inspect items and assemble parts in a factory, you could enter:

[0953] "RobotA's task: inspecting items"

[0954] "RobotB's task: Assembling parts"

[0955] "Robot availability:

[0956] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[0957] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[0958] Use this data to generate a proposed work agenda and schedule.

[0959] By inputting these prompts into a generative AI model, optimal agendas and schedules are proposed, improving work efficiency. This system enables the efficiency and quality of meetings and factory work to be improved.

[0960] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0961] Step 1:

[0962] Users enter information about the topic and purpose of meetings and tasks

[0963] Users input information about the topic and purpose of the meeting or task through their terminal. This information becomes the initial input data for the system. For example, they can input content such as "kickoff meeting for a new project" or "inspection and assembly of parts."

[0964] Input: Theme and purpose of the meeting or task

[0965] Output: Initial information sent to the server

[0966] Step 2:

[0967] The server creates the agenda proposal using a generative AI model

[0968] Based on the received information, the server sends prompts to the generative AI model to generate a proposed agenda. OpenAI's model is used as the generative AI model. For example, for the topic "Kickoff meeting for a new project," the server generates agenda proposals such as "Project overview" and "Setting main goals and timeline."

[0969] Input: The topic and purpose of the meeting or task entered by the user

[0970] Output: Agenda proposal obtained from the generative AI model

[0971] Step 3:

[0972] The server collects schedule information for meeting participants and workers.

[0973] The server collects schedule information for meeting participants and workers from the company's calendar system and uses this information to create prompts for generating optimal meeting dates and work schedules.

[0974] Input: List of meeting participants and workers

[0975] Output: Collected schedule information

[0976] Step 4:

[0977] The server creates prompts for schedule adjustment and creates a schedule proposal using a generative AI model.

[0978] Based on the collected schedule information, the server creates prompts to send to the generative AI model, which then generates optimal meeting date candidates and work schedule proposals. For example, it might present suggestions such as "next Monday 14:00-15:00" or "next Wednesday 16:00-17:00."

[0979] Input: Collected schedule information

[0980] Output: Optimal schedule proposal obtained from the generative AI model

[0981] Step 5:

[0982] The user reviews the generated agenda and schedule and makes any necessary revisions.

[0983] The user checks the agenda and schedule proposals provided by the server on their device and makes any necessary corrections. This process of checking and correcting determines the final agenda and schedule.

[0984] Input: Generated agenda and schedule proposals

[0985] Output: Confirmed agenda and schedule as reviewed and modified by the user

[0986] Step 6:

[0987] The server analyzes the progress of meetings and tasks in real time and provides suggestions and feedback for the facilitator.

[0988] As meetings and tasks progress, the server collects voice input and real-time data and analyzes it with a generative AI model. This provides real-time suggestions for comments to the facilitator and feedback to participants. For example, it provides specific suggestions such as "It's time to move on to the next agenda item" or "It's time to move on to the next step."

[0989] Input: Real-time voice input and real-time data collected in real time

[0990] Output: Proposed comments and feedback obtained from the generative AI model

[0991] Step 7:

[0992] Once a meeting or work session is over, the server automatically creates minutes and work records using a generation AI model.

[0993] Once a meeting or task is completed, the server collects all audio data and text logs, analyzes them with a generative AI model, and automatically creates meeting minutes and task logs, generating documents that summarize key points and decisions.

[0994] Input: Audio data and text logs of completed meetings and tasks

[0995] Output: Meeting minutes and work records obtained from generative AI models

[0996] Step 8:

[0997] The server creates an email delivery plan based on the generated minutes and work records, and the email is sent after the user checks and modifies it.

[0998] The server creates a proposal for sending emails to meeting participants and workers based on the generated minutes and work records. Users can check and modify this proposal on their devices, and the emails are finally sent.

[0999] Input: Generated minutes and work logs

[1000] Output: The email delivered after being reviewed and corrected by the user

[1001] Specifically, for coordinating robots that inspect items and assemble parts in a factory, the prompt might look like this:

[1002] "RobotA's task: inspecting items"

[1003] "RobotB's task: Assembling parts"

[1004] "Robot availability:

[1005] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[1006] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[1007] Use this data to generate a proposed work agenda and schedule.

[1008] This system will enable more efficient meetings and work within the factory, as well as improve quality.

[1009] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1010] This invention is a system that not only improves the efficiency of meeting management but also recognizes the emotions of meeting participants. This system uses generative AI and an emotion engine to support the entire meeting. The specific program processing and operation are explained below.

[1011] Automatic generation of agenda proposals

[1012] User:

[1013] The user enters the meeting topic, purpose, and related information into the system.

[1014] server:

[1015] The server receives this information and sends it to the generation AI module, which then generates a draft agenda based on this information.

[1016] Terminal (user):

[1017] The generated agenda plan is provided to the user, who then checks the plan and modifies it as necessary.

[1018] Examples:

[1019] If a user enters "new product launch meeting," the server uses generative AI to generate agenda proposals such as "1. Product overview explanation," "2. Marketing strategy," and "3. Sales plan," and displays them to the user.

[1020] Providing possible meeting dates

[1021] User:

[1022] The user enters a list of conference participants.

[1023] server:

[1024] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[1025] server:

[1026] The server uses a generation AI to generate optimal meeting date options based on available time.

[1027] Terminal (user):

[1028] The generated schedule candidates are presented to the user, who then selects the final schedule.

[1029] Examples:

[1030] Based on the schedules of participants A, B, and C, the generation AI presents candidate dates such as "next Tuesday 14:00-15:00" and "next Thursday 16:00-17:00," which the user can confirm.

[1031] Generates comments for the facilitator and a summary of the meeting content

[1032] server:

[1033] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[1034] server:

[1035] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[1036] Terminal (user):

[1037] The generative AI displays suggested comments and a summary of the discussion to the facilitator in real time.

[1038] Examples:

[1039] During the meeting, the server displays suggested comments on the device, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1040] Automatic minutes creation

[1041] server:

[1042] Once the meeting is over, the server collects all the voice data and chat logs.

[1043] server:

[1044] The collected data is sent to a generation AI, which automatically generates minutes.

[1045] Terminal (user):

[1046] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[1047] Examples:

[1048] After the meeting, the AI ​​generates minutes summarizing the meeting, such as "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy," which the user can then review.

[1049] Creating a draft for email distribution of meeting minutes

[1050] server:

[1051] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[1052] Terminal (user):

[1053] The user checks and edits the email content, and the final email is sent.

[1054] Examples:

[1055] The server generates a draft email with the subject "The minutes of the meeting will be shared as follows," and after the user confirms it, it is sent to all participants.

[1056] Collaboration with emotion engine

[1057] server:

[1058] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[1059] server:

[1060] The emotion engine recognizes participants' emotions in real time and provides that information to the generative AI.

[1061] Terminal (user):

[1062] During the meeting, participants' emotional information is displayed on the facilitator's device, allowing the facilitator to take appropriate action.

[1063] Examples:

[1064] If the emotion engine recognizes that Participant A is feeling stressed, the server will display a suggestion to the facilitator such as, "Participant A seems tense. Please add a comment to help them relax before asking for other opinions."

[1065] After the meeting, the generated minutes also include emotional data of participants recognized during the meeting, providing a detailed record of the meeting atmosphere and participants' reactions, which can be used for future improvements.

[1066] As described above, this invention is a system that combines generative AI and an emotion engine to not only improve the efficiency of meeting management but also improve the quality of discussions based on the emotions of participants.

[1067] The processing flow will be explained below.

[1068] Automatic generation of agenda proposals

[1069] Step 1:

[1070] The user enters the meeting topic, purpose, and related information.

[1071] Step 2:

[1072] The server receives this information and sends it to the generation AI module.

[1073] Step 3:

[1074] The generation AI on the server generates a draft agenda based on the input information.

[1075] Step 4:

[1076] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[1077] Providing possible meeting dates

[1078] Step 1:

[1079] The user enters a list of conference participants.

[1080] Step 2:

[1081] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[1082] Step 3:

[1083] The server uses a generation AI to generate optimal meeting date options based on available time.

[1084] Step 4:

[1085] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[1086] Generates comments for the facilitator and a summary of the meeting content

[1087] Step 1:

[1088] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[1089] Step 2:

[1090] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[1091] Step 3:

[1092] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[1093] Automatic minutes creation

[1094] Step 1:

[1095] Once the meeting is over, the server collects all the voice data and chat logs.

[1096] Step 2:

[1097] The server sends the collected data to the generation AI, which automatically generates minutes.

[1098] Step 3:

[1099] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[1100] Creating a draft for email distribution of meeting minutes

[1101] Step 1:

[1102] The server generates a mail distribution plan for the meeting participants based on the generated minutes.

[1103] Step 2:

[1104] The server generates a draft email and provides it to the user.

[1105] Step 3:

[1106] The user checks the email content and corrects it if necessary.

[1107] Step 4:

[1108] After the user's confirmation, the server delivers the final email to the conference participants.

[1109] Collaboration with emotion engine

[1110] Step 1:

[1111] Before the meeting starts, the server starts the emotion engine, which analyzes the facial expressions and voices of the meeting participants in real time.

[1112] Step 2:

[1113] During the meeting, the server collects emotion data obtained from the emotion engine and provides it to the generation AI.

[1114] Step 3:

[1115] The server's generating AI generates appropriate response suggestions for the facilitator based on the emotional state of the participants.

[1116] Step 4:

[1117] The server displays participants' emotional states and suggested responses in real time on the facilitator's terminal.

[1118] Examples:

[1119] If the emotion engine recognizes that Participant A is nervous, the server will use the generation AI to generate a suggestion such as, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions," and display this on the facilitator's device. Emotional data will also be reflected in the proposed comments and discussion summaries displayed during the meeting.

[1120] Recording and utilizing emotional data

[1121] Step 1:

[1122] After the meeting ends, the server collects all emotional data and provides it to the minutes generation AI.

[1123] Step 2:

[1124] The server uses generation AI to add emotional data to meeting minutes and generate detailed records.

[1125] Step 3:

[1126] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[1127] Examples:

[1128] The generated minutes include not only a summary of each topic, but also the emotional state of the participants at the time of each comment (e.g., "tension" or "excitement"), which allows you to review the atmosphere of the meeting and the reactions of the participants later.

[1129] In this way, the present invention is a system that combines generative AI and an emotion engine to not only make meeting management more efficient, but also improve the quality of discussions based on the emotions of participants.

[1130] Example 2

[1131] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1132] Conventional meeting management systems have the problem that it takes a lot of time and effort to prepare for the meeting, coordinate participants' schedules, create and distribute minutes, and manage the progress of the meeting. Furthermore, it is difficult to grasp the emotional state of meeting participants in real time and respond appropriately, making it difficult to improve the quality of the meeting. There is a need to solve these problems and improve the efficiency and quality of meeting management.

[1133] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1134] In this invention, the server includes a generating means for receiving information about the theme and purpose of the conference and automatically generating a proposed conference content plan based on this information, a generating means for collecting schedule information of prospective conference participants and generating optimal conference date candidates, a generating means for generating and proposing speech and comment proposals for the facilitator in real time during the conference, a generating means for collecting conference speech data and written information and automatically generating a conference record, a generating means for generating a proposal for email distribution to conference participants based on the generated conference record, and a generating means for analyzing participants' facial expressions and voices and proposing the progress of the conference based on emotion analysis.This enables efficient meeting preparation, schedule adjustment, minutes creation and distribution, and meeting progress, as well as high-quality conference management through emotion analysis.

[1135] "Information about the theme and purpose of the meeting" is data indicating the content, purpose, target, etc. of the meeting.

[1136] The "means for automatically generating a meeting agenda" refers to a device or system that has the function of generating a meeting agenda based on input information.

[1137] "Schedule information of participants in the conference" is data related to the calendars and schedules of the participants.

[1138] The "means for generating optimal conference date candidates" refers to a device or system that has the function of automatically proposing optimal conference dates based on the schedule information of the participants.

[1139] The "means for generating and proposing speech comment proposals for the facilitator in real time" is a device or system that generates and displays speech proposals in real time so that the facilitator can appropriately conduct the meeting during the meeting.

[1140] "Speech data and written information" refers to document data such as voice data and chat logs recorded during a conference.

[1141] The "means for automatically generating meeting records" refers to a device or system that has the function of automatically creating minutes of a meeting based on speech data and written information.

[1142] The "means for generating a proposal for email distribution to conference participants" is a device or system that creates a template email for distributing the contents of the conference to participants by email based on the generated minutes.

[1143] The "generation means for analyzing facial expressions and voice and proposing meeting progress based on emotional analysis" is a device or system that has the function of analyzing the facial expressions and voice data of participants and proposing meeting progress based on their emotional state.

[1144] This invention is a system that uses a generative AI model and an emotion analysis engine to improve the efficiency and quality of meetings. This system exchanges data between a server, a terminal, and a user, and supports the management of meetings using various generation methods.

[1145] Automatic generation of agenda proposals

[1146] The user inputs the topic of the meeting (e.g., "launch meeting for a new product"), the purpose (e.g., "explanation of the product overview"), and related information into the terminal.

[1147] The server sends the information received from the user as a prompt to a generative AI model (e.g., GPT-3). The generative AI model generates a draft agenda based on the input information and sends it back to the server.

[1148] The terminal (user) provides the generated agenda proposal to the user, who then modifies it as necessary.

[1149] Example: If a user inputs "launch meeting for a new product," the server requests and receives agenda proposals such as "1. Product overview explanation," "2. Marketing strategy," and "3. Sales plan" from the generative AI model.

[1150] Providing possible meeting dates

[1151] The user inputs a list of conference participants into the terminal.

[1152] The server uses APIs to collect participants' schedule information from corporate calendars and personal calendar systems (e.g., Google Calendar). Based on the collected data, a generative AI model generates optimal meeting date candidates.

[1153] The terminal (user) presents the generated schedule candidates to the user, who then selects the final schedule.

[1154] Example: Based on the schedules of participants A, B, and C, suggest possible dates such as "next Tuesday 14:00-15:00" and "next Thursday 16:00-17:00."

[1155] Generates comments for the facilitator and a summary of the meeting content

[1156] The server collects audio data and chat logs (e.g., Zoom transcripts) in real time during the meeting.

[1157] The generative AI model generates proposed comments and discussion progression suggestions in real time based on the collected data and sends them back to the server.

[1158] The terminal (user) displays the generated proposed remarks and summaries to the facilitator.

[1159] Example: Displaying suggested comments during a meeting, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1160] Automatic minutes creation

[1161] The server collects all voice data and chat logs once the conference is over.

[1162] The generative AI model automatically generates minutes based on the collected data and sends them back to the server.

[1163] The terminal (user) provides the generated minutes to the user, who checks the contents and corrects them as necessary.

[1164] Example: After a meeting, a generative AI model generates a summary such as, "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy."

[1165] Creating a draft for email distribution of meeting minutes

[1166] The server uses a generative AI model based on the generated minutes to create email suggestions for sending to meeting participants.

[1167] The terminal (user) provides the email content to the user, who then checks the content and corrects it if necessary.

[1168] The user finally clicks the "Send" button to deliver the email.

[1169] Example: Generate a draft email with the subject "We will share the meeting minutes as follows," and send it to all participants after the user has confirmed it.

[1170] Collaboration with emotion engine

[1171] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[1172] The emotion engine provides analyzed emotion information to the generative AI model.

[1173] The generative AI model proposes meeting progress based on emotional information and sends it back to the server.

[1174] The terminal (user) displays emotional information on the facilitator's terminal in real time.

[1175] Example: If the emotion engine recognizes that participant A is nervous, it will display a suggestion such as "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions." After the meeting, emotion data is added to the generated minutes.

[1176] The above is an embodiment of the present invention. This system makes it possible to prepare for a meeting, adjust schedules, create and distribute minutes, improve the efficiency of the meeting, and manage a high-quality meeting through emotion analysis.

[1177] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1178] Step 1:

[1179] The user inputs the topic, purpose and related information of the meeting into the terminal.

[1180] Input: Meeting topic (e.g., "New product launch meeting"), purpose (e.g., "Product overview"), and related information.

[1181] Specific action: The user types the topic, purpose, and related information into an input form.

[1182] Output: The meeting topic, purpose, and related information are saved as data on the device.

[1183] Step 2:

[1184] The server sends the information received from the user to the generative AI model as a prompt sentence.

[1185] Input: User-entered meeting topic, purpose, and related information.

[1186] Data processing: The server generates a prompt based on this information.

[1187] Specific behavior: The server creates a prompt sentence, "New product launch meeting," and sends it to the generative AI model.

[1188] Output: A prompt to the generative AI model.

[1189] Step 3:

[1190] The generative AI model generates a draft agenda based on the prompt text and sends it back to the server.

[1191] Input: The prompt text sent by the server.

[1192] Data computation: A generative AI model analyzes the prompt and generates a draft agenda.

[1193] Specific operation: The generative AI model creates agenda proposals such as "1. Product overview description," "2. Marketing strategy," and "3. Sales plan."

[1194] Output: The generated draft agenda.

[1195] Step 4:

[1196] The server transmits the generated agenda plan to the terminal.

[1197] Input: Agenda proposal returned by the generative AI model.

[1198] Specific operation: The server receives the agenda proposal and sends it to the user's device.

[1199] Output: The proposed agenda displayed on the user's device.

[1200] Step 5:

[1201] The terminal (user) checks the generated agenda plan and corrects it if necessary.

[1202] Input: Proposed agenda displayed on terminal.

[1203] What happens: User reviews the draft agenda and makes revisions using the editor.

[1204] Output: Revised proposed agenda.

[1205] Step 6:

[1206] A user inputs a list of conference participants into a terminal.

[1207] Input: Participant information such as name and email address.

[1208] Specific behavior: The user types participant information into an input form.

[1209] Output: The participant list is saved as data on the device.

[1210] Step 7:

[1211] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[1212] Input: Participant list.

[1213] Data processing: The server uses the API to collect participants' schedule information.

[1214] Specific operation: The server calls the Google Calendar API or similar to obtain the schedule information of participants.

[1215] Output: Collected schedule information.

[1216] Step 8:

[1217] Based on the schedule information collected by the server, the generative AI model generates optimal candidate meeting dates.

[1218] Input: Collected schedule information.

[1219] Data calculation: A generative AI model analyzes schedule information and generates optimal schedule candidates.

[1220] Specific operation: The generative AI model creates candidate dates such as "Next Tuesday 14:00-15:00" or "Next Thursday 16:00-17:00."

[1221] Output: The generated candidate dates.

[1222] Step 9:

[1223] The terminal (user) presents the generated schedule candidates to the user, who then selects the final schedule.

[1224] Input: Generated candidate dates.

[1225] Specific operation: Candidate dates are displayed on the user's screen, and the user selects the desired date.

[1226] Output: The final selected dates.

[1227] Step 10:

[1228] The server collects the audio data and chat logs of the conference.

[1229] Input: Audio data and chat logs generated during the conference.

[1230] Data processing: The server collects these data in real time.

[1231] Specific operation: The server uses a speech recognition API to convert the voice data into text and saves it together with the chat log.

[1232] Output: Collected meeting data.

[1233] Step 11:

[1234] The generative AI model generates suggested comments and summaries in real time based on the meeting's audio data and chat logs.

[1235] Input: Audio data and chat logs.

[1236] Data computation: Generative AI models analyze data and generate suggested comments and discussion summaries.

[1237] What it does: The generative AI model makes suggestions such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1238] Output: Generated comment proposals and summaries.

[1239] Step 12:

[1240] The terminal (user) displays the generated speech comment proposals and summaries to the facilitator.

[1241] Input: Generated comment proposals and summaries.

[1242] Specific operation: Proposed comments and summaries are displayed in real time on the facilitator's screen.

[1243] Output: Displayed information for the facilitator.

[1244] Step 13:

[1245] The server collects all voice data and chat logs after the conference ends.

[1246] Input: Audio data and chat logs generated during the meeting.

[1247] Data processing: The server collects these data again at the end of the meeting and manages them centrally.

[1248] What happens: The server uses the speech recognition API again to retrieve the voice and chat data.

[1249] Output: All collected meeting data.

[1250] Step 14:

[1251] The generative AI model automatically generates meeting minutes based on the collected data.

[1252] Input: Audio data and chat logs.

[1253] Data computation: A generative AI model analyzes data and generates meeting minutes.

[1254] What it does: The generative AI model creates a summary such as, "Today's agenda item 1 explained the details of the new product, and agenda item 2 discussed the marketing strategy."

[1255] Output: The generated transcript.

[1256] Step 15:

[1257] The terminal (user) provides the generated minutes to the user, who then checks and modifies the contents.

[1258] Input: The generated minutes.

[1259] What happens: The minutes are displayed on the user's screen and the user can make edits using the editor.

[1260] Output: Revised minutes.

[1261] Step 16:

[1262] The server uses a generative AI model based on the generated minutes to create email suggestions for participants.

[1263] Input: The generated minutes.

[1264] Data calculation: A generative AI model generates email delivery suggestions based on the meeting minutes.

[1265] What happens: The generative AI model creates an email with the subject line "We're sharing the meeting minutes as follows."

[1266] Output: The generated email delivery proposal.

[1267] Step 17:

[1268] The terminal (user) provides the email content to the user, who then checks, modifies, and distributes the content.

[1269] Input: The generated email proposal.

[1270] Specific behavior: The draft text is displayed in the user's email client and can be edited. The user clicks the send button and the email is sent to all participants.

[1271] Output: The email delivered.

[1272] Step 18:

[1273] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[1274] Input: Facial expression and voice data of meeting participants.

[1275] Data calculation: The emotion engine analyzes the data and generates emotion information.

[1276] Specific operation: The server calls the emotion engine API and starts analysis.

[1277] Output: The generated emotion information.

[1278] Step 19:

[1279] A generative AI model suggests meeting progress based on emotional information.

[1280] Input: Generated emotion information.

[1281] Data computation: A generative AI model analyzes emotional information and generates suggestions for how to proceed with the meeting.

[1282] What happens: The generative AI model makes suggestions like, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions."

[1283] Output: The generated meeting progress proposal.

[1284] Step 20:

[1285] The terminal (user) displays the generated conference progress proposal to the user.

[1286] Input: The generated meeting progress proposal.

[1287] What happens: The suggestion will appear on the facilitator's screen.

[1288] Output: Displayed suggestion information for the facilitator.

[1289] Step 21:

[1290] The generative AI model analyzes emotional data after the meeting and adds it to the minutes.

[1291] Input: Parsed emotion information and meeting data.

[1292] Data computation: Generative AI models analyze emotional data and incorporate it into meeting minutes.

[1293] Specific operation: The generative AI model adds information such as "Participant A seemed nervous during the meeting" to the minutes.

[1294] Output: Meeting transcripts with sentiment data added.

[1295] In this way, the system streamlines conference management and supports high-quality conference progress through each processing step.

[1296] (Application example 2)

[1297] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1298] Managing a meeting requires a great deal of time and effort, including creating an agenda, arranging meeting dates, providing real-time support, and creating minutes. Furthermore, it is necessary to understand participants' emotions in real time and devise ways to improve the quality of the meeting. In particular, in busy environments such as factories, there is a demand for systems that can efficiently manage these elements.

[1299] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1300] In this invention, the server includes a generating means for receiving information on the theme and purpose of the meeting and automatically generating a proposed meeting agenda based on this information, a generating means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, a generating means for generating and proposing proposed remarks for the facilitator in real time during the meeting, a generating means for collecting meeting remark data and automatically generating minutes, a generating means for generating email distribution proposals to the meeting participants based on the generated minutes, and a generating means for analyzing the facial expressions and voices of the participants during the meeting to recognize their emotions and supporting the progress of the meeting based on this information. This makes it possible to run the meeting more efficiently and further improve the quality of the meeting through real-time emotion recognition.

[1301] "Receiving information about the subject and purpose of the meeting" refers to obtaining data input from the user regarding the subject and purpose of the meeting.

[1302] An "agenda proposal" refers to a plan that summarizes the items and topics necessary for the meeting to proceed.

[1303] "Generation means" refers to a means that allows new information or data to be created using a generation method or device.

[1304] "Schedule information" refers to data related to the schedules and time management of each person scheduled to attend the meeting.

[1305] "Potential Meeting Dates" refers to possible options for dates and times for holding a conference.

[1306] "Suggested comments" refers to AI suggestions for comments or instructions that the facilitator should make during a meeting.

[1307] "Real-time" refers to processing or operations occurring continuously in actual time without delay.

[1308] "Speech data" is data indicating the content of oral statements made during a conference.

[1309] "Minutes" refers to a document that records what was discussed and what decisions were made at a meeting.

[1310] "Email distribution proposal" refers to a format that proposes the content of the email to be sent to participants after the meeting.

[1311] "Recognizing emotions by analyzing participants' facial expressions and voices" means analyzing the facial expressions and tone of voice of meeting participants to identify the emotions they are feeling.

[1312] This invention is a system aimed at streamlining meeting management, providing various functions using generative AI models and emotion engines, particularly for efficient meeting management in factory environments.

[1313] First, the user inputs information about the theme and purpose of the meeting into the system. The server receives this information and uses a generative AI model to generate a proposed agenda for the meeting. The generated agenda is provided to the user on their device, who can make any necessary revisions. For example, if the user inputs "New product development meeting," the server will generate a proposed agenda such as "1. Introduction of the new product," "2. Sharing of market research results," and "3. Confirmation of the development schedule."

[1314] Next, the server collects schedule information from the people planning to attend the meeting. The server obtains this information from corporate and personal calendar systems and generates optimal meeting date candidates. For example, based on the schedules of participants A, B, and C, the AI ​​generator might present candidate dates such as "next Tuesday, 2:00 PM - 3:00 PM" or "next Thursday, 4:00 PM - 5:00 PM."

[1315] During the meeting, the server collects the voice input and chat logs of the meeting in real time and generates suggested comments and discussion progress suggestions for the facilitator. This information is displayed on the device in real time. For example, the server generates suggested comments in real time, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1316] After the meeting ends, the server collects all audio data and chat logs and automatically generates meeting minutes using a generative AI model. The generated minutes are provided to the user on their device and can be revised as needed. For example, the server generates minutes summarizing the meeting, such as "Today's agenda item 1 explained the details of the new product, and agenda item 2 discussed marketing strategies," and the user can review them.

[1317] Furthermore, the server generates a proposal for sending emails to the meeting participants based on the generated minutes. The user can then check and modify the email content, and the final email is sent to the participants.

[1318] It also works in conjunction with an emotion engine. During a meeting, the server analyzes participants' facial expressions and voices to recognize their emotions in real time. This information is displayed on the facilitator's device and helps support the progress of the meeting. For example, if the emotion engine recognizes that one participant is feeling stressed, it will make a suggestion such as, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions."

[1319] The hardware used includes computers, smart devices, and devices with cameras. The software includes the OpenAI API, a calendar integration module, and a proprietary emotion engine. The generative AI model uses prompts as input and generates the necessary information.

[1320] For example, by inputting a prompt such as "New product development meeting," the system automatically generates a draft agenda and meeting minutes. It also suggests optimal meeting dates based on the participants' schedules, provides support during the meeting, and analyzes participants' sentiment.

[1321] In this way, the present invention provides a system for improving the efficiency and quality of conference management even in busy environments such as factories.

[1322] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1323] Step 1: Enter the meeting theme and purpose and automatically generate a draft agenda

[1324] The user inputs information about the meeting's theme and purpose into the device. The server receives this information and sends prompts to the generative AI model. The generative AI model generates a draft agenda and returns the results to the server. The server displays the draft agenda on the device, allowing the user to review and modify it.

[1325] Input: Meeting topic and purpose (e.g., "New product development meeting")

[1326] Output: Agenda proposal (e.g., "1. Introduce new product," "2. Share market research results," "3. Confirm development schedule")

[1327] How it works: The user enters information into the device, the server sends it to the generative AI model, which then generates a draft agenda and displays the results on the device.

[1328] Step 2: Collect schedules of meeting participants and generate candidate meeting dates

[1329] The server collects schedule information of meeting participants from corporate and personal calendar systems. Based on the collected data, it uses a generative AI model to propose optimal meeting date candidates. The server then displays the generated date candidates on the device for the user to confirm.

[1330] Input: List of expected attendees (e.g., Participants A, B, C) and their schedule information

[1331] Output: Best possible dates (e.g. "Next Tuesday 14:00-15:00" or "Next Thursday 16:00-17:00")

[1332] Specific operation: The server collects schedule information from the calendar system, analyzes it, and generates schedule suggestions. The results are displayed on the device for the user to confirm.

[1333] Step 3: Real-time comment proposals and discussion direction suggestions

[1334] During the meeting, the server collects the meeting's voice input and chat logs in real time. The generative AI model uses this data to generate suggested comments and discussion progress suggestions for the facilitator. The generated suggestions are displayed on the device in real time.

[1335] Input: Meeting audio data and chat logs

[1336] Output: Proposals for comments and discussion progression (e.g., "Let's move on to the next topic" or "Let's hear the opinions of other participants")

[1337] Specific operation: The server collects and analyzes voice data and chat logs, and the generative AI model generates suggested comments and displays them on the device.

[1338] Step 4: Automatic meeting transcript generation

[1339] When the meeting ends, the server collects all audio data and chat logs and sends them to the generative AI model. The generative AI model then summarizes the meeting content and automatically generates minutes. The generated minutes are displayed on the device, allowing the user to review and edit the content.

[1340] Input: Collected meeting audio data and chat logs

[1341] Output: Generated meeting minutes (e.g., "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy.")

[1342] Specific operation: After the server collects voice data and chat logs, it uses a generative AI model to generate minutes, which are then displayed on the device for the user to review and edit.

[1343] Step 5: Generate email suggestions based on the meeting minutes

[1344] The server uses a generative AI model to create email proposals to send to meeting participants based on the generated minutes. The generated email proposals are displayed on the device, where the user can check and edit the content before finally sending them.

[1345] Input: Generated minutes

[1346] Output: Proposal for email distribution (e.g. "The minutes of the meeting will be shared as follows")

[1347] Specific operation: The server generates an email distribution proposal based on the minutes and displays it on the terminal. After the user confirms and corrects it, the proposal is distributed.

[1348] Step 6: Recognizing participants' emotions and supporting the progress of the meeting using an emotion engine

[1349] The server collects participants' facial expressions and voices through cameras and microphones during the meeting, and analyzes them with an emotion engine to recognize their emotions. Based on the recognized emotional information, a generative AI model generates support suggestions for the progress of the meeting and displays them on the device in real time.

[1350] Input: Participant's facial expression and voice data

[1351] Output: Emotion recognition results and meeting facilitation suggestions (e.g., "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions.")

[1352] Specific operation: The server collects data from the camera and microphone, analyzes it with an emotion engine, and generates progress support measures using a generative AI model based on the recognized emotions and displays them on the device.

[1353] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1354] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1355] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1356] [Third embodiment]

[1357] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1358] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1359] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1360] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1361] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1362] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1363] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1364] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1365] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1366] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1367] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1368] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1369] The present invention is a system that utilizes generative AI to improve the efficiency of meeting management, and its program and processing will be explained based on specific procedures.

[1370] 1. Automatic generation of agenda proposals

[1371] User:

[1372] The user inputs the meeting topic, purpose, relevant information, etc. into the system.

[1373] server:

[1374] The server receives this information and sends it to an internal generation AI, which generates a draft agenda based on this data.

[1375] Terminal (user):

[1376] The generated agenda plan is provided to the user, who can then modify and confirm it as necessary.

[1377] Examples:

[1378] If a user types in "kickoff meeting for a new project," the server uses generation AI to generate a draft agenda, including "1. Project overview," "2. Setting main goals and timeline," and "3. Confirming the roles and responsibilities of each person involved," and displays it to the user.

[1379] 2. Providing possible meeting dates

[1380] User:

[1381] The user inputs a list of members to attend the meeting.

[1382] server:

[1383] The server collects each participant's schedule information from the company's calendar system and uses generation AI to create optimal meeting date options.

[1384] Terminal (user):

[1385] The generated schedule candidates are presented to the user, who then selects and confirms the final schedule.

[1386] Examples:

[1387] Based on the schedules of participants A, B, and C, the generation AI proposes possible dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00" and displays them to the user.

[1388] 3. Generate comments for the facilitator and a summary of the meeting content

[1389] server:

[1390] Once the conference begins, the server analyzes the conference's voice input and chat log in real time.

[1391] Terminal (user):

[1392] The server provides the facilitator with real-time suggestions for comments and discussions, and also summarizes the meeting content and displays it to the facilitator.

[1393] Examples:

[1394] During the meeting, the server displays suggested comments on the device, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[1395] 4. Automatic minutes creation

[1396] server:

[1397] After the meeting ends, the server collects all audio data and chat logs, and automatically generates minutes using a generation AI.

[1398] Terminal (user):

[1399] The generated minutes are provided to the user, who can then review and edit the contents.

[1400] Examples:

[1401] After the meeting, the generative AI generates minutes summarizing the main points and decisions of the meeting, and presents the user with a document stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later."

[1402] 5. Create a draft of email distribution of meeting minutes

[1403] server:

[1404] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[1405] Terminal (user):

[1406] The user checks and edits the email content, and then sends the final email.

[1407] Examples:

[1408] The server creates a draft email containing the minutes of the meeting with the subject "We will share the minutes of the meeting as follows" and displays it to the user. After the user confirms it, the generation AI sends the email to all participants.

[1409] As described above, this invention is a system that utilizes generative AI to provide integrated support for meeting preparation, progress, discussion stimulation, and the creation and distribution of meeting minutes. This system significantly improves the efficiency of meeting management, saves resources, and improves meeting quality.

[1410] The processing flow will be explained below.

[1411] Automatic generation of agenda proposals

[1412] Step 1:

[1413] The user enters the meeting topic, purpose, and related information.

[1414] Step 2:

[1415] The server receives input information from the user and sends it to the generation AI module.

[1416] Step 3:

[1417] The generation AI on the server generates a draft agenda based on the input information.

[1418] Step 4:

[1419] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[1420] Providing possible meeting dates

[1421] Step 1:

[1422] The user enters a list of conference participants.

[1423] Step 2:

[1424] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[1425] Step 3:

[1426] The server uses a generation AI to generate optimal meeting date options based on available time.

[1427] Step 4:

[1428] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[1429] Generates comments for the facilitator and a summary of the meeting content

[1430] Step 1:

[1431] When the meeting starts, the server collects the meeting's voice input and chat log in real time.

[1432] Step 2:

[1433] The server uses generative AI to create suggested comments for the facilitator and suggestions for how to proceed with the discussion.

[1434] Step 3:

[1435] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[1436] Automatic minutes creation

[1437] Step 1:

[1438] After the meeting ends, the server collects all audio data and chat logs.

[1439] Step 2:

[1440] The server sends the collected data to the generation AI, which automatically generates minutes.

[1441] Step 3:

[1442] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[1443] Creating a draft for email distribution of meeting minutes

[1444] Step 1:

[1445] The server creates a proposal for sending emails to the meeting participants based on the generated minutes.

[1446] Step 2:

[1447] The server generates a draft email and provides it to the user.

[1448] Step 3:

[1449] The user checks the email content and corrects it if necessary.

[1450] Step 4:

[1451] After the user's confirmation, the server delivers the final email to the conference participants.

[1452] Example 1

[1453] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1454] Preparing for, conducting, and post-conference administrative tasks require time and effort, and efficiency is especially important for large-scale meetings. Furthermore, manual tasks such as scheduling and creating minutes can easily lead to errors, potentially lowering the quality of the meeting. Given this current situation, it is necessary to utilize generative AI models to centrally manage meeting operations and improve efficiency.

[1455] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1456] In this invention, the server includes means for receiving information about the theme and purpose of the meeting and automatically generating a meeting agenda based on this information, means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, and means for generating and proposing speech and comment proposals for the facilitator in real time during the meeting. This allows for centralized management of meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving efficiency.

[1457] "Information regarding the theme and purpose of the meeting" is information necessary to identify the content of the meeting, and mainly includes the theme, purpose, and related information of the meeting.

[1458] "Schedule information" is information indicating the schedule of people who plan to attend a conference, and refers to data acquired from a calendar system or the like.

[1459] A "generative AI model" is an artificial intelligence model that generates draft agendas, draft remarks, minutes, etc. based on information about the theme and purpose of a meeting.

[1460] "Minutes" are documents that record the contents of a meeting, statements made, decisions made, etc., and are automatically generated.

[1461] The "email distribution proposal" is a proposal for the content of emails to be sent to the meeting participants, which is created based on the generated minutes.

[1462] "Biometric authentication" is a technology that uses biological characteristics such as fingerprint authentication and facial authentication to verify a user's identity.

[1463] A "user" refers to a person who operates the system and whose role is to input the theme and purpose of the meeting, a list of participants, and so on.

[1464] A "facilitator" is a person whose role is to proceed and manage the meeting, and who guides the discussion by referring to suggestions and summaries from the system.

[1465] A "prompt sentence" is an input sentence given to a generative AI model, which contains instructions for the model to generate a specific output.

[1466] This invention is a system that utilizes generative AI models to improve the efficiency of meeting management. The program of this system is mainly composed of a server, terminals, and users, each of which plays a specific role.

[1467] Automatic generation of agenda proposals

[1468] The user inputs the meeting theme, purpose, and related information into the device. The server receives this information and sends it to a generation AI (e.g., OpenAI's GPT-4). The generation AI generates a draft agenda based on this data. The generated draft agenda is again sent to the device via the server and provided to the user. For example, if the user inputs "kickoff meeting for a new project," the generation AI will generate a draft agenda including "1. Overview of the project," "2. Setting of main goals and timeline," and "3. Confirmation of roles and responsibilities of each person involved," and display it to the user.

[1469] Example prompt sentence:

[1470] Create an agenda for the kickoff meeting for a new project.

[1471] Providing possible meeting dates

[1472] The user inputs a list of members participating in the meeting into the device. The server collects schedule information for each participant from the company's calendar system (e.g., Google Calendar, Microsoft Outlook). The collected schedule information is sent to the generation AI, which generates optimal meeting date candidates. These date candidates are presented to the user via the server, who selects and confirms the final date. For example, based on the schedules of participants A, B, and C, the generation AI proposes candidate dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00."

[1473] Example prompt sentence:

[1474] Please suggest the best possible meeting date based on the schedules of participants A, B, and C.

[1475] Generates comments for the facilitator and a summary of the meeting content

[1476] When a meeting begins, the device collects the meeting's voice input and chat log in real time and sends it to the server. The server analyzes this data in real time and sends it to a generation AI to generate suggested comments and a summary of the meeting content. The generated suggested comments and summaries are displayed on the device via the server and used by the facilitator to guide the meeting. For example, during a meeting, the server displays suggested comments on the device, such as "It's time to move on to the next agenda item" or "We should ask for other opinions on this agenda item."

[1477] Example prompt sentence:

[1478] Analyze meeting audio data and chat logs in real time and provide suggested comments for the facilitator.

[1479] Automatic minutes creation

[1480] When the meeting ends, the server collects all audio data and chat logs and sends them to the generation AI, which automatically generates minutes. The generated minutes are then sent to the device via the server, where the user can review and edit the contents. For example, after a meeting, the generation AI generates minutes stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later," and presents them to the user.

[1481] Example prompt sentence:

[1482] Automatically generate meeting minutes based on meeting audio data and chat logs.

[1483] Creating a draft for email distribution of meeting minutes

[1484] Based on the generated minutes, the server sends a prompt to the generation AI to generate a proposed email to send to the meeting participants. This proposed email is sent to the terminal via the server, where the user can confirm and modify it, and then finally send the email. For example, the server creates a proposed email with the subject "We will share the minutes of the meeting as follows" and displays it to the user.

[1485] Example prompt sentence:

[1486] Please create a proposal for emailing meeting minutes in the following format.

[1487] This system utilizes generative AI models to centrally manage and efficiently support meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving the overall efficiency and quality of meeting management.

[1488] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1489] Automatic generation of agenda proposals

[1490] Step 1:

[1491] The user inputs the theme, purpose, and related information of the meeting into an input form on the terminal. In this example, the user inputs information on the theme "kick-off meeting for a new project."

[1492] Input: Meeting topic, objectives, relevant information

[1493] Output: Initial input data sent from the device to the server

[1494] Step 2:

[1495] The server receives the input data and sends it to the generation AI, which then formats the input data appropriately as a prompt and provides it to the generation AI.

[1496] Input: Initial input data from the user

[1497] Output: Formatted prompt to send to the generation AI

[1498] Step 3:

[1499] The AI ​​generates a draft agenda based on the prompts, such as "1. Overview of the project" and "2. Setting main goals and timelines."

[1500] Input: Server formatted prompt

[1501] Output: The generated agenda proposal

[1502] Step 4:

[1503] The server receives the output from the generation AI and sends the proposed agenda to the terminal.

[1504] Input: Agenda proposal from the generative AI

[1505] Output: Draft agenda displayed on terminal

[1506] Step 5:

[1507] The user checks the agenda proposal displayed on the device and makes corrections or additions as necessary, for example, adding new items or modifying existing items.

[1508] Input: Generated agenda proposal

[1509] Output: Revised and confirmed agenda proposal

[1510] Providing possible meeting dates

[1511] Step 1:

[1512] The user inputs the list of participants in the conference into the terminal. For example, the user inputs the names and email addresses of "Participants A, B, and C."

[1513] Input: Participant list

[1514] Output: Participant data sent from the device to the server

[1515] Step 2:

[1516] The server accesses the company's calendar system (e.g., Google Calendar, Microsoft Outlook) and collects schedule information for the specified participants.

[1517] Input: Participant data

[1518] Output: Collected schedule information

[1519] Step 3:

[1520] The server sends the schedule information as a prompt to the generation AI, which then generates optimal meeting date candidates. For example, it uses the prompt "Tell me a date when everyone is free."

[1521] Input: Collected schedule information

[1522] Output: Generated itinerary candidates

[1523] Step 4:

[1524] The server receives the output of the generated AI and sends the proposed dates to the terminal.

[1525] Input: Generated date candidates

[1526] Output: Date options displayed on the terminal

[1527] Step 5:

[1528] The user checks the schedule options displayed on the device and selects and confirms the final date.

[1529] Input: Generated date candidates

[1530] Output: Confirmed meeting dates

[1531] Generates comments for the facilitator and a summary of the meeting content

[1532] Step 1:

[1533] When a meeting starts, the device collects the meeting's voice input and chat log in real time and sends them to the server, where the voice data is converted into text using automatic speech recognition (ASR) technology.

[1534] Input: Conference audio input, chat log

[1535] Output: Text data sent to the server

[1536] Step 2:

[1537] The server analyzes the received data in real time and sends it to the generating AI.

[1538] Input: Voice-to-text data, chat logs

[1539] Output: The prompt sent to the generation AI

[1540] Step 3:

[1541] The generative AI generates suggested comments and summaries of meeting content, such as "It's time to move on to the next agenda item" or "We should seek other opinions on this topic."

[1542] Input: The prompt sent to the generation AI

[1543] Output: Generated comment proposals and a summary of the meeting content

[1544] Step 4:

[1545] The server receives the output of the generated AI and sends proposed comments and meeting summaries to the device.

[1546] Input: Generated speech and comment proposals, summary of the meeting content

[1547] Output: Proposed comments and summary displayed on terminal

[1548] Step 5:

[1549] The facilitator checks the proposed comments and meeting summary displayed on the terminal and leads the discussion.

[1550] Input: Generated speech and comment proposals, summary of the meeting content

[1551] Output: Moderated meeting

[1552] Automatic minutes creation

[1553] Step 1:

[1554] When the meeting ends, the server collects all audio data and chat logs.

[1555] Input: Audio data of completed meetings, chat logs

[1556] Output: Data stored on the server

[1557] Step 2:

[1558] The server sends the collected data to the generation AI and requests it to generate minutes.

[1559] Input: Accumulated meeting data

[1560] Output: The prompt sent to the generation AI

[1561] Step 3:

[1562] The AI ​​automatically generates minutes based on the provided data, including key points and decisions.

[1563] Input: The prompt sent to the generation AI

[1564] Output: Auto-generated meeting transcript

[1565] Step 4:

[1566] The server receives the generated minutes and sends them to the terminal.

[1567] Input: Auto-generated meeting minutes

[1568] Output: Transcript displayed on terminal

[1569] Step 5:

[1570] The user checks the minutes on the terminal and corrects them as necessary.

[1571] Input: Auto-generated meeting minutes

[1572] Output: Corrected and confirmed minutes

[1573] Creating a draft for email distribution of meeting minutes

[1574] Step 1:

[1575] Based on the generated minutes, the server creates a prompt to request the AI ​​to generate email delivery suggestions.

[1576] Input: Amended and confirmed minutes

[1577] Output: prompt statement

[1578] Step 2:

[1579] The server sends the prompt text to the generation AI, which generates an email delivery proposal.

[1580] Input: prompt statement

[1581] Output: Generated email delivery proposal

[1582] Step 3:

[1583] The server receives the generated email delivery proposal and sends it to the terminal.

[1584] Input: Generated email delivery proposal

[1585] Output: Email delivery proposal displayed on the terminal

[1586] Step 4:

[1587] The user checks the email delivery proposal on the device and modifies and sends it as necessary.

[1588] Input: Generated email delivery proposal

[1589] Output: Confirmed email delivery proposal

[1590] (Application example 1)

[1591] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1592] Meeting management involves a lot of time-consuming and labor-intensive tasks, such as creating agendas, scheduling meetings, and creating and distributing minutes. Similarly, robots and workers working together in factories also require a lot of time and effort, such as creating work plans, managing progress, and creating and sharing work records. There is a need for a system that can streamline these tasks, save resources, and improve work quality.

[1593] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1594] In this invention, the server includes a generating means for receiving information on the theme and purpose of the meeting and automatically generating a proposed meeting agenda based on this information, a generating means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, a generating means for generating and proposing proposed comments for the facilitator in real time during the meeting, a generating means for receiving information on the theme and purpose of the work and automatically generating a proposed work agenda based on this information, a generating means for collecting schedule information of the workers and generating an optimal work schedule, a generating means for generating and displaying feedback on the progress of the work in real time while the work is in progress, a generating means for collecting work comment data and automatically generating work records, and a generating means for generating proposed email distribution to work participants based on the generated work records. This makes it possible to improve the efficiency of meetings and work in the factory.

[1595] A "theme" refers to the central subject of discussion or activity in a meeting or task.

[1596] "Purpose" refers to the goal or outcome you are trying to achieve when conducting a meeting or task.

[1597] "Information" refers to data and knowledge related to the progress of meetings and work.

[1598] "Generator" refers to a technical device or software that produces a particular output based on specified input.

[1599] "Schedule information" refers to data that includes the schedules of conference participants and workers.

[1600] A "facilitator" is a person whose role is to help the meeting proceed and facilitate smooth discussion.

[1601] "Suggested remarks and comments" are sentences that provide advice or instructions to encourage appropriate remarks during meetings or work.

[1602] "Real-time" refers to immediate processing and feedback on the spot.

[1603] Minutes are a document that records the progress and decisions made at a meeting.

[1604] "Email distribution proposal" refers to the proposed content of the email to be sent to participants in a meeting or task.

[1605] An "agenda proposal" is a plan that shows the topics and order of proceedings in a meeting or task.

[1606] "Worker" refers to the person or robot who actually performs the work in a factory or work site.

[1607] A "schedule" is a plan that allocates events by time.

[1608] "Progress" refers to the progress of work or meetings.

[1609] "Feedback" is information that provides immediate evaluation and advice regarding the progress of a task or meeting.

[1610] "Work records" refer to documents or data that record the work content and progress results.

[1611] In order to put this invention into practice, it is necessary to build a system that combines the above-described generating means. The specific program and processing method of this system will be described below.

[1612] The server first receives information about the meeting's theme and purpose. This information is entered by the user through their device. The server then uses a generative AI model to automatically generate a meeting agenda proposal based on this information. OpenAI's model is a suitable model for this purpose. This model generates an appropriate agenda proposal based on the input information.

[1613] Next, the server collects schedule information for meeting participants. In this step, it obtains data from corporate calendar systems and individual scheduling systems and uses a generative AI model to propose optimal meeting dates. Again, using OpenAI's generative model enables highly accurate schedule adjustments.

[1614] During the meeting, the server generates and suggests comments for the facilitator in real time. This is done in combination with speech analysis technology, and the generative AI model suggests appropriate comments. For example, it provides specific comments in real time, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[1615] For work within the factory, the server receives information about the work theme and objectives and automatically generates a proposed work agenda. This is also entered by the user via a terminal, and the proposed agenda is created by the generative AI model. Similarly, worker schedule information is collected and an optimal work schedule is generated.

[1616] As work progresses, the server generates real-time feedback on the progress and displays it to the worker via their device. This allows them to make appropriate improvements and adjustments to the work in progress immediately. For example, feedback such as "when to proceed to the next step" or "points to note to improve work efficiency" could be considered.

[1617] After the meeting ends, the server collects all voice data and chat logs, and automatically generates minutes and work records using a generative AI model. Based on the generated minutes and work records, the server then creates email distribution proposals for meeting participants and workers, which are then sent after the user has confirmed and revised them.

[1618] For example, to coordinate robots that inspect items and assemble parts in a factory, you could enter:

[1619] "RobotA's task: inspecting items"

[1620] "RobotB's task: Assembling parts"

[1621] "Robot availability:

[1622] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[1623] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[1624] Use this data to generate a proposed work agenda and schedule.

[1625] By inputting these prompts into a generative AI model, optimal agendas and schedules are proposed, improving work efficiency. This system enables the efficiency and quality of meetings and factory work to be improved.

[1626] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1627] Step 1:

[1628] Users enter information about the topic and purpose of meetings and tasks

[1629] Users input information about the topic and purpose of the meeting or task through their terminal. This information becomes the initial input data for the system. For example, they can input content such as "kickoff meeting for a new project" or "inspection and assembly of parts."

[1630] Input: Theme and purpose of the meeting or task

[1631] Output: Initial information sent to the server

[1632] Step 2:

[1633] The server creates the agenda proposal using a generative AI model

[1634] Based on the received information, the server sends prompts to the generative AI model to generate a proposed agenda. OpenAI's model is used as the generative AI model. For example, for the topic "Kickoff meeting for a new project," the server generates agenda proposals such as "Project overview" and "Setting main goals and timeline."

[1635] Input: The topic and purpose of the meeting or task entered by the user

[1636] Output: Agenda proposal obtained from the generative AI model

[1637] Step 3:

[1638] The server collects schedule information for meeting participants and workers.

[1639] The server collects schedule information for meeting participants and workers from the company's calendar system and uses this information to create prompts for generating optimal meeting dates and work schedules.

[1640] Input: List of meeting participants and workers

[1641] Output: Collected schedule information

[1642] Step 4:

[1643] The server creates prompts for schedule adjustment and creates a schedule proposal using a generative AI model.

[1644] Based on the collected schedule information, the server creates prompts to send to the generative AI model, which then generates optimal meeting date candidates and work schedule proposals. For example, it might present suggestions such as "next Monday 14:00-15:00" or "next Wednesday 16:00-17:00."

[1645] Input: Collected schedule information

[1646] Output: Optimal schedule proposal obtained from the generative AI model

[1647] Step 5:

[1648] The user reviews the generated agenda and schedule and makes any necessary revisions.

[1649] The user checks the agenda and schedule proposals provided by the server on their device and makes any necessary corrections. This process of checking and correcting determines the final agenda and schedule.

[1650] Input: Generated agenda and schedule proposals

[1651] Output: Confirmed agenda and schedule as reviewed and modified by the user

[1652] Step 6:

[1653] The server analyzes the progress of meetings and tasks in real time and provides suggestions and feedback for the facilitator.

[1654] As meetings and tasks progress, the server collects voice input and real-time data and analyzes it with a generative AI model. This provides real-time suggestions for comments to the facilitator and feedback to participants. For example, it provides specific suggestions such as "It's time to move on to the next agenda item" or "It's time to move on to the next step."

[1655] Input: Real-time voice input and real-time data collected in real time

[1656] Output: Proposed comments and feedback obtained from the generative AI model

[1657] Step 7:

[1658] Once a meeting or work session is over, the server automatically creates minutes and work records using a generation AI model.

[1659] Once a meeting or task is completed, the server collects all audio data and text logs, analyzes them with a generative AI model, and automatically creates meeting minutes and task logs, generating documents that summarize key points and decisions.

[1660] Input: Audio data and text logs of completed meetings and tasks

[1661] Output: Meeting minutes and work records obtained from generative AI models

[1662] Step 8:

[1663] The server creates an email delivery plan based on the generated minutes and work records, and the email is sent after the user checks and modifies it.

[1664] The server creates a proposal for sending emails to meeting participants and workers based on the generated minutes and work records. Users can check and modify this proposal on their devices, and the emails are finally sent.

[1665] Input: Generated minutes and work logs

[1666] Output: The email delivered after being reviewed and corrected by the user

[1667] Specifically, for coordinating robots that inspect items and assemble parts in a factory, the prompt might look like this:

[1668] "RobotA's task: inspecting items"

[1669] "RobotB's task: Assembling parts"

[1670] "Robot availability:

[1671] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[1672] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[1673] Use this data to generate a proposed work agenda and schedule.

[1674] This system will enable more efficient meetings and work within the factory, as well as improve quality.

[1675] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1676] This invention is a system that not only improves the efficiency of meeting management but also recognizes the emotions of meeting participants. This system uses generative AI and an emotion engine to support the entire meeting. The specific program processing and operation are explained below.

[1677] Automatic generation of agenda proposals

[1678] User:

[1679] The user enters the meeting topic, purpose, and related information into the system.

[1680] server:

[1681] The server receives this information and sends it to the generation AI module, which then generates a draft agenda based on this information.

[1682] Terminal (user):

[1683] The generated agenda plan is provided to the user, who then checks the plan and modifies it as necessary.

[1684] Examples:

[1685] If a user enters "new product launch meeting," the server uses generative AI to generate agenda proposals such as "1. Product overview explanation," "2. Marketing strategy," and "3. Sales plan," and displays them to the user.

[1686] Providing possible meeting dates

[1687] User:

[1688] The user enters a list of conference participants.

[1689] server:

[1690] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[1691] server:

[1692] The server uses a generation AI to generate optimal meeting date options based on available time.

[1693] Terminal (user):

[1694] The generated schedule candidates are presented to the user, who then selects the final schedule.

[1695] Examples:

[1696] Based on the schedules of participants A, B, and C, the generation AI presents candidate dates such as "next Tuesday 14:00-15:00" and "next Thursday 16:00-17:00," which the user can confirm.

[1697] Generates comments for the facilitator and a summary of the meeting content

[1698] server:

[1699] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[1700] server:

[1701] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[1702] Terminal (user):

[1703] The generative AI displays suggested comments and a summary of the discussion to the facilitator in real time.

[1704] Examples:

[1705] During the meeting, the server displays suggested comments on the device, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1706] Automatic minutes creation

[1707] server:

[1708] Once the meeting is over, the server collects all the voice data and chat logs.

[1709] server:

[1710] The collected data is sent to a generation AI, which automatically generates minutes.

[1711] Terminal (user):

[1712] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[1713] Examples:

[1714] After the meeting, the AI ​​generates minutes summarizing the meeting, such as "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy," which the user can then review.

[1715] Creating a draft for email distribution of meeting minutes

[1716] server:

[1717] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[1718] Terminal (user):

[1719] The user checks and edits the email content, and the final email is sent.

[1720] Examples:

[1721] The server generates a draft email with the subject "The minutes of the meeting will be shared as follows," and after the user confirms it, it is sent to all participants.

[1722] Collaboration with emotion engine

[1723] server:

[1724] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[1725] server:

[1726] The emotion engine recognizes participants' emotions in real time and provides that information to the generative AI.

[1727] Terminal (user):

[1728] During the meeting, participants' emotional information is displayed on the facilitator's device, allowing the facilitator to take appropriate action.

[1729] Examples:

[1730] If the emotion engine recognizes that Participant A is feeling stressed, the server will display a suggestion to the facilitator such as, "Participant A seems tense. Please add a comment to help them relax before asking for other opinions."

[1731] After the meeting, the generated minutes also include emotional data of participants recognized during the meeting, providing a detailed record of the meeting atmosphere and participants' reactions, which can be used for future improvements.

[1732] As described above, this invention is a system that combines generative AI and an emotion engine to not only improve the efficiency of meeting management but also improve the quality of discussions based on the emotions of participants.

[1733] The processing flow will be explained below.

[1734] Automatic generation of agenda proposals

[1735] Step 1:

[1736] The user enters the meeting topic, purpose, and related information.

[1737] Step 2:

[1738] The server receives this information and sends it to the generation AI module.

[1739] Step 3:

[1740] The generation AI on the server generates a draft agenda based on the input information.

[1741] Step 4:

[1742] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[1743] Providing possible meeting dates

[1744] Step 1:

[1745] The user enters a list of conference participants.

[1746] Step 2:

[1747] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[1748] Step 3:

[1749] The server uses a generation AI to generate optimal meeting date options based on available time.

[1750] Step 4:

[1751] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[1752] Generates comments for the facilitator and a summary of the meeting content

[1753] Step 1:

[1754] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[1755] Step 2:

[1756] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[1757] Step 3:

[1758] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[1759] Automatic minutes creation

[1760] Step 1:

[1761] Once the meeting is over, the server collects all the voice data and chat logs.

[1762] Step 2:

[1763] The server sends the collected data to the generation AI, which automatically generates minutes.

[1764] Step 3:

[1765] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[1766] Creating a draft for email distribution of meeting minutes

[1767] Step 1:

[1768] The server generates a mail distribution plan for the meeting participants based on the generated minutes.

[1769] Step 2:

[1770] The server generates a draft email and provides it to the user.

[1771] Step 3:

[1772] The user checks the email content and corrects it if necessary.

[1773] Step 4:

[1774] After the user's confirmation, the server delivers the final email to the conference participants.

[1775] Collaboration with emotion engine

[1776] Step 1:

[1777] Before the meeting starts, the server starts the emotion engine, which analyzes the facial expressions and voices of the meeting participants in real time.

[1778] Step 2:

[1779] During the meeting, the server collects emotion data obtained from the emotion engine and provides it to the generation AI.

[1780] Step 3:

[1781] The server's generating AI generates appropriate response suggestions for the facilitator based on the emotional state of the participants.

[1782] Step 4:

[1783] The server displays participants' emotional states and suggested responses in real time on the facilitator's terminal.

[1784] Examples:

[1785] If the emotion engine recognizes that Participant A is nervous, the server will use the generation AI to generate a suggestion such as, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions," and display this on the facilitator's device. Emotional data will also be reflected in the proposed comments and discussion summaries displayed during the meeting.

[1786] Recording and utilizing emotional data

[1787] Step 1:

[1788] After the meeting ends, the server collects all emotional data and provides it to the minutes generation AI.

[1789] Step 2:

[1790] The server uses generation AI to add emotional data to meeting minutes and generate detailed records.

[1791] Step 3:

[1792] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[1793] Examples:

[1794] The generated minutes include not only a summary of each topic, but also the emotional state of the participants at the time of each comment (e.g., "tension" or "excitement"), which allows you to review the atmosphere of the meeting and the reactions of the participants later.

[1795] In this way, the present invention is a system that combines generative AI and an emotion engine to not only make meeting management more efficient, but also improve the quality of discussions based on the emotions of participants.

[1796] Example 2

[1797] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1798] Conventional meeting management systems have the problem that it takes a lot of time and effort to prepare for the meeting, coordinate participants' schedules, create and distribute minutes, and manage the progress of the meeting. Furthermore, it is difficult to grasp the emotional state of meeting participants in real time and respond appropriately, making it difficult to improve the quality of the meeting. There is a need to solve these problems and improve the efficiency and quality of meeting management.

[1799] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1800] In this invention, the server includes a generating means for receiving information about the theme and purpose of the conference and automatically generating a proposed conference content plan based on this information, a generating means for collecting schedule information of prospective conference participants and generating optimal conference date candidates, a generating means for generating and proposing speech and comment proposals for the facilitator in real time during the conference, a generating means for collecting conference speech data and written information and automatically generating a conference record, a generating means for generating a proposal for email distribution to conference participants based on the generated conference record, and a generating means for analyzing participants' facial expressions and voices and proposing the progress of the conference based on emotion analysis.This enables efficient meeting preparation, schedule adjustment, minutes creation and distribution, and meeting progress, as well as high-quality conference management through emotion analysis.

[1801] "Information about the theme and purpose of the meeting" is data indicating the content, purpose, target, etc. of the meeting.

[1802] The "means for automatically generating a meeting agenda" refers to a device or system that has the function of generating a meeting agenda based on input information.

[1803] "Schedule information of participants in the conference" is data related to the calendars and schedules of the participants.

[1804] The "means for generating optimal conference date candidates" refers to a device or system that has the function of automatically proposing optimal conference dates based on the schedule information of the participants.

[1805] The "means for generating and proposing speech comment proposals for the facilitator in real time" is a device or system that generates and displays speech proposals in real time so that the facilitator can appropriately conduct the meeting during the meeting.

[1806] "Speech data and written information" refers to document data such as voice data and chat logs recorded during a conference.

[1807] The "means for automatically generating meeting records" refers to a device or system that has the function of automatically creating minutes of a meeting based on speech data and written information.

[1808] The "means for generating a proposal for email distribution to conference participants" is a device or system that creates a template email for distributing the contents of the conference to participants by email based on the generated minutes.

[1809] The "generation means for analyzing facial expressions and voice and proposing meeting progress based on emotional analysis" is a device or system that has the function of analyzing the facial expressions and voice data of participants and proposing meeting progress based on their emotional state.

[1810] This invention is a system that uses a generative AI model and an emotion analysis engine to improve the efficiency and quality of meetings. This system exchanges data between a server, a terminal, and a user, and supports the management of meetings using various generation methods.

[1811] Automatic generation of agenda proposals

[1812] The user inputs the topic of the meeting (e.g., "launch meeting for a new product"), the purpose (e.g., "explanation of the product overview"), and related information into the terminal.

[1813] The server sends the information received from the user as a prompt to a generative AI model (e.g., GPT-3). The generative AI model generates a draft agenda based on the input information and sends it back to the server.

[1814] The terminal (user) provides the generated agenda proposal to the user, who then modifies it as necessary.

[1815] Example: If a user inputs "launch meeting for a new product," the server requests and receives agenda proposals such as "1. Product overview explanation," "2. Marketing strategy," and "3. Sales plan" from the generative AI model.

[1816] Providing possible meeting dates

[1817] The user inputs a list of conference participants into the terminal.

[1818] The server uses APIs to collect participants' schedule information from corporate calendars and personal calendar systems (e.g., Google Calendar). Based on the collected data, a generative AI model generates optimal meeting date candidates.

[1819] The terminal (user) presents the generated schedule candidates to the user, who then selects the final schedule.

[1820] Example: Based on the schedules of participants A, B, and C, suggest possible dates such as "next Tuesday 14:00-15:00" and "next Thursday 16:00-17:00."

[1821] Generates comments for the facilitator and a summary of the meeting content

[1822] The server collects audio data and chat logs (e.g., Zoom transcripts) in real time during the meeting.

[1823] The generative AI model generates proposed comments and discussion progression suggestions in real time based on the collected data and sends them back to the server.

[1824] The terminal (user) displays the generated proposed remarks and summaries to the facilitator.

[1825] Example: Displaying suggested comments during a meeting, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1826] Automatic minutes creation

[1827] The server collects all voice data and chat logs once the conference is over.

[1828] The generative AI model automatically generates minutes based on the collected data and sends them back to the server.

[1829] The terminal (user) provides the generated minutes to the user, who checks the contents and corrects them as necessary.

[1830] Example: After a meeting, a generative AI model generates a summary such as, "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy."

[1831] Creating a draft for email distribution of meeting minutes

[1832] The server uses a generative AI model based on the generated minutes to create email suggestions for sending to meeting participants.

[1833] The terminal (user) provides the email content to the user, who then checks the content and corrects it if necessary.

[1834] The user finally clicks the "Send" button to deliver the email.

[1835] Example: Generate a draft email with the subject "We will share the meeting minutes as follows," and send it to all participants after the user has confirmed it.

[1836] Collaboration with emotion engine

[1837] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[1838] The emotion engine provides analyzed emotion information to the generative AI model.

[1839] The generative AI model proposes meeting progress based on emotional information and sends it back to the server.

[1840] The terminal (user) displays emotional information on the facilitator's terminal in real time.

[1841] Example: If the emotion engine recognizes that participant A is nervous, it will display a suggestion such as "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions." After the meeting, emotion data is added to the generated minutes.

[1842] The above is an embodiment of the present invention. This system makes it possible to prepare for a meeting, adjust schedules, create and distribute minutes, improve the efficiency of the meeting, and manage a high-quality meeting through emotion analysis.

[1843] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1844] Step 1:

[1845] The user inputs the topic, purpose and related information of the meeting into the terminal.

[1846] Input: Meeting topic (e.g., "New product launch meeting"), purpose (e.g., "Product overview"), and related information.

[1847] Specific action: The user types the topic, purpose, and related information into an input form.

[1848] Output: The meeting topic, purpose, and related information are saved as data on the device.

[1849] Step 2:

[1850] The server sends the information received from the user to the generative AI model as a prompt sentence.

[1851] Input: User-entered meeting topic, purpose, and related information.

[1852] Data processing: The server generates a prompt based on this information.

[1853] Specific behavior: The server creates a prompt sentence, "New product launch meeting," and sends it to the generative AI model.

[1854] Output: A prompt to the generative AI model.

[1855] Step 3:

[1856] The generative AI model generates a draft agenda based on the prompt text and sends it back to the server.

[1857] Input: The prompt text sent by the server.

[1858] Data computation: A generative AI model analyzes the prompt and generates a draft agenda.

[1859] Specific operation: The generative AI model creates agenda proposals such as "1. Product overview description," "2. Marketing strategy," and "3. Sales plan."

[1860] Output: The generated draft agenda.

[1861] Step 4:

[1862] The server transmits the generated agenda plan to the terminal.

[1863] Input: Agenda proposal returned by the generative AI model.

[1864] Specific operation: The server receives the agenda proposal and sends it to the user's device.

[1865] Output: The proposed agenda displayed on the user's device.

[1866] Step 5:

[1867] The terminal (user) checks the generated agenda plan and corrects it if necessary.

[1868] Input: Proposed agenda displayed on terminal.

[1869] What happens: User reviews the draft agenda and makes revisions using the editor.

[1870] Output: Revised proposed agenda.

[1871] Step 6:

[1872] A user inputs a list of conference participants into a terminal.

[1873] Input: Participant information such as name and email address.

[1874] Specific behavior: The user types participant information into an input form.

[1875] Output: The participant list is saved as data on the device.

[1876] Step 7:

[1877] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[1878] Input: Participant list.

[1879] Data processing: The server uses the API to collect participants' schedule information.

[1880] Specific operation: The server calls the Google Calendar API or similar to obtain the schedule information of participants.

[1881] Output: Collected schedule information.

[1882] Step 8:

[1883] Based on the schedule information collected by the server, the generative AI model generates optimal candidate meeting dates.

[1884] Input: Collected schedule information.

[1885] Data calculation: A generative AI model analyzes schedule information and generates optimal schedule candidates.

[1886] Specific operation: The generative AI model creates candidate dates such as "Next Tuesday 14:00-15:00" or "Next Thursday 16:00-17:00."

[1887] Output: The generated candidate dates.

[1888] Step 9:

[1889] The terminal (user) presents the generated schedule candidates to the user, who then selects the final schedule.

[1890] Input: Generated candidate dates.

[1891] Specific operation: Candidate dates are displayed on the user's screen, and the user selects the desired date.

[1892] Output: The final selected dates.

[1893] Step 10:

[1894] The server collects the audio data and chat logs of the conference.

[1895] Input: Audio data and chat logs generated during the conference.

[1896] Data processing: The server collects these data in real time.

[1897] Specific operation: The server uses a speech recognition API to convert the voice data into text and saves it together with the chat log.

[1898] Output: Collected meeting data.

[1899] Step 11:

[1900] The generative AI model generates suggested comments and summaries in real time based on the meeting's audio data and chat logs.

[1901] Input: Audio data and chat logs.

[1902] Data computation: Generative AI models analyze data and generate suggested comments and discussion summaries.

[1903] What it does: The generative AI model makes suggestions such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1904] Output: Generated comment proposals and summaries.

[1905] Step 12:

[1906] The terminal (user) displays the generated speech comment proposals and summaries to the facilitator.

[1907] Input: Generated comment proposals and summaries.

[1908] Specific operation: Proposed comments and summaries are displayed in real time on the facilitator's screen.

[1909] Output: Displayed information for the facilitator.

[1910] Step 13:

[1911] The server collects all voice data and chat logs after the conference ends.

[1912] Input: Audio data and chat logs generated during the meeting.

[1913] Data processing: The server collects these data again at the end of the meeting and manages them centrally.

[1914] What happens: The server uses the speech recognition API again to retrieve the voice and chat data.

[1915] Output: All collected meeting data.

[1916] Step 14:

[1917] The generative AI model automatically generates meeting minutes based on the collected data.

[1918] Input: Audio data and chat logs.

[1919] Data computation: A generative AI model analyzes data and generates meeting minutes.

[1920] What it does: The generative AI model creates a summary such as, "Today's agenda item 1 explained the details of the new product, and agenda item 2 discussed the marketing strategy."

[1921] Output: The generated transcript.

[1922] Step 15:

[1923] The terminal (user) provides the generated minutes to the user, who then checks and modifies the contents.

[1924] Input: The generated minutes.

[1925] What happens: The minutes are displayed on the user's screen and the user can make edits using the editor.

[1926] Output: Revised minutes.

[1927] Step 16:

[1928] The server uses a generative AI model based on the generated minutes to create email suggestions for participants.

[1929] Input: The generated minutes.

[1930] Data calculation: A generative AI model generates email delivery suggestions based on the meeting minutes.

[1931] What happens: The generative AI model creates an email with the subject line "We're sharing the meeting minutes as follows."

[1932] Output: The generated email delivery proposal.

[1933] Step 17:

[1934] The terminal (user) provides the email content to the user, who then checks, modifies, and distributes the content.

[1935] Input: The generated email proposal.

[1936] Specific behavior: The draft text is displayed in the user's email client and can be edited. The user clicks the send button and the email is sent to all participants.

[1937] Output: The email delivered.

[1938] Step 18:

[1939] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[1940] Input: Facial expression and voice data of meeting participants.

[1941] Data calculation: The emotion engine analyzes the data and generates emotion information.

[1942] Specific operation: The server calls the emotion engine API and starts analysis.

[1943] Output: The generated emotion information.

[1944] Step 19:

[1945] A generative AI model suggests meeting progress based on emotional information.

[1946] Input: Generated emotion information.

[1947] Data computation: A generative AI model analyzes emotional information and generates suggestions for how to proceed with the meeting.

[1948] What happens: The generative AI model makes suggestions like, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions."

[1949] Output: The generated meeting progress proposal.

[1950] Step 20:

[1951] The terminal (user) displays the generated conference progress proposal to the user.

[1952] Input: The generated meeting progress proposal.

[1953] What happens: The suggestion will appear on the facilitator's screen.

[1954] Output: Displayed suggestion information for the facilitator.

[1955] Step 21:

[1956] The generative AI model analyzes emotional data after the meeting and adds it to the minutes.

[1957] Input: Parsed emotion information and meeting data.

[1958] Data computation: Generative AI models analyze emotional data and incorporate it into meeting minutes.

[1959] Specific operation: The generative AI model adds information such as "Participant A seemed nervous during the meeting" to the minutes.

[1960] Output: Meeting transcripts with sentiment data added.

[1961] In this way, the system streamlines conference management and supports high-quality conference progress through each processing step.

[1962] (Application example 2)

[1963] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1964] Managing a meeting requires a great deal of time and effort, including creating an agenda, arranging meeting dates, providing real-time support, and creating minutes. Furthermore, it is necessary to understand participants' emotions in real time and devise ways to improve the quality of the meeting. In particular, in busy environments such as factories, there is a demand for systems that can efficiently manage these elements.

[1965] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1966] In this invention, the server includes a generating means for receiving information on the theme and purpose of the meeting and automatically generating a proposed meeting agenda based on this information, a generating means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, a generating means for generating and proposing proposed remarks for the facilitator in real time during the meeting, a generating means for collecting meeting remark data and automatically generating minutes, a generating means for generating email distribution proposals to the meeting participants based on the generated minutes, and a generating means for analyzing the facial expressions and voices of the participants during the meeting to recognize their emotions and supporting the progress of the meeting based on this information. This makes it possible to run the meeting more efficiently and further improve the quality of the meeting through real-time emotion recognition.

[1967] "Receiving information about the subject and purpose of the meeting" refers to obtaining data input from the user regarding the subject and purpose of the meeting.

[1968] An "agenda proposal" refers to a plan that summarizes the items and topics necessary for the meeting to proceed.

[1969] "Generation means" refers to a means that allows new information or data to be created using a generation method or device.

[1970] "Schedule information" refers to data related to the schedules and time management of each person scheduled to attend the meeting.

[1971] "Potential Meeting Dates" refers to possible options for dates and times for holding a conference.

[1972] "Suggested comments" refers to AI suggestions for comments or instructions that the facilitator should make during a meeting.

[1973] "Real-time" refers to processing or operations occurring continuously in actual time without delay.

[1974] "Speech data" is data indicating the content of oral statements made during a conference.

[1975] "Minutes" refers to a document that records what was discussed and what decisions were made at a meeting.

[1976] "Email distribution proposal" refers to a format that proposes the content of the email to be sent to participants after the meeting.

[1977] "Recognizing emotions by analyzing participants' facial expressions and voices" means analyzing the facial expressions and tone of voice of meeting participants to identify the emotions they are feeling.

[1978] This invention is a system aimed at streamlining meeting management, providing various functions using generative AI models and emotion engines, particularly for efficient meeting management in factory environments.

[1979] First, the user inputs information about the theme and purpose of the meeting into the system. The server receives this information and uses a generative AI model to generate a proposed agenda for the meeting. The generated agenda is provided to the user on their device, who can make any necessary revisions. For example, if the user inputs "New product development meeting," the server will generate a proposed agenda such as "1. Introduction of the new product," "2. Sharing of market research results," and "3. Confirmation of the development schedule."

[1980] Next, the server collects schedule information from the people planning to attend the meeting. The server obtains this information from corporate and personal calendar systems and generates optimal meeting date candidates. For example, based on the schedules of participants A, B, and C, the AI ​​generator might present candidate dates such as "next Tuesday, 2:00 PM - 3:00 PM" or "next Thursday, 4:00 PM - 5:00 PM."

[1981] During the meeting, the server collects the voice input and chat logs of the meeting in real time and generates suggested comments and discussion progress suggestions for the facilitator. This information is displayed on the device in real time. For example, the server generates suggested comments in real time, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[1982] After the meeting ends, the server collects all audio data and chat logs and automatically generates meeting minutes using a generative AI model. The generated minutes are provided to the user on their device and can be revised as needed. For example, the server generates minutes summarizing the meeting, such as "Today's agenda item 1 explained the details of the new product, and agenda item 2 discussed marketing strategies," and the user can review them.

[1983] Furthermore, the server generates a proposal for sending emails to the meeting participants based on the generated minutes. The user can then check and modify the email content, and the final email is sent to the participants.

[1984] It also works in conjunction with an emotion engine. During a meeting, the server analyzes participants' facial expressions and voices to recognize their emotions in real time. This information is displayed on the facilitator's device and helps support the progress of the meeting. For example, if the emotion engine recognizes that one participant is feeling stressed, it will make a suggestion such as, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions."

[1985] The hardware used includes computers, smart devices, and devices with cameras. The software includes the OpenAI API, a calendar integration module, and a proprietary emotion engine. The generative AI model uses prompts as input and generates the necessary information.

[1986] For example, by inputting a prompt such as "New product development meeting," the system automatically generates a draft agenda and meeting minutes. It also suggests optimal meeting dates based on the participants' schedules, provides support during the meeting, and analyzes participants' sentiment.

[1987] In this way, the present invention provides a system for improving the efficiency and quality of conference management even in busy environments such as factories.

[1988] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1989] Step 1: Enter the meeting theme and purpose and automatically generate a draft agenda

[1990] The user inputs information about the meeting's theme and purpose into the device. The server receives this information and sends prompts to the generative AI model. The generative AI model generates a draft agenda and returns the results to the server. The server displays the draft agenda on the device, allowing the user to review and modify it.

[1991] Input: Meeting topic and purpose (e.g., "New product development meeting")

[1992] Output: Agenda proposal (e.g., "1. Introduce new product," "2. Share market research results," "3. Confirm development schedule")

[1993] How it works: The user enters information into the device, the server sends it to the generative AI model, which then generates a draft agenda and displays the results on the device.

[1994] Step 2: Collect schedules of meeting participants and generate candidate meeting dates

[1995] The server collects schedule information of meeting participants from corporate and personal calendar systems. Based on the collected data, it uses a generative AI model to propose optimal meeting date candidates. The server then displays the generated date candidates on the device for the user to confirm.

[1996] Input: List of expected attendees (e.g., Participants A, B, C) and their schedule information

[1997] Output: Best possible dates (e.g. "Next Tuesday 14:00-15:00" or "Next Thursday 16:00-17:00")

[1998] Specific operation: The server collects schedule information from the calendar system, analyzes it, and generates schedule suggestions. The results are displayed on the device for the user to confirm.

[1999] Step 3: Real-time comment proposals and discussion direction suggestions

[2000] During the meeting, the server collects the meeting's voice input and chat logs in real time. The generative AI model uses this data to generate suggested comments and discussion progress suggestions for the facilitator. The generated suggestions are displayed on the device in real time.

[2001] Input: Meeting audio data and chat logs

[2002] Output: Proposals for comments and discussion progression (e.g., "Let's move on to the next topic" or "Let's hear the opinions of other participants")

[2003] Specific operation: The server collects and analyzes voice data and chat logs, and the generative AI model generates suggested comments and displays them on the device.

[2004] Step 4: Automatic meeting transcript generation

[2005] When the meeting ends, the server collects all audio data and chat logs and sends them to the generative AI model. The generative AI model then summarizes the meeting content and automatically generates minutes. The generated minutes are displayed on the device, allowing the user to review and edit the content.

[2006] Input: Collected meeting audio data and chat logs

[2007] Output: Generated meeting minutes (e.g., "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy.")

[2008] Specific operation: After the server collects voice data and chat logs, it uses a generative AI model to generate minutes, which are then displayed on the device for the user to review and edit.

[2009] Step 5: Generate email suggestions based on the meeting minutes

[2010] The server uses a generative AI model to create email proposals to send to meeting participants based on the generated minutes. The generated email proposals are displayed on the device, where the user can check and edit the content before finally sending them.

[2011] Input: Generated minutes

[2012] Output: Proposal for email distribution (e.g. "The minutes of the meeting will be shared as follows")

[2013] Specific operation: The server generates an email distribution proposal based on the minutes and displays it on the terminal. After the user confirms and corrects it, the proposal is distributed.

[2014] Step 6: Recognizing participants' emotions and supporting the progress of the meeting using an emotion engine

[2015] The server collects participants' facial expressions and voices through cameras and microphones during the meeting, and analyzes them with an emotion engine to recognize their emotions. Based on the recognized emotional information, a generative AI model generates support suggestions for the progress of the meeting and displays them on the device in real time.

[2016] Input: Participant's facial expression and voice data

[2017] Output: Emotion recognition results and meeting facilitation suggestions (e.g., "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions.")

[2018] Specific operation: The server collects data from the camera and microphone, analyzes it with an emotion engine, and generates progress support measures using a generative AI model based on the recognized emotions and displays them on the device.

[2019] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[2020] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2021] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[2022] [Fourth embodiment]

[2023] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[2024] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[2025] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[2026] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[2027] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[2028] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[2029] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[2030] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[2031] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[2032] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[2033] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[2034] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[2035] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2036] The present invention is a system that utilizes generative AI to improve the efficiency of meeting management, and its program and processing will be explained based on specific procedures.

[2037] 1. Automatic generation of agenda proposals

[2038] User:

[2039] The user inputs the meeting topic, purpose, relevant information, etc. into the system.

[2040] server:

[2041] The server receives this information and sends it to an internal generation AI, which generates a draft agenda based on this data.

[2042] Terminal (user):

[2043] The generated agenda plan is provided to the user, who can then modify and confirm it as necessary.

[2044] Examples:

[2045] If a user types in "kickoff meeting for a new project," the server uses generation AI to generate a draft agenda, including "1. Project overview," "2. Setting main goals and timeline," and "3. Confirming the roles and responsibilities of each person involved," and displays it to the user.

[2046] 2. Providing possible meeting dates

[2047] User:

[2048] The user inputs a list of members to attend the meeting.

[2049] server:

[2050] The server collects each participant's schedule information from the company's calendar system and uses generation AI to create optimal meeting date options.

[2051] Terminal (user):

[2052] The generated schedule candidates are presented to the user, who then selects and confirms the final schedule.

[2053] Examples:

[2054] Based on the schedules of participants A, B, and C, the generation AI proposes possible dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00" and displays them to the user.

[2055] 3. Generate comments for the facilitator and a summary of the meeting content

[2056] server:

[2057] Once the conference begins, the server analyzes the conference's voice input and chat log in real time.

[2058] Terminal (user):

[2059] The server provides the facilitator with real-time suggestions for comments and discussions, and also summarizes the meeting content and displays it to the facilitator.

[2060] Examples:

[2061] During the meeting, the server displays suggested comments on the device, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[2062] 4. Automatic minutes creation

[2063] server:

[2064] After the meeting ends, the server collects all audio data and chat logs, and automatically generates minutes using a generation AI.

[2065] Terminal (user):

[2066] The generated minutes are provided to the user, who can then review and edit the contents.

[2067] Examples:

[2068] After the meeting, the generative AI generates minutes summarizing the main points and decisions of the meeting, and presents the user with a document stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later."

[2069] 5. Create a draft of email distribution of meeting minutes

[2070] server:

[2071] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[2072] Terminal (user):

[2073] The user checks and edits the email content, and then sends the final email.

[2074] Examples:

[2075] The server creates a draft email containing the minutes of the meeting with the subject "We will share the minutes of the meeting as follows" and displays it to the user. After the user confirms it, the generation AI sends the email to all participants.

[2076] As described above, this invention is a system that utilizes generative AI to provide integrated support for meeting preparation, progress, discussion stimulation, and the creation and distribution of meeting minutes. This system significantly improves the efficiency of meeting management, saves resources, and improves meeting quality.

[2077] The processing flow will be explained below.

[2078] Automatic generation of agenda proposals

[2079] Step 1:

[2080] The user enters the meeting topic, purpose, and related information.

[2081] Step 2:

[2082] The server receives input information from the user and sends it to the generation AI module.

[2083] Step 3:

[2084] The generation AI on the server generates a draft agenda based on the input information.

[2085] Step 4:

[2086] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[2087] Providing possible meeting dates

[2088] Step 1:

[2089] The user enters a list of conference participants.

[2090] Step 2:

[2091] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[2092] Step 3:

[2093] The server uses a generation AI to generate optimal meeting date options based on available time.

[2094] Step 4:

[2095] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[2096] Generates comments for the facilitator and a summary of the meeting content

[2097] Step 1:

[2098] When the meeting starts, the server collects the meeting's voice input and chat log in real time.

[2099] Step 2:

[2100] The server uses generative AI to create suggested comments for the facilitator and suggestions for how to proceed with the discussion.

[2101] Step 3:

[2102] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[2103] Automatic minutes creation

[2104] Step 1:

[2105] After the meeting ends, the server collects all audio data and chat logs.

[2106] Step 2:

[2107] The server sends the collected data to the generation AI, which automatically generates minutes.

[2108] Step 3:

[2109] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[2110] Creating a draft for email distribution of meeting minutes

[2111] Step 1:

[2112] The server creates a proposal for sending emails to the meeting participants based on the generated minutes.

[2113] Step 2:

[2114] The server generates a draft email and provides it to the user.

[2115] Step 3:

[2116] The user checks the email content and corrects it if necessary.

[2117] Step 4:

[2118] After the user's confirmation, the server delivers the final email to the conference participants.

[2119] Example 1

[2120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2121] Preparing for, conducting, and post-conference administrative tasks require time and effort, and efficiency is especially important for large-scale meetings. Furthermore, manual tasks such as scheduling and creating minutes can easily lead to errors, potentially lowering the quality of the meeting. Given this current situation, it is necessary to utilize generative AI models to centrally manage meeting operations and improve efficiency.

[2122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2123] In this invention, the server includes means for receiving information about the theme and purpose of the meeting and automatically generating a meeting agenda based on this information, means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, and means for generating and proposing speech and comment proposals for the facilitator in real time during the meeting. This allows for centralized management of meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving efficiency.

[2124] "Information regarding the theme and purpose of the meeting" is information necessary to identify the content of the meeting, and mainly includes the theme, purpose, and related information of the meeting.

[2125] "Schedule information" is information indicating the schedule of people who plan to attend a conference, and refers to data acquired from a calendar system or the like.

[2126] A "generative AI model" is an artificial intelligence model that generates draft agendas, draft remarks, minutes, etc. based on information about the theme and purpose of a meeting.

[2127] "Minutes" are documents that record the contents of a meeting, statements made, decisions made, etc., and are automatically generated.

[2128] The "email distribution proposal" is a proposal for the content of emails to be sent to the meeting participants, which is created based on the generated minutes.

[2129] "Biometric authentication" is a technology that uses biological characteristics such as fingerprint authentication and facial authentication to verify a user's identity.

[2130] A "user" refers to a person who operates the system and whose role is to input the theme and purpose of the meeting, a list of participants, and so on.

[2131] A "facilitator" is a person whose role is to proceed and manage the meeting, and who guides the discussion by referring to suggestions and summaries from the system.

[2132] A "prompt sentence" is an input sentence given to a generative AI model, which contains instructions for the model to generate a specific output.

[2133] This invention is a system that utilizes generative AI models to improve the efficiency of meeting management. The program of this system is mainly composed of a server, terminals, and users, each of which plays a specific role.

[2134] Automatic generation of agenda proposals

[2135] The user inputs the meeting theme, purpose, and related information into the device. The server receives this information and sends it to a generation AI (e.g., OpenAI's GPT-4). The generation AI generates a draft agenda based on this data. The generated draft agenda is again sent to the device via the server and provided to the user. For example, if the user inputs "kickoff meeting for a new project," the generation AI will generate a draft agenda including "1. Overview of the project," "2. Setting of main goals and timeline," and "3. Confirmation of roles and responsibilities of each person involved," and display it to the user.

[2136] Example prompt sentence:

[2137] Create an agenda for the kickoff meeting for a new project.

[2138] Providing possible meeting dates

[2139] The user inputs a list of members participating in the meeting into the device. The server collects schedule information for each participant from the company's calendar system (e.g., Google Calendar, Microsoft Outlook). The collected schedule information is sent to the generation AI, which generates optimal meeting date candidates. These date candidates are presented to the user via the server, who selects and confirms the final date. For example, based on the schedules of participants A, B, and C, the generation AI proposes candidate dates such as "next Monday 14:00-15:00" and "next Wednesday 16:00-17:00."

[2140] Example prompt sentence:

[2141] Please suggest the best possible meeting date based on the schedules of participants A, B, and C.

[2142] Generates comments for the facilitator and a summary of the meeting content

[2143] When a meeting begins, the device collects the meeting's voice input and chat log in real time and sends it to the server. The server analyzes this data in real time and sends it to a generation AI to generate suggested comments and a summary of the meeting content. The generated suggested comments and summaries are displayed on the device via the server and used by the facilitator to guide the meeting. For example, during a meeting, the server displays suggested comments on the device, such as "It's time to move on to the next agenda item" or "We should ask for other opinions on this agenda item."

[2144] Example prompt sentence:

[2145] Analyze meeting audio data and chat logs in real time and provide suggested comments for the facilitator.

[2146] Automatic minutes creation

[2147] When the meeting ends, the server collects all audio data and chat logs and sends them to the generation AI, which automatically generates minutes. The generated minutes are then sent to the device via the server, where the user can review and edit the contents. For example, after a meeting, the generation AI generates minutes stating, "Plan A was approved at today's meeting, and the next meeting is scheduled for two weeks later," and presents them to the user.

[2148] Example prompt sentence:

[2149] Automatically generate meeting minutes based on meeting audio data and chat logs.

[2150] Creating a draft for email distribution of meeting minutes

[2151] Based on the generated minutes, the server sends a prompt to the generation AI to generate a proposed email to send to the meeting participants. This proposed email is sent to the terminal via the server, where the user can confirm and modify it, and then finally send the email. For example, the server creates a proposed email with the subject "We will share the minutes of the meeting as follows" and displays it to the user.

[2152] Example prompt sentence:

[2153] Please create a proposal for emailing meeting minutes in the following format.

[2154] This system utilizes generative AI models to centrally manage and efficiently support meeting preparation, progress, and the creation and distribution of meeting minutes, thereby improving the overall efficiency and quality of meeting management.

[2155] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2156] Automatic generation of agenda proposals

[2157] Step 1:

[2158] The user inputs the theme, purpose, and related information of the meeting into an input form on the terminal. In this example, the user inputs information on the theme "kick-off meeting for a new project."

[2159] Input: Meeting topic, objectives, relevant information

[2160] Output: Initial input data sent from the device to the server

[2161] Step 2:

[2162] The server receives the input data and sends it to the generation AI, which then formats the input data appropriately as a prompt and provides it to the generation AI.

[2163] Input: Initial input data from the user

[2164] Output: Formatted prompt to send to the generation AI

[2165] Step 3:

[2166] The AI ​​generates a draft agenda based on the prompts, such as "1. Overview of the project" and "2. Setting main goals and timelines."

[2167] Input: Server formatted prompt

[2168] Output: The generated agenda proposal

[2169] Step 4:

[2170] The server receives the output from the generation AI and sends the proposed agenda to the terminal.

[2171] Input: Agenda proposal from the generative AI

[2172] Output: Draft agenda displayed on terminal

[2173] Step 5:

[2174] The user checks the agenda proposal displayed on the device and makes corrections or additions as necessary, for example, adding new items or modifying existing items.

[2175] Input: Generated agenda proposal

[2176] Output: Revised and confirmed agenda proposal

[2177] Providing possible meeting dates

[2178] Step 1:

[2179] The user inputs the list of participants in the conference into the terminal. For example, the user inputs the names and email addresses of "Participants A, B, and C."

[2180] Input: Participant list

[2181] Output: Participant data sent from the device to the server

[2182] Step 2:

[2183] The server accesses the company's calendar system (e.g., Google Calendar, Microsoft Outlook) and collects schedule information for the specified participants.

[2184] Input: Participant data

[2185] Output: Collected schedule information

[2186] Step 3:

[2187] The server sends the schedule information as a prompt to the generation AI, which then generates optimal meeting date candidates. For example, it uses the prompt "Tell me a date when everyone is free."

[2188] Input: Collected schedule information

[2189] Output: Generated itinerary candidates

[2190] Step 4:

[2191] The server receives the output of the generated AI and sends the proposed dates to the terminal.

[2192] Input: Generated date candidates

[2193] Output: Date options displayed on the terminal

[2194] Step 5:

[2195] The user checks the schedule options displayed on the device and selects and confirms the final date.

[2196] Input: Generated date candidates

[2197] Output: Confirmed meeting dates

[2198] Generates comments for the facilitator and a summary of the meeting content

[2199] Step 1:

[2200] When a meeting starts, the device collects the meeting's voice input and chat log in real time and sends them to the server, where the voice data is converted into text using automatic speech recognition (ASR) technology.

[2201] Input: Conference audio input, chat log

[2202] Output: Text data sent to the server

[2203] Step 2:

[2204] The server analyzes the received data in real time and sends it to the generating AI.

[2205] Input: Voice-to-text data, chat logs

[2206] Output: The prompt sent to the generation AI

[2207] Step 3:

[2208] The generative AI generates suggested comments and summaries of meeting content, such as "It's time to move on to the next agenda item" or "We should seek other opinions on this topic."

[2209] Input: The prompt sent to the generation AI

[2210] Output: Generated comment proposals and a summary of the meeting content

[2211] Step 4:

[2212] The server receives the output of the generated AI and sends proposed comments and meeting summaries to the device.

[2213] Input: Generated speech and comment proposals, summary of the meeting content

[2214] Output: Proposed comments and summary displayed on terminal

[2215] Step 5:

[2216] The facilitator checks the proposed comments and meeting summary displayed on the terminal and leads the discussion.

[2217] Input: Generated speech and comment proposals, summary of the meeting content

[2218] Output: Moderated meeting

[2219] Automatic minutes creation

[2220] Step 1:

[2221] When the meeting ends, the server collects all audio data and chat logs.

[2222] Input: Audio data of completed meetings, chat logs

[2223] Output: Data stored on the server

[2224] Step 2:

[2225] The server sends the collected data to the generation AI and requests it to generate minutes.

[2226] Input: Accumulated meeting data

[2227] Output: The prompt sent to the generation AI

[2228] Step 3:

[2229] The AI ​​automatically generates minutes based on the provided data, including key points and decisions.

[2230] Input: The prompt sent to the generation AI

[2231] Output: Auto-generated meeting transcript

[2232] Step 4:

[2233] The server receives the generated minutes and sends them to the terminal.

[2234] Input: Auto-generated meeting minutes

[2235] Output: Transcript displayed on terminal

[2236] Step 5:

[2237] The user checks the minutes on the terminal and corrects them as necessary.

[2238] Input: Auto-generated meeting minutes

[2239] Output: Corrected and confirmed minutes

[2240] Creating a draft for email distribution of meeting minutes

[2241] Step 1:

[2242] Based on the generated minutes, the server creates a prompt to request the AI ​​to generate email delivery suggestions.

[2243] Input: Amended and confirmed minutes

[2244] Output: prompt statement

[2245] Step 2:

[2246] The server sends the prompt text to the generation AI, which generates an email delivery proposal.

[2247] Input: prompt statement

[2248] Output: Generated email delivery proposal

[2249] Step 3:

[2250] The server receives the generated email delivery proposal and sends it to the terminal.

[2251] Input: Generated email delivery proposal

[2252] Output: Email delivery proposal displayed on the terminal

[2253] Step 4:

[2254] The user checks the email delivery proposal on the device and modifies and sends it as necessary.

[2255] Input: Generated email delivery proposal

[2256] Output: Confirmed email delivery proposal

[2257] (Application example 1)

[2258] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2259] Meeting management involves a lot of time-consuming and labor-intensive tasks, such as creating agendas, scheduling meetings, and creating and distributing minutes. Similarly, robots and workers working together in factories also require a lot of time and effort, such as creating work plans, managing progress, and creating and sharing work records. There is a need for a system that can streamline these tasks, save resources, and improve work quality.

[2260] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2261] In this invention, the server includes a generating means for receiving information on the theme and purpose of the meeting and automatically generating a proposed meeting agenda based on this information, a generating means for collecting schedule information of the people planning to attend the meeting and generating optimal meeting date candidates, a generating means for generating and proposing proposed comments for the facilitator in real time during the meeting, a generating means for receiving information on the theme and purpose of the work and automatically generating a proposed work agenda based on this information, a generating means for collecting schedule information of the workers and generating an optimal work schedule, a generating means for generating and displaying feedback on the progress of the work in real time while the work is in progress, a generating means for collecting work comment data and automatically generating work records, and a generating means for generating proposed email distribution to work participants based on the generated work records. This makes it possible to improve the efficiency of meetings and work in the factory.

[2262] A "theme" refers to the central subject of discussion or activity in a meeting or task.

[2263] "Purpose" refers to the goal or outcome you are trying to achieve when conducting a meeting or task.

[2264] "Information" refers to data and knowledge related to the progress of meetings and work.

[2265] "Generator" refers to a technical device or software that produces a particular output based on specified input.

[2266] "Schedule information" refers to data that includes the schedules of conference participants and workers.

[2267] A "facilitator" is a person whose role is to help the meeting proceed and facilitate smooth discussion.

[2268] "Suggested remarks and comments" are sentences that provide advice or instructions to encourage appropriate remarks during meetings or work.

[2269] "Real-time" refers to immediate processing and feedback on the spot.

[2270] Minutes are a document that records the progress and decisions made at a meeting.

[2271] "Email distribution proposal" refers to the proposed content of the email to be sent to participants in a meeting or task.

[2272] An "agenda proposal" is a plan that shows the topics and order of proceedings in a meeting or task.

[2273] "Worker" refers to the person or robot who actually performs the work in a factory or work site.

[2274] A "schedule" is a plan that allocates events by time.

[2275] "Progress" refers to the progress of work or meetings.

[2276] "Feedback" is information that provides immediate evaluation and advice regarding the progress of a task or meeting.

[2277] "Work records" refer to documents or data that record the work content and progress results.

[2278] In order to put this invention into practice, it is necessary to build a system that combines the above-described generating means. The specific program and processing method of this system will be described below.

[2279] The server first receives information about the meeting's theme and purpose. This information is entered by the user through their device. The server then uses a generative AI model to automatically generate a meeting agenda proposal based on this information. OpenAI's model is a suitable model for this purpose. This model generates an appropriate agenda proposal based on the input information.

[2280] Next, the server collects schedule information for meeting participants. In this step, it obtains data from corporate calendar systems and individual scheduling systems and uses a generative AI model to propose optimal meeting dates. Again, using OpenAI's generative model enables highly accurate schedule adjustments.

[2281] During the meeting, the server generates and suggests comments for the facilitator in real time. This is done in combination with speech analysis technology, and the generative AI model suggests appropriate comments. For example, it provides specific comments in real time, such as "It's time to move on to the next topic" or "We should ask for other opinions on this topic."

[2282] For work within the factory, the server receives information about the work theme and objectives and automatically generates a proposed work agenda. This is also entered by the user via a terminal, and the proposed agenda is created by the generative AI model. Similarly, worker schedule information is collected and an optimal work schedule is generated.

[2283] As work progresses, the server generates real-time feedback on the progress and displays it to the worker via their device. This allows them to make appropriate improvements and adjustments to the work in progress immediately. For example, feedback such as "when to proceed to the next step" or "points to note to improve work efficiency" could be considered.

[2284] After the meeting ends, the server collects all voice data and chat logs, and automatically generates minutes and work records using a generative AI model. Based on the generated minutes and work records, the server then creates email distribution proposals for meeting participants and workers, which are then sent after the user has confirmed and revised them.

[2285] For example, to coordinate robots that inspect items and assemble parts in a factory, you could enter:

[2286] "RobotA's task: inspecting items"

[2287] "RobotB's task: Assembling parts"

[2288] "Robot availability:

[2289] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[2290] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[2291] Use this data to generate a proposed work agenda and schedule.

[2292] By inputting these prompts into a generative AI model, optimal agendas and schedules are proposed, improving work efficiency. This system enables the efficiency and quality of meetings and factory work to be improved.

[2293] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2294] Step 1:

[2295] Users enter information about the topic and purpose of meetings and tasks

[2296] Users input information about the topic and purpose of the meeting or task through their terminal. This information becomes the initial input data for the system. For example, they can input content such as "kickoff meeting for a new project" or "inspection and assembly of parts."

[2297] Input: Theme and purpose of the meeting or task

[2298] Output: Initial information sent to the server

[2299] Step 2:

[2300] The server creates the agenda proposal using a generative AI model

[2301] Based on the received information, the server sends prompts to the generative AI model to generate a proposed agenda. OpenAI's model is used as the generative AI model. For example, for the topic "Kickoff meeting for a new project," the server generates agenda proposals such as "Project overview" and "Setting main goals and timeline."

[2302] Input: The topic and purpose of the meeting or task entered by the user

[2303] Output: Agenda proposal obtained from the generative AI model

[2304] Step 3:

[2305] The server collects schedule information for meeting participants and workers.

[2306] The server collects schedule information for meeting participants and workers from the company's calendar system and uses this information to create prompts for generating optimal meeting dates and work schedules.

[2307] Input: List of meeting participants and workers

[2308] Output: Collected schedule information

[2309] Step 4:

[2310] The server creates prompts for schedule adjustment and creates a schedule proposal using a generative AI model.

[2311] Based on the collected schedule information, the server creates prompts to send to the generative AI model, which then generates optimal meeting date candidates and work schedule proposals. For example, it might present suggestions such as "next Monday 14:00-15:00" or "next Wednesday 16:00-17:00."

[2312] Input: Collected schedule information

[2313] Output: Optimal schedule proposal obtained from the generative AI model

[2314] Step 5:

[2315] The user reviews the generated agenda and schedule and makes any necessary revisions.

[2316] The user checks the agenda and schedule proposals provided by the server on their device and makes any necessary corrections. This process of checking and correcting determines the final agenda and schedule.

[2317] Input: Generated agenda and schedule proposals

[2318] Output: Confirmed agenda and schedule as reviewed and modified by the user

[2319] Step 6:

[2320] The server analyzes the progress of meetings and tasks in real time and provides suggestions and feedback for the facilitator.

[2321] As meetings and tasks progress, the server collects voice input and real-time data and analyzes it with a generative AI model. This provides real-time suggestions for comments to the facilitator and feedback to participants. For example, it provides specific suggestions such as "It's time to move on to the next agenda item" or "It's time to move on to the next step."

[2322] Input: Real-time voice input and real-time data collected in real time

[2323] Output: Proposed comments and feedback obtained from the generative AI model

[2324] Step 7:

[2325] Once a meeting or work session is over, the server automatically creates minutes and work records using a generation AI model.

[2326] Once a meeting or task is completed, the server collects all audio data and text logs, analyzes them with a generative AI model, and automatically creates meeting minutes and task logs, generating documents that summarize key points and decisions.

[2327] Input: Audio data and text logs of completed meetings and tasks

[2328] Output: Meeting minutes and work records obtained from generative AI models

[2329] Step 8:

[2330] The server creates an email delivery plan based on the generated minutes and work records, and the email is sent after the user checks and modifies it.

[2331] The server creates a proposal for sending emails to meeting participants and workers based on the generated minutes and work records. Users can check and modify this proposal on their devices, and the emails are finally sent.

[2332] Input: Generated minutes and work logs

[2333] Output: The email delivered after being reviewed and corrected by the user

[2334] Specifically, for coordinating robots that inspect items and assemble parts in a factory, the prompt might look like this:

[2335] "RobotA's task: inspecting items"

[2336] "RobotB's task: Assembling parts"

[2337] "Robot availability:

[2338] RobotA: ["2023-10-01 09:00", "2023-10-01 10:00"]

[2339] RobotB: ["2023-10-01 10:00", "2023-10-01 11:00"]

[2340] Use this data to generate a proposed work agenda and schedule.

[2341] This system will enable more efficient meetings and work within the factory, as well as improve quality.

[2342] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2343] This invention is a system that not only improves the efficiency of meeting management but also recognizes the emotions of meeting participants. This system uses generative AI and an emotion engine to support the entire meeting. The specific program processing and operation are explained below.

[2344] Automatic generation of agenda proposals

[2345] User:

[2346] The user enters the meeting topic, purpose, and related information into the system.

[2347] server:

[2348] The server receives this information and sends it to the generation AI module, which then generates a draft agenda based on this information.

[2349] Terminal (user):

[2350] The generated agenda plan is provided to the user, who then checks the plan and modifies it as necessary.

[2351] Examples:

[2352] If a user enters "new product launch meeting," the server uses generative AI to generate agenda proposals such as "1. Product overview explanation," "2. Marketing strategy," and "3. Sales plan," and displays them to the user.

[2353] Providing possible meeting dates

[2354] User:

[2355] The user enters a list of conference participants.

[2356] server:

[2357] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[2358] server:

[2359] The server uses a generation AI to generate optimal meeting date options based on available time.

[2360] Terminal (user):

[2361] The generated schedule candidates are presented to the user, who then selects the final schedule.

[2362] Examples:

[2363] Based on the schedules of participants A, B, and C, the generation AI presents candidate dates such as "next Tuesday 14:00-15:00" and "next Thursday 16:00-17:00," which the user can confirm.

[2364] Generates comments for the facilitator and a summary of the meeting content

[2365] server:

[2366] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[2367] server:

[2368] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[2369] Terminal (user):

[2370] The generative AI displays suggested comments and a summary of the discussion to the facilitator in real time.

[2371] Examples:

[2372] During the meeting, the server displays suggested comments on the device, such as "Let's move on to the next topic" or "Let's hear the opinions of other participants."

[2373] Automatic minutes creation

[2374] server:

[2375] Once the meeting is over, the server collects all the voice data and chat logs.

[2376] server:

[2377] The collected data is sent to a generation AI, which automatically generates minutes.

[2378] Terminal (user):

[2379] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[2380] Examples:

[2381] After the meeting, the AI ​​generates minutes summarizing the meeting, such as "Today's agenda item 1 was a detailed explanation of the new product, and agenda item 2 was a discussion of the marketing strategy," which the user can then review.

[2382] Creating a draft for email distribution of meeting minutes

[2383] server:

[2384] Based on the generated minutes, the server uses a generation AI to create a draft email to send to meeting participants.

[2385] Terminal (user):

[2386] The user checks and edits the email content, and the final email is sent.

[2387] Examples:

[2388] The server generates a draft email with the subject "The minutes of the meeting will be shared as follows," and after the user confirms it, it is sent to all participants.

[2389] Collaboration with emotion engine

[2390] server:

[2391] The server activates an emotion engine that analyzes participants' facial expressions and voices during the meeting.

[2392] server:

[2393] The emotion engine recognizes participants' emotions in real time and provides that information to the generative AI.

[2394] Terminal (user):

[2395] During the meeting, participants' emotional information is displayed on the facilitator's device, allowing the facilitator to take appropriate action.

[2396] Examples:

[2397] If the emotion engine recognizes that Participant A is feeling stressed, the server will display a suggestion to the facilitator such as, "Participant A seems tense. Please add a comment to help them relax before asking for other opinions."

[2398] After the meeting, the generated minutes also include emotional data of participants recognized during the meeting, providing a detailed record of the meeting atmosphere and participants' reactions, which can be used for future improvements.

[2399] As described above, this invention is a system that combines generative AI and an emotion engine to not only improve the efficiency of meeting management but also improve the quality of discussions based on the emotions of participants.

[2400] The processing flow will be explained below.

[2401] Automatic generation of agenda proposals

[2402] Step 1:

[2403] The user enters the meeting topic, purpose, and related information.

[2404] Step 2:

[2405] The server receives this information and sends it to the generation AI module.

[2406] Step 3:

[2407] The generation AI on the server generates a draft agenda based on the input information.

[2408] Step 4:

[2409] The server generates a draft agenda and provides it to the user, who can review it and make any necessary changes.

[2410] Providing possible meeting dates

[2411] Step 1:

[2412] The user enters a list of conference participants.

[2413] Step 2:

[2414] The server collects participants' schedule information from corporate calendars and personal calendar systems.

[2415] Step 3:

[2416] The server uses a generation AI to generate optimal meeting date options based on available time.

[2417] Step 4:

[2418] The server presents the generated schedule candidates to the user, and the user selects the final schedule.

[2419] Generates comments for the facilitator and a summary of the meeting content

[2420] Step 1:

[2421] Once the conference is started, the server collects the conference's voice input and chat log in real time.

[2422] Step 2:

[2423] The server's generation AI creates suggested comments and suggestions for discussion progression for the facilitator.

[2424] Step 3:

[2425] The server generates real-time comment proposals and discussion summaries and displays them on the facilitator's device.

[2426] Automatic minutes creation

[2427] Step 1:

[2428] Once the meeting is over, the server collects all the voice data and chat logs.

[2429] Step 2:

[2430] The server sends the collected data to the generation AI, which automatically generates minutes.

[2431] Step 3:

[2432] The server provides the generated minutes to the user's terminal, where the user can check the minutes and make corrections as necessary.

[2433] Creating a draft for email distribution of meeting minutes

[2434] Step 1:

[2435] The server generates a mail distribution plan for the meeting participants based on the generated minutes.

[2436] Step 2:

[2437] The server generates a draft email and provides it to the user.

[2438] Step 3:

[2439] The user checks the email content and corrects it if necessary.

[2440] Step 4:

[2441] After the user's confirmation, the server delivers the final email to the conference participants.

[2442] Collaboration with emotion engine

[2443] Step 1:

[2444] Before the meeting starts, the server starts the emotion engine, which analyzes the facial expressions and voices of the meeting participants in real time.

[2445] Step 2:

[2446] During the meeting, the server collects emotion data obtained from the emotion engine and provides it to the generation AI.

[2447] Step 3:

[2448] The server's generating AI generates appropriate response suggestions for the facilitator based on the emotional state of the participants.

[2449] Step 4:

[2450] The server displays participants' emotional states and suggested responses in real time on the facilitator's terminal.

[2451] Examples:

[2452] If the emotion engine recognizes that Participant A is nervous, the server will use the generation AI to generate a suggestion such as, "Participant A seems nervous. Please add a comment to help them relax before asking for other opinions," and display this on the facilitator's device. Emotional data will also be reflected in the proposed comments and discussion summaries displayed during the meeting.

[2453] Recording and utilizing emotional data

[2454] Step 1:

[2455] After the meeting ends, the server collects all emotional data and provides it to the minutes generation AI.

[2456] Step 2:

[2457] The server uses generation AI to add emotional data to meeting minutes and generate detailed records.

[2458] Step 3:

[2459] The generated minutes are provided to the user, who checks the contents and corrects them as necessary.

[2460] Examples:

[2461] The generated minutes include not only a summary of each topic, but also the emotional state of the participants at the time of each comment (e.g., "tension" or "excitement"), which allows you to review the...

Claims

1. generating means for receiving information about the theme and purpose of the meeting and automatically generating a draft agenda for the meeting based thereon; A generating means for collecting schedule information of prospective participants of a meeting and generating optimal meeting date candidates; A generating means for generating and proposing comment proposals for the facilitator in real time during a meeting; A generating means for collecting speech data of a meeting and automatically generating minutes of the meeting; A generating means for generating a proposal for email distribution to the meeting participants based on the generated minutes; A system including:

2. 2. The system according to claim 1, further comprising means for analyzing the progress of the discussion in real time during the meeting and providing a summary of the meeting contents to the facilitator.

3. The system according to claim 1 , further comprising means for enabling a user to confirm and modify the generated minutes.

Images