System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a purpose and a goal of a conference; means for automatically generating an agenda based on the input purpose and goal; means for displaying the agenda; means for uploading related materials; means for analyzing the uploaded materials and organizing necessary information; means for providing the organized materials; means for inputting a participant list and a possible schedule range; means for confirming a schedule of a participant and proposing an optimal schedule; and means for notifying and confirming the proposed schedule.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] In traditional meetings, organizers must expend a great deal of effort in advance preparation, such as clarifying the purpose and goals of the meeting, creating an agenda, and preparing relevant materials. Furthermore, discussions can easily go off track during meetings, and progress is often stalled by participants' long speeches and position-taking. Furthermore, after the meeting, a lot of post-processing is required, such as summarizing the main points of the discussion, setting tasks for the next meeting, and reporting to participants. This series of tasks reduces the efficiency of the meeting and increases the workload of the organizer. [Means for solving the problem]

[0005] The present invention provides a system that efficiently automates the processes of meeting preparation, progress, and closing. Specifically, the system includes a means for inputting the purpose and goals of a meeting, a means for automatically generating an agenda based on the input purpose and goals, and a means for displaying the agenda. It also includes a means for uploading, analyzing, and organizing related materials, and a means for providing the organized materials. Furthermore, the system includes a means for inputting a participant list and a schedule range, a means for checking the participants' schedules and proposing the optimal schedule, and a means for notifying and confirming the proposed schedule.

[0006] At the start of a meeting, the system includes a means for projecting the agenda and meeting materials and announcing the start of the meeting via audio. It also includes a means for analyzing comments in real time, detecting position-taking and long comments and appropriately suppressing them, as well as a means for organizing the comments and extracting and providing important points. After the meeting ends, the system includes a means for summarizing the content of the discussion, setting the allocation of responsibilities and deadlines for the next task, and providing this information and reporting it to participants. In this way, a series of processes related to meetings are automated, reducing the burden on the organizer and improving meeting efficiency.

[0007] "Means for inputting the purpose and goals of the meeting" refers to an interface that allows users to input the theme of the meeting and the results they want to achieve into the system.

[0008] "Means for automatically generating an agenda" refers to a function that automatically creates a meeting plan and agenda based on the purpose and goals of the meeting.

[0009] The "means for displaying the agenda" refers to a display device such as a display or monitor for visually presenting the generated agenda to the user.

[0010] "Means for uploading related materials" refers to an interface that allows a user to send documents and files required for a conference to the system.

[0011] "Means for analyzing and organizing necessary information" refers to the function for analyzing the content of uploaded materials and extracting and organizing important information.

[0012] "Means for providing organized materials" refers to the download link and file transmission function for providing analyzed and organized materials to users.

[0013] "Means for inputting a list of participants and a range of available dates" refers to an interface for inputting the members who will be participating in the meeting and their available times into the system.

[0014] "Means to check participants' schedules and suggest the most suitable dates" refers to a function that automatically calculates and suggests the most suitable meeting dates based on the entered date range and participants' schedule information.

[0015] "Means for communicating and confirming proposed dates" refers to the functionality for implementing the process of informing participants of proposed meeting dates and obtaining their confirmation.

[0016] "Means for analyzing statements in real time and detecting position-taking or lengthy speech" refers to technology that analyzes the statements of each participant in a meeting in real time to detect biased opinions or excessive speech durations by specific individuals.

[0017] "Measures to curb prolonged speech" refers to a function that prompts a specific person to stop speaking appropriately when they have been speaking for a long time.

[0018] "Means for automatically organizing statements and extracting key points" refers to a function that automatically classifies and summarizes statements and automatically extracts important points from discussions.

[0019] "Means for providing a summary and task list" refers to a display or transmission function for listing the main points of discussion and upcoming tasks and providing them to the user.

[0020] "Means for reporting the summary and task list to participants" refers to the function of communicating the summary of the discussion and the set tasks to participants via email or notification after the meeting. [Brief explanation of the drawings]

[0021] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0022] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0023] First, the terms used in the following description will be explained.

[0024] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0025] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0026] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0027] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0028] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0029] [First embodiment]

[0030] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0031] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0032] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0033] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0034] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0035] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0036] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0037] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0038] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0039] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0040] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0041] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0042] The present invention relates to a system for automating a series of processes from advance preparation to progress and closing of a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[0043] Meeting preparation

[0044] Agenda creation

[0045] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[0046] Preparation of meeting materials

[0047] Users upload relevant materials for use in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format. The terminal then provides the converted materials to the user.

[0048] Scheduling and communication

[0049] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[0050] Conference moderator

[0051] Display projection and speech

[0052] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[0053] Speech Management

[0054] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[0055] Closing the meeting

[0056] Summary and reporting

[0057] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, and the terminal provides them to the user. Furthermore, the server sends the summary and task list to all participants using email or notifications.

[0058] Specific examples

[0059] For example, when hosting a project progress meeting, the user inputs the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task" into the system. Based on this, the server automatically generates the following agenda:

[0060] 1. Opening Remarks

[0061] 2. Project X progress report

[0062] 3. Announcement of second half plan

[0063] 4. Question and Answer Session

[0064] 5. Next task setting

[0065] 6. Closing Remarks

[0066] The user then uploads relevant materials (minutes of the previous meeting, progress reports, etc.) to the server, which analyzes them and organizes the necessary information. When the user inputs the list of meeting participants and a range of possible dates (e.g., a weekday morning next week), the server proposes the optimal date and notifies the user via the device.

[0067] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. Speech during the meeting is also analyzed in real time, and messages are displayed to limit long speeches as necessary. Furthermore, the server automatically extracts important speech content and provides it to the user.

[0068] After the meeting, the server summarizes the discussion and sets the next task, its assignment, and deadline. The summary and task list are provided to the user via their terminal and reported to all participants.

[0069] This efficiently automates the entire meeting process, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[0070] The processing flow will be explained below.

[0071] Meeting preparation

[0072] Agenda creation

[0073] Step 1:

[0074] The user inputs the purpose and goals of the meeting using the system's interface.

[0075] Step 2:

[0076] The server receives the input objectives and goals, analyzes the information, and automatically generates an agenda based on the analysis results.

[0077] Step 3:

[0078] The server sends the generated agenda to the terminal.

[0079] Step 4:

[0080] The terminal displays the received agenda to the user, who then checks the displayed agenda.

[0081] Preparation of meeting materials

[0082] Step 1:

[0083] The user uploads the relevant materials to be used in the meeting to the server via the terminal.

[0084] Step 2:

[0085] The server scans the uploaded materials and extracts and organizes the necessary information.

[0086] Step 3:

[0087] The server converts the organized material into a downloadable format.

[0088] Step 4:

[0089] The terminal provides the converted material to the user, who then downloads and checks the provided material.

[0090] Scheduling and communication

[0091] Step 1:

[0092] The user inputs the list of participants and the available dates. The user inputs the list using the system interface.

[0093] Step 2:

[0094] The server receives the entered data and checks the schedules of all participants.

[0095] Step 3:

[0096] The server performs the analysis and suggests the best meeting date.

[0097] Step 4:

[0098] The server sends the proposed itinerary to the terminal.

[0099] Step 5:

[0100] The terminal notifies the user of the proposed schedule, and the user confirms the schedule and makes a final decision.

[0101] Step 6:

[0102] The server will notify all participants of the confirmed schedule.

[0103] Conference moderator

[0104] Display projection and speech

[0105] Step 1:

[0106] The server projects the agenda created at the start of the meeting and the prepared meeting materials onto a display.

[0107] Step 2:

[0108] The server uses voice synthesis technology to announce the start of the conference.

[0109] Speech Management

[0110] Step 1:

[0111] The server analyzes each participant's comments in real time.

[0112] Step 2:

[0113] The server detects position talk and long speech.

[0114] Step 3:

[0115] If a long speech is detected, the server sends that information to the terminal.

[0116] Step 4:

[0117] The terminal displays a message to the user at an appropriate timing to prevent long-term speech.

[0118] Organizing the discussion

[0119] Step 1:

[0120] The server automatically organizes the comments in real time.

[0121] Step 2:

[0122] The server extracts the key points.

[0123] Step 3:

[0124] The server transmits the extracted points to the terminal.

[0125] Step 4:

[0126] The terminal displays the extracted points to the user.

[0127] Closing the meeting

[0128] Summary and reporting

[0129] Step 1:

[0130] The server summarizes the discussion.

[0131] Step 2:

[0132] The server sets the allocation of roles and deadlines for the next task.

[0133] Step 3:

[0134] The server sends the summarized content and task list to the terminal.

[0135] Step 4:

[0136] The terminal provides the summary and task list to the user, who then checks the information provided.

[0137] Step 5:

[0138] The server notifies all participants of the same information.

[0139] Example 1

[0140] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0141] In traditional meeting management, the entire process of preparing, conducting, and closing a meeting is largely manual, requiring time and effort. It can also be difficult for all participants to quickly share necessary information, reducing meeting efficiency. Furthermore, tasks such as managing comments and organizing discussions during a meeting are time-consuming, making it difficult to capture important points.

[0142] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0143] In this invention, the server includes means for inputting the purpose and goals of the meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information using natural language processing, means for providing the organized materials, means for inputting a list of participants and a range of possible dates, means for checking the schedules of participants and proposing the optimal date using a generative AI model, and means for notifying and confirming the proposed date. This efficiently automates a series of processes from advance preparation to progress and closing of the meeting, reducing the burden on the organizer and enabling all participants to quickly share necessary information.

[0144] "Means for inputting the purpose and goals of the meeting" refers to an interface that allows the user to input the purpose of the meeting and the specific goals to be achieved.

[0145] "Means for automatically generating an agenda based on input purpose and goals" refers to a function that analyzes the purpose and goals of a meeting input by the user and automatically creates a meeting schedule based on them.

[0146] "Means for displaying an agenda" refers to a device or interface for visually displaying the automatically generated agenda to a user.

[0147] "Means for uploading related materials" refers to an interface that allows a user to upload documents and materials to be used in a conference to the system.

[0148] "Means for analyzing uploaded materials and organizing necessary information using natural language processing" refers to the function of scanning uploaded materials and using natural language processing technology to extract and organize important information.

[0149] "Means for providing organized material" refers to an interface for providing organized information or transformed material to a user.

[0150] "Means for inputting participant list and possible date range" refers to an interface for inputting into the system a list of people who will be attending the conference and a possible date range for the conference.

[0151] "Means of checking participants' schedules and proposing optimal dates using a generative AI model" refers to a function that collects participants' schedule information and calculates optimal meeting dates using a generative AI model.

[0152] "Means for communicating and confirming proposed dates" refers to the interface or notification system for notifying participants of proposed meeting dates and obtaining confirmation.

[0153] "Means for announcing the start of a conference by voice using voice synthesis technology" refers to a function for announcing the start of a conference by voice using voice synthesis technology.

[0154] "Means of analyzing statements in real time and detecting position-taking and long speech" refers to a function that analyzes statements made during a meeting in real time and identifies specific speech patterns and long speech.

[0155] "Means for displaying a message to suppress long speech" refers to an interface for notifying participants when a long speech is detected.

[0156] "Means for automatically organizing statements and extracting important points" refers to a function that automatically classifies and organizes statements made during a meeting and extracts important points.

[0157] "Means for providing extracted points" refers to an interface for providing automatically extracted important points to a user.

[0158] The present invention relates to a system for automating a series of processes for preparing, conducting, and closing a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[0159] Meeting preparation

[0160] Agenda creation

[0161] Users use their devices to input the purpose and goals of a meeting into the system. This input information is sent to the server, which receives it and stores it in a database. The server then uses a generative AI model to automatically generate an agenda based on the input purpose and goals. The generated agenda is then sent from the server to the device, which displays it to the user.

[0162] Specifically, if a user enters "Project X progress report and second half plan" as the purpose and "Check this month's progress and set next task" as the goal, the server will automatically generate an agenda like the one below.

[0163] 1. Opening Remarks

[0164] 2. Project X progress report

[0165] 3. Announcement of second half plan

[0166] 4. Question and Answer Session

[0167] 5. Next task setting

[0168] 6. Closing Remarks

[0169] Preparation of meeting materials

[0170] Users upload relevant materials to the server via their devices. The server scans the uploaded materials and uses natural language processing technology to extract and organize the necessary information. The server then converts the organized materials into a downloadable format (e.g., PDF or Word file). The server then provides the organized and converted materials to the device, which then displays them to the user.

[0171] Specifically, when a user uploads "minutes of the previous meeting" or "progress report," the server analyzes it, extracts important data such as progress rate and issues, and generates organized documents.

[0172] Scheduling and communication

[0173] The user uses the device to input the participant list and possible date range. This data is also sent to the server, which checks the participants' schedule information. The server then uses the generative AI model to propose the optimal meeting date. The proposed date is sent from the server to the device, which notifies the user. The user reviews the proposed date and finalizes it. The server notifies all participants of the confirmed date.

[0174] Specifically, when a user inputs a weekday morning of the following week as a possible meeting date, the server checks the calendars of the participants and suggests the optimal date that everyone can attend (e.g., 10:00 a.m. next Wednesday).

[0175] Meeting proceedings

[0176] Display projection and audio guide

[0177] When a meeting starts, the server projects a pre-generated agenda and prepared materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[0178] Specifically, at the start of the meeting, an audio announcement will be made saying "The meeting is starting," while the agenda and materials will be displayed on the projector.

[0179] Speech Management

[0180] During the meeting, the server analyzes each participant's speech in real time to detect position-taking and long speech. If detected, the information is sent to the device, which then displays a message prompting the user to moderate their speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are also sent from the server to the device, which then displays them to the user.

[0181] Specifically, when a participant is detected speaking for a long time during a meeting, the server displays the message "Your speech is long" on the terminal.

[0182] Closing the meeting

[0183] Summary and reporting

[0184] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, which provides them to the user. The server then sends the summary and task list to all participants via email or notifications.

[0185] Specifically, after the meeting ends, the server automatically generates a "list of next tasks" and sends it to all participants by email.

[0186] Prompt Sentence Examples

[0187] Enter the purpose of the meeting: "Project X progress report and second half plan"

[0188] Upload related materials: "Minutes of the previous meeting, progress report"

[0189] Enter participant list and date range: "Next weekday morning"

[0190] This system efficiently automates the entire process of a meeting, reducing the burden on organizers and enabling participants to quickly share information.

[0191] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0192] Step 1:

[0193] The user inputs the purpose and goals of the meeting.

[0194] Specifically, the user uses a terminal to input the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task."

[0195] Input: Meeting purpose and goals

[0196] Output: The input data is sent to the server.

[0197] Step 2:

[0198] The server receives the input meeting purpose and goal and stores them in a database.

[0199] Then, a generative AI model is used to auto-generate the agenda.

[0200] Specifically, an agenda is generated that includes items such as "opening remarks" and "setting tasks for the next meeting."

[0201] Input: Meeting purpose and goals

[0202] Output: Auto-generated agenda

[0203] Step 3:

[0204] The server transmits the generated agenda to the terminal.

[0205] The terminal displays the received agenda to the user.

[0206] Specifically, each item on the agenda is listed on the screen.

[0207] Input: Generated agenda

[0208] Output: The agenda is displayed in the terminal.

[0209] Step 4:

[0210] Users upload relevant materials.

[0211] "Minutes of the previous meeting" and "progress reports" are uploaded to the server via the terminal.

[0212] Input: Related materials

[0213] Output: The data is sent to the server.

[0214] Step 5:

[0215] The server receives the uploaded materials and stores them in a database.

[0216] Next, natural language processing technology is used to extract and organize the necessary information.

[0217] Specifically, important data such as progress rate and issues are extracted.

[0218] Input: Uploaded materials

[0219] Output: Organized data

[0220] Step 6:

[0221] The server converts the organized data into a downloadable format.

[0222] The terminal provides this data to the user.

[0223] Specifically, materials can be downloaded in document format (PDF or Word files).

[0224] Input: Organized data

[0225] Output: Downloadable materials

[0226] Step 7:

[0227] The user inputs a list of participants and a range of possible dates.

[0228] Enter a date range such as "morning on a weekday next week" through the terminal.

[0229] Input: Participant list and date range

[0230] Output: The input data is sent to the server

[0231] Step 8:

[0232] The server checks each participant's schedule information and uses a generative AI model to suggest the optimal date.

[0233] The proposed itinerary is again transmitted to the terminal, which notifies the user of this.

[0234] Specifically, it will display the optimal date, such as "10:00 AM next Wednesday."

[0235] Input: Participant list and date range

[0236] Output: Suggested optimal schedule

[0237] Step 9:

[0238] The user reviews and finalizes the proposed itinerary.

[0239] The server will notify all participants of the confirmed schedule.

[0240] Specifically, participants will be notified of the schedule via email or message.

[0241] Input: Proposed optimal dates

[0242] Output: Notification of confirmed schedule

[0243] Step 10:

[0244] At the start of the meeting, the server projects the agenda and meeting materials onto a projector and announces the start of the meeting using voice synthesis technology.

[0245] Specifically, a voice announcement will be made saying "The meeting is about to begin," and the agenda and materials will be displayed on the projector.

[0246] Input: Trigger for starting the meeting (date and time)

[0247] Output: Meeting opening announcement and projection

[0248] Step 11:

[0249] During the meeting, the server analyzes each participant's comments in real time and detects position-taking and long comments.

[0250] If necessary, the device will display a suppression message such as "Your speech is getting long."

[0251] Input: Speech data during the meeting

[0252] Output: Suppression message

[0253] Step 12:

[0254] At the end of the meeting, the server summarizes the discussion and sets the next tasks, assignment of roles, and deadlines.

[0255] The organized summary and task list are sent from the server to the terminal and provided to the user.

[0256] Specifically, a list of upcoming tasks will be displayed on the device and sent to all participants by email.

[0257] Input: Meeting discussion data

[0258] Output: Summary and task list notifications

[0259] This system efficiently automates the entire process of a meeting, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[0260] (Application example 1)

[0261] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0262] The process of preparing, conducting, and closing meetings has traditionally been done manually, requiring time and effort, and is prone to human error and delays. Especially on factory floors, there is a need to streamline these meeting processes and ensure they proceed quickly and accurately. Furthermore, there is a need for effective methods for individual tasks, such as coordinating participant schedules, organizing materials, and managing comments.

[0263] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0264] In this invention, the server includes means for inputting the purpose and goals of a meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information, means for providing the organized materials, means for inputting a participant list and a range of possible dates, means for checking participant schedules and proposing the optimal date, means for notifying and confirming the proposed date, means for analyzing the time and content of speech, means for issuing a warning if the speech time is too long, means for summarizing the main points and automatically setting the next tasks and role allocation, and means for notifying the main points and task information. This automates the entire process of preparing, conducting, and closing a meeting, making it possible to perform it efficiently and accurately.

[0265] The "means for inputting the purpose and goal of the meeting" refers to an input device or interface that allows the user to input the specific purpose of the meeting and the goal that the user wants to achieve into the system.

[0266] A "means for automatically generating an agenda based on input objectives and goals" is an algorithm or software that analyzes the objectives and goals of a meeting and automatically generates an effective agenda.

[0267] A "means for displaying an agenda" is a display device or display interface that allows a user to view the generated agenda.

[0268] The "means for uploading relevant materials" refers to a device or interface for uploading materials required for a conference to the system.

[0269] "Means for analyzing uploaded materials and organizing necessary information" refers to software or algorithms that analyze uploaded materials and extract and organize important information.

[0270] A "means for providing organized material" is an interface or device for providing the analyzed and organized material to a user.

[0271] The "means for inputting a participant list and a possible date range" is an interface or device for inputting a participant list and a possible date range of a conference into the system.

[0272] The "means for checking the schedules of participants and proposing the optimal date" is software or an algorithm that checks the schedules of the participants that have been input and proposes the optimal meeting date.

[0273] A "means for notifying and confirming proposed dates" is an interface or device for notifying participants of proposed meeting dates and receiving confirmation.

[0274] The "means for analyzing the time and content of speech" is software or an algorithm that analyzes the time and content of speech during a meeting in real time.

[0275] The "means for issuing a warning when a speech duration is too long" is an interface or device for detecting an excessively long speech duration and issuing a warning.

[0276] "Means for summarizing key points and automatically setting the next tasks and division of roles" refers to software or an algorithm that summarizes the key points of a meeting and automatically sets the next tasks and division of roles.

[0277] The "means for notifying key points and task information" is an interface or device for notifying all participants of the organized key points and set task information.

[0278] The present invention provides a system for automating the preparation, progress, and closing of a meeting. A specific embodiment of this system will be described below.

[0279] Meeting preparation

[0280] Agenda creation

[0281] First, the user speaks to the robot to input the purpose and goal of the meeting. The robot converts the purpose and goal of the meeting into text data using speech recognition technology and sends it to the server. The server then uses a generative AI model (for example, OpenAI's GPT-3 model) to automatically generate an agenda based on the purpose and goal. This agenda is then displayed on the robot.

[0282] Preparation of meeting materials

[0283] Users upload meeting materials to the robot using a USB memory stick or similar device. The uploaded materials are then sent to a server, which scans and analyzes them, extracting and organizing the necessary information. This process uses document analysis software, and the robot then provides the organized materials to the user.

[0284] Scheduling and communication

[0285] The user inputs a list of participants and a range of possible dates into the robot. The input data is sent to the server, which checks the participants' schedules and automatically proposes the most suitable meeting date. The robot notifies the user of this proposal by voice or on-screen display. The user confirms and confirms the final date, and the server notifies all participants of the confirmed date.

[0286] Conference moderator

[0287] Display projection and speech

[0288] When a meeting starts, the robot retrieves the pre-generated agenda and prepared meeting materials from the server, projects them onto a projector or monitor, and uses voice synthesis technology to announce the start of the meeting.

[0289] Speech Management

[0290] During the meeting, the robot analyzes each participant's speech in real time. It uses speech recognition and natural language processing technology to analyze the content and time of speech and detect long speeches and position-taking. If this is detected, the robot issues a warning and adjusts the speech as necessary. The server also automatically organizes the speech and extracts key points. This information is also provided by the robot.

[0291] Closing the meeting

[0292] Summary and reporting

[0293] At the end of the meeting, the server summarizes the content of the discussion and automatically sets the next tasks, assignment of roles, and deadlines. This summary and task list are provided to the user via the robot, and all participants are notified using the email sending function.

[0294] Specific examples

[0295] For example, to hold a progress meeting for a new product project, the user inputs to the robot, "I would like to hold a progress meeting for the new product project. The purpose is to check the progress and set the next task." The robot sends this information to the server, which generates the following agenda:

[0296] Purpose of the meeting: Check the progress of the new product project

[0297] Goal: Set the next task

[0298] Generate an agenda.

[0299] The final agenda generated by the server looks like this:

[0300] 1. Opening Remarks

[0301] 2. Project progress reports

[0302] 3. Announcement of next plan

[0303] 4. Question and Answer Session

[0304] 5. Next task setting

[0305] 6. Closing Remarks

[0306] This agenda is projected by the robot and announced aloud. The robot also analyzes comments made during the meeting, extracting and displaying important points. After the meeting ends, the server summarizes the main points of the discussion, sets the next tasks and allocation of roles, and notifies all participants.

[0307] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0308] Step 1:

[0309] The user speaks to the robot to input the purpose and goals of the meeting.

[0310] Input: Voice input of the purpose and goals of the meeting

[0311] Output: Meeting objectives and goals as text data

[0312] Data processing and calculation: The robot uses voice recognition technology (e.g., speech_recognition module) to convert voice data into text data.

[0313] Step 2:

[0314] The robot sends the converted text data to a server, which then uses a generative AI model (e.g., GPT-3) to automatically generate an agenda.

[0315] Input: Meeting objectives and goals as text data

[0316] Output: Agenda text data

[0317] Data processing and calculation: The server forms prompt sentences based on the purpose and goal, inputs them into the generative AI model, and generates an agenda.

[0318] Step 3:

[0319] The server sends the generated agenda to the robot, which displays the agenda.

[0320] Input: Agenda text data

[0321] Output: Agenda displayed on the display

[0322] Data processing and calculation: The robot displays the acquired agenda on its built-in display or an external monitor.

[0323] Step 4:

[0324] The user uploads meeting materials to the robot using a USB memory stick, etc. The robot then sends the materials to the server.

[0325] Input: Materials uploaded via USB memory or LAN connection

[0326] Output: Data transferred to the server

[0327] Data processing and calculation: The robot securely transmits the uploaded data to the server.

[0328] Step 5:

[0329] The server scans and analyzes the uploaded materials and organizes the necessary information.

[0330] Input: Uploaded material data

[0331] Output: Organized data

[0332] Data processing and calculation: The server uses document analysis software to scan the documents, extract text, and organize the information.

[0333] Step 6:

[0334] The participant list and possible date range are input to the robot, which then sends this to the server, which then suggests the best meeting date.

[0335] Input: List of participants and possible date range

[0336] Output: Proposed meeting dates

[0337] Data processing and calculation: The server checks the participants' schedules and uses an algorithm to calculate the optimal schedule.

[0338] Step 7:

[0339] The robot will notify the user of the schedule proposed by the server by voice or on-screen display, and the user will confirm it.

[0340] Input: Proposed meeting dates

[0341] Output: Confirmed or revised meeting dates

[0342] Data processing and calculation: The robot presents the proposed schedule and accepts confirmation from the user.

[0343] Step 8:

[0344] At the start of a meeting, the robot retrieves the agenda and prepared meeting materials, projects them onto a projector or monitor, and announces the start of the meeting using voice synthesis technology.

[0345] Input: Meeting agenda and materials data

[0346] Output: Agenda and materials projected on a projector or monitor, audio announcements to start

[0347] Data processing and calculation: The robot visually displays the agenda and materials and announces the start of the meeting by voice.

[0348] Step 9:

[0349] During meetings, the robot analyzes comments in real time and detects long speeches and position-taking.

[0350] Input: Participant's voice

[0351] Output: Parsed speech and warning messages

[0352] Data processing and calculation: The robot uses voice recognition and natural language processing technology to analyze the time and content of speech and display warnings as necessary.

[0353] Step 10:

[0354] At the end of the meeting, the server summarizes the content of the discussion, sets the next tasks, assignment of roles, and deadlines, and notifies all participants of the summary and task list.

[0355] Input: What was said during the meeting and notes

[0356] Output: Summary of discussion and task list

[0357] Data processing and calculation: The server uses natural language processing technology to summarize the discussion content, automatically set the next tasks and role allocation, and notify participants.

[0358] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0359] The present invention combines a system that automates a series of processes from advance preparation to progress and closing of a meeting with an emotion engine that recognizes the user's emotions, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments are described below.

[0360] Meeting preparation

[0361] Agenda creation

[0362] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[0363] Preparation of meeting materials

[0364] Users upload relevant materials to be used in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format and provides them to users via their devices.

[0365] Scheduling and communication

[0366] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[0367] Conference moderator

[0368] Display projection and speech

[0369] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[0370] Speech Management

[0371] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[0372] Additional features of the Emotion Engine

[0373] Furthermore, an emotion engine is built in that analyzes the user's speech and facial expressions to recognize their emotions. The emotion engine analyzes the audio and video data acquired by the server in real time to identify the user's emotions (e.g., joy, anger, sadness, surprise, etc.).

[0374] The server uses the recognized emotions to adjust the progress of the conference. For example, if a user is feeling stressed, the server will adjust the progress of the conference to be more gradual. Also, if a specific emotion is detected, the server will give priority to allowing the corresponding participant to speak.

[0375] Closing the meeting

[0376] Summary and reporting

[0377] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the device, which provides them to the user. The server then sends the summary and task list to all participants via email or notification. It is also possible to provide individual feedback after the meeting based on the emotional data recognized by the emotion engine.

[0378] Specific examples

[0379] For example, when hosting a project progress meeting, the user inputs the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task" into the system. Based on this, the server automatically generates the following agenda:

[0380] 1. Opening Remarks

[0381] 2. Project X progress report

[0382] 3. Announcement of second half plan

[0383] 4. Question and Answer Session

[0384] 5. Next task setting

[0385] 6. Closing Remarks

[0386] The user then uploads relevant materials (minutes of the previous meeting, progress reports, etc.) to the server, which analyzes them and organizes the necessary information. After entering the list of meeting participants and the range of possible dates, the server will suggest the optimal date and notify the user via the device.

[0387] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. Comments made during the meeting are also analyzed in real time, and messages are displayed to discourage long comments as necessary. The server also automatically extracts important comments and provides them to the user.

[0388] Furthermore, the emotion engine analyzes the remarks and facial expressions of meeting participants to recognize their emotions. For example, if a participant is feeling stressed, the server will use that information to adjust the pace of the meeting and prioritize the comments of other participants. After the meeting ends, individual feedback can be provided based on the emotional data.

[0389] This efficiently automates the entire meeting process, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need. In addition, the introduction of an emotion engine enables flexible meeting progression that takes into account the emotional state of participants.

[0390] The processing flow will be explained below.

[0391] Meeting preparation

[0392] Agenda creation

[0393] Step 1:

[0394] The user inputs the purpose and goals of the meeting using the system's interface.

[0395] Step 2:

[0396] The server receives the input objectives and goals, analyzes the information, and automatically generates an agenda based on the analysis results.

[0397] Step 3:

[0398] The server sends the generated agenda to the terminal.

[0399] Step 4:

[0400] The terminal displays the received agenda to the user, who then checks the displayed agenda.

[0401] Preparation of meeting materials

[0402] Step 1:

[0403] The user uploads the relevant materials to be used in the meeting to the server via the terminal.

[0404] Step 2:

[0405] The server scans the uploaded materials and extracts and organizes the necessary information.

[0406] Step 3:

[0407] The server converts the organized material into a downloadable format.

[0408] Step 4:

[0409] The terminal provides the converted material to the user, who then downloads and checks the provided material.

[0410] Scheduling and communication

[0411] Step 1:

[0412] The user inputs the list of participants and the available dates. The user inputs the list using the system interface.

[0413] Step 2:

[0414] The server receives the entered data and checks the schedules of all participants.

[0415] Step 3:

[0416] The server performs the analysis and suggests the best meeting date.

[0417] Step 4:

[0418] The server sends the proposed itinerary to the terminal.

[0419] Step 5:

[0420] The terminal notifies the user of the proposed schedule, and the user confirms the schedule and makes a final decision.

[0421] Step 6:

[0422] The server will notify all participants of the confirmed schedule.

[0423] Conference moderator

[0424] Display projection and speech

[0425] Step 1:

[0426] The server projects the agenda created at the start of the meeting and the prepared meeting materials onto a display.

[0427] Step 2:

[0428] The server uses voice synthesis technology to announce the start of the conference.

[0429] Speech Management

[0430] Step 1:

[0431] The server analyzes each participant's comments in real time.

[0432] Step 2:

[0433] The server detects position talk and long speech.

[0434] Step 3:

[0435] If a long speech is detected, the server sends that information to the terminal.

[0436] Step 4:

[0437] The terminal displays a message to the user at an appropriate timing to prevent long-term speech.

[0438] Organizing the discussion

[0439] Step 1:

[0440] The server automatically organizes the comments in real time.

[0441] Step 2:

[0442] The server extracts the key points.

[0443] Step 3:

[0444] The server transmits the extracted points to the terminal.

[0445] Step 4:

[0446] The terminal displays the extracted points to the user.

[0447] Additional features of the Emotion Engine

[0448] Step 1:

[0449] The server captures audio and video data in real time during the conference.

[0450] Step 2:

[0451] An emotion engine built into the server analyzes the captured data and recognizes the user's emotions.

[0452] Step 3:

[0453] The server acquires the emotion data recognized by the emotion engine and reflects it in the progress of the conference as appropriate.

[0454] Step 4:

[0455] If a user displays strong emotions, the server adjusts the conference progress and allows priority permission to speak as necessary.

[0456] Closing the meeting

[0457] Summary and reporting

[0458] Step 1:

[0459] The server summarizes the discussion at the end of the meeting.

[0460] Step 2:

[0461] The server sets the allocation of roles and deadlines for the next task.

[0462] Step 3:

[0463] The server sends the summarized content and task list to the terminal.

[0464] Step 4:

[0465] The terminal provides the summary and task list to the user, who then checks the information provided.

[0466] Step 5:

[0467] The server notifies all participants of the same information.

[0468] Step 6:

[0469] Based on the emotional data recognized by the emotion engine, the server provides individual feedback after the meeting ends.

[0470] Example 2

[0471] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0472] In conventional conference systems, the process of preparing, conducting, and closing a meeting is done manually, which takes a lot of time and effort. It is also difficult to analyze participants' comments and behavior in real time and provide appropriate feedback. Another problem is that it is difficult to conduct a meeting flexibly while taking into account participants' emotions.

[0473] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting the purpose and goal of the meeting, means for automatically generating an agenda based on the input purpose and goal, means for displaying the agenda, means for uploading materials, means for analyzing the uploaded materials and organizing the necessary information, means for providing the organized materials, means for inputting a participant list and a range of available dates, means for checking the schedules of participants and proposing an optimal date, means for notifying and confirming the proposed date, means for analyzing comments in real time, means for automatically organizing the content of comments and extracting important points, means for providing the extracted points, means for analyzing emotions and adjusting the progress of the meeting, and means for providing individual feedback based on emotion data. This efficiently automates the processes of preparing, conducting, and closing the meeting, enabling flexible meeting progress that takes into account the comments and emotions of participants.

[0474] The "means for inputting the purpose and goal of the meeting" is an interface device that allows the user to input the specific purpose of the meeting and the goal that the user wants to achieve into the system.

[0475] The "means for automatically generating an agenda" is a software module that automatically generates a meeting schedule and agenda based on the input purpose and goals of the meeting.

[0476] A "means for displaying an agenda" is a display device or digital screen that visually displays the generated agenda to a user.

[0477] A "means for uploading materials" is an interface device that allows a user to upload materials or documents to be used in a conference to the system.

[0478] The "means for analyzing uploaded materials and organizing necessary information" is a software module that scans uploaded materials and extracts and classifies necessary information.

[0479] A "means for providing organized material" is an interface or file serving system that provides the organized material in a format that is usable by the user.

[0480] The "means for inputting a participant list and a range of possible dates" is an interface device for inputting participants of a meeting and possible dates for the meeting.

[0481] The "means of checking participants' schedules and proposing the optimal date" is an algorithm that calculates and proposes the optimal meeting date based on the participants' schedules entered.

[0482] A "means for notifying and confirming proposed dates" is a notification system or confirmation interface for notifying participants of proposed meeting dates and obtaining their confirmation.

[0483] "Means for analyzing speech in real time" refers to speech recognition technology that analyzes speech made by participants in a meeting in real time and understands the content.

[0484] The "means for automatically organizing speech content and extracting important points" is a software module that automatically organizes analyzed speech content and extracts important points.

[0485] The "means for providing extracted points" is an interface device for providing the extracted important points to the user.

[0486] The "means for analyzing emotions and adjusting the progress of the meeting" is an algorithm that analyzes participants' emotions from audio and video data and adjusts the progress of the meeting based on their emotional state.

[0487] The "means for providing individual feedback based on emotion data" is an interface device for providing feedback to individual participants based on emotion analysis data.

[0488] This invention combines a system that automates the process of preparing for, conducting, and closing a meeting with an emotion engine that recognizes the user's emotions. This system is implemented mainly through the roles of the server, the terminal, and the user.

[0489] Meeting preparation

[0490] Agenda creation

[0491] The user inputs the purpose and goals of the meeting on their device. This information is sent to the server. The server analyzes the input data and automatically generates an agenda using a generative AI model. The generated agenda is sent from the server to the device, which displays it to the user.

[0492] Preparation of meeting materials

[0493] Users upload relevant materials for meetings from their devices to the server. The server scans the uploaded materials and uses OCR (optical character recognition) technology to extract and organize the necessary information. The server then converts the organized materials into downloadable formats such as PDFs or spreadsheets and provides them to users via their devices.

[0494] Scheduling and communication

[0495] The user inputs the participant list and available dates from their device and sends them to the server. The server checks each participant's schedule and uses an algorithm to propose the optimal meeting date. The proposed optimal date is sent from the server to the device, which notifies the user. The user then confirms and confirms the final date. The server notifies all participants of the confirmed date.

[0496] Conference moderator

[0497] Display projection and speech

[0498] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[0499] Speech Management

[0500] During a meeting, the server uses voice analysis technology to analyze each participant's speech in real time. If the server detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[0501] Additional features of the Emotion Engine

[0502] Furthermore, the server operates an emotion engine that analyzes users' comments and facial expressions to recognize their emotions. The emotion engine analyzes the audio and video data acquired by the server in real time to identify the user's emotions (e.g., joy, anger, sadness, surprise, etc.). The recognized emotions are used to adjust the progress of the meeting. For example, if a user is feeling stressed, the server will adjust the progress of the speech to be more gradual. Furthermore, if a specific emotion is detected, the server will give priority to allowing the corresponding participant to speak.

[0503] Closing the meeting

[0504] Summary and reporting

[0505] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the device, which provides them to the user. The server then sends the summary and task list to all participants via email or notification. It is also possible to provide individual feedback after the meeting based on the emotional data recognized by the emotion engine.

[0506] Specific examples

[0507] For example, when hosting a project progress meeting, the user inputs the purpose "Project X progress report and second half plan" and the goal "Confirm this month's progress and set next tasks" from the terminal. Based on this, the server automatically generates the following agenda:

[0508] Opening remarks

[0509] Project X progress report

[0510] Announcement of second half plan

[0511] Q&A

[0512] Next task settings

[0513] Closing remarks

[0514] Next, the user uploads relevant materials (minutes of the previous meeting, progress reports, etc.) from their device to the server, which then analyzes them and organizes the necessary information. When the user enters the participant list and available dates, the server proposes the optimal dates and notifies them via the device.

[0515] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. During the meeting, the server analyzes the content of comments and emotional data, adjusting the progress as necessary. After the meeting ends, a summary of key points and a task list are provided, and individual feedback based on the emotional data is distributed.

[0516] This system efficiently automates the entire process of a meeting, significantly reducing the burden on the organizer and enabling flexible meeting proceedings that take into account the emotions of participants.

[0517] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0518] Step 1:

[0519] The user enters the purpose and goals of the meeting.

[0520] Input: The user enters the "purpose of the meeting" and "goal of the meeting" into the input form on the terminal.

[0521] Data processing: The data entered by the user is sent from the terminal to the server.

[0522] Output: The server uses the received data for analysis.

[0523] Specific operation: The user inputs "Marketing strategy for new product" as the "Objective of the meeting" on the terminal screen, and "Determine the strategy content" as the "Goal of the meeting."

[0524] Step 2:

[0525] The server receives the objectives and goals and generates the agenda.

[0526] Input: The server receives the "meeting purpose" and "meeting goal" entered by the user.

[0527] Data computation: The server analyzes the input data and automatically generates an agenda using a generative AI model.

[0528] Output: The generated agenda is saved on the server.

[0529] Specific operation: Based on the received data, the server generates an agenda including the flow and content of the meeting.

[0530] Step 3:

[0531] The server sends the agenda to the terminal, which displays it to the user.

[0532] Input: The server has the generated agenda data.

[0533] Data processing: The server sends the agenda data to the terminal.

[0534] Output: The submitted agenda is displayed on the terminal.

[0535] Specific operation: The generated agenda is displayed on the terminal screen to the user. The user confirms it.

[0536] Step 4:

[0537] The user uploads the meeting materials to the server.

[0538] Input: The user selects and uploads relevant materials (PDFs, documents, etc.) to be used in the meeting from their terminal.

[0539] Data processing: The terminal sends the selected materials to the server.

[0540] Output: The server receives the uploaded material.

[0541] Specific operation: The user selects "minutes of the previous meeting" or "progress report" from their device and presses the upload button.

[0542] Step 5:

[0543] The server scans the documents and extracts and organizes the necessary information.

[0544] Input: The server has the uploaded data.

[0545] Data calculation: The server uses OCR technology to scan the text in the document, extract and organize the necessary information.

[0546] Output: The extracted and organized information is saved on the server.

[0547] What it does: The server parses the uploaded PDF and extracts important text data.

[0548] Step 6:

[0549] The server transmits the organized materials to the terminal, which then provides them to the user.

[0550] Input: The server has organized data.

[0551] Data processing: The server converts the organized materials into PDF or spreadsheet format and sends them to the terminal.

[0552] Output: The transmitted material is displayed on the terminal and provided to the user.

[0553] Specific operation: The user can view and download the organized materials displayed on the terminal.

[0554] Step 7:

[0555] The user enters the participant list and date range.

[0556] Input: The user inputs the participant list and possible date range of the meeting from the terminal.

[0557] Data processing: The terminal sends the input information to the server.

[0558] Output: The server uses the received data for analysis.

[0559] Specific operation: The user uploads the participant list in an Excel file and selects and enters the available date range.

[0560] Step 8:

[0561] The server will suggest the best schedule and notify the user via the terminal.

[0562] Input: The server has the attendee list and date range data.

[0563] Data calculation: The server checks the schedules of each participant and uses an algorithm to calculate the optimal meeting date.

[0564] Output: The proposed optimal schedule is stored on the server and sent to the terminal.

[0565] Specific operation: A notification suggesting the best date will be displayed on the user's device.

[0566] Step 9:

[0567] The user checks and confirms the final schedule.

[0568] Input: The user confirms the proposed itinerary on the terminal.

[0569] Data processing: When the user presses the confirm button, the information is sent to the server.

[0570] Output: The confirmed schedule is saved on the server.

[0571] Specific operation: The user checks the proposed schedule and clicks the confirm button.

[0572] Step 10:

[0573] The server will notify all participants of the confirmed schedule.

[0574] Input: The server has the confirmed schedule data.

[0575] Data processing: The server sends the confirmed schedule to all participants via email or notification.

[0576] Output: All participants are notified of the meeting date.

[0577] Specific behavior: A schedule notification will be sent to all participants via email.

[0578] Step 11:

[0579] At the start of the meeting, the server projects the agenda and materials and announces the start of the meeting with a voice message.

[0580] Input: The server has the generated agenda and organized document data.

[0581] Data processing: The server projects the agenda and materials onto a projector and announces the start of the meeting using voice synthesis technology.

[0582] Output: Projected agenda and materials, audio start announcement.

[0583] Specific operation: The meeting agenda and materials are displayed on the projector, and an automated voice announces the start of the meeting.

[0584] Step 12:

[0585] The server analyzes the comments in real time and sends messages to the device as needed.

[0586] Input: Audio data during the conference is sent to the server.

[0587] Data calculation: The server performs real-time voice analysis and determines the content of the speech.

[0588] Output: If position talk or long speech is detected, the information is sent to the device.

[0589] Specific behavior: A warning message will be displayed on the user's device to discourage long-term comments.

[0590] Step 13:

[0591] The server organizes and extracts the content of comments and provides it to the user.

[0592] Input: Speech data analyzed during the meeting is saved on the server.

[0593] Data calculation: The server runs an algorithm to extract important points.

[0594] Output: The extracted key points are stored on the server and sent to the device.

[0595] Specific operation: A summary of important comments is displayed on the user's device.

[0596] Step 14:

[0597] The server uses an emotion engine to analyze and recognize participants' emotions.

[0598] Input: Audio and video data during the conference are sent to the server.

[0599] Data calculation: The emotion engine analyzes data in real time and recognizes emotions.

[0600] Output: The recognized emotion data is stored on the server.

[0601] Specific operation: The server monitors the participants' emotional states based on the emotional data analyzed.

[0602] Step 15:

[0603] The server adjusts the progress of the conference based on the emotion data.

[0604] Input: Emotion data stored on the server.

[0605] Data processing: The server runs an algorithm to adjust the progress of the meeting based on the emotion data.

[0606] Output: Adjusted meeting progress data.

[0607] Specific actions: For example, adjustments may be made such as slowing down the pace for participants who are feeling stressed, or prioritizing the comments of participants who have detected certain emotions.

[0608] Step 16:

[0609] The server organizes the key points and sets tasks and roles.

[0610] Input: Speech and emotion data captured during the meeting.

[0611] Data calculation: The server summarizes the discussion and sets the next task and allocation of roles.

[0612] Output: A concise summary and a task list.

[0613] Specific operation: The server generates summarized discussion points and task lists and compiles them as materials for the next meeting.

[0614] Step 17:

[0615] The server sends the summary and task list to all participants.

[0616] Input: An organized summary and a task list.

[0617] Data processing: The server sends the summary and task list to all participants.

[0618] Output: Summary and task list sent to all participants.

[0619] What happens: A summary and task list will be sent to all participants via email and notifications.

[0620] Step 18:

[0621] After the meeting, the server provides individual feedback based on the emotional data.

[0622] Input: Emotion data collected during the meeting.

[0623] Data calculation: The server creates individual feedback based on the analysis results.

[0624] Output: Individual feedback report.

[0625] Specific operation: The server generates individual feedback for each participant based on the emotional data and provides it via email or notification.

[0626] (Application example 2)

[0627] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0628] In modern factory environments, there is a demand for improved work efficiency and strengthened quality control. However, there is also the issue that emotional factors such as worker stress and fatigue have a significant impact on work efficiency. This has led to issues that traditional work management methods are no longer able to address. Furthermore, when errors occur during work, a system is needed to respond immediately, but traditional systems have issues with responsiveness. Furthermore, creating reports after task completion takes time, so efficient improvement measures are needed.

[0629] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for inputting the purpose and goal of the meeting; a means for automatically generating an agenda based on the input purpose and goal; a means for displaying the agenda; a means for uploading related materials; a means for analyzing the uploaded materials and organizing the necessary information; a means for providing the organized materials; a means for inputting a participant list and a range of possible dates; a means for checking the schedules of participants and proposing an optimal date; a means for notifying and confirming the proposed date; a means for acquiring task data and analyzing it using machine learning technology; a means for acquiring voice and video data of workers in real time; a means for analyzing the acquired data using an emotion analysis engine; a means for detecting errors and notifying workers in real time; and a means for automatically generating a report after task completion. This enables improved work efficiency and enhanced quality control in factories. Furthermore, analyzing and responding to workers' emotional states in real time can also improve the work environment. In addition, immediate response to errors and automatic report generation enable rapid implementation of corrective measures.

[0630] "Meeting objectives and goals" refers to the specific goals that are to be achieved by holding the meeting and the results that are to be achieved at the end of the meeting.

[0631] An "agenda" refers to a list of topics or items set in advance to plan the progress of a meeting or work.

[0632] "Related materials" refers to documents and reference information required for the progress of meetings and work.

[0633] "Upload" refers to the act of sending data from a device to a remote system such as a server or cloud.

[0634] "Machine learning technology" refers to computer programs and algorithms that automatically learn patterns from data and make subsequent predictions or decisions.

[0635] "Emotion analysis engine" refers to software and algorithms that analyze audio and video data and identify human emotional states.

[0636] "Real-time" refers to data or events being processed and analyzed almost immediately after they occur.

[0637] "Error detection" refers to the act of identifying anomalies or errors that occur during a task or system process.

[0638] "Report" refers to a document or digital compilation of the content, progress, and results of a meeting or task.

[0639] MODE FOR CARRYING OUT THE INVENTION

[0640] This invention is a system for improving the work efficiency and quality control of factory robots, which is implemented through the roles of a server, a terminal, and a user. This system is based on a conference automation system and focuses on specific applications in a factory environment.

[0641] Task setting and advance preparation

[0642] The server sends work task data to the factory robots. The task data is analyzed on the server using machine learning technology (such as TensorFlow or PyTorch) to generate specific work procedures. The generated procedures are then sent as instructions to the factory robots.

[0643] Real-time progress management

[0644] The device is equipped with a camera (such as an Intel RealSense or Raspberry Pi camera module) and microphone to capture the worker's voice and video data. The captured data is then sent to a server where it is analyzed in real time by an emotion analysis engine (using OpenCV and NLP technology). This allows the worker's emotional state to be identified and work procedures to be adjusted as necessary.

[0645] Error Detection and Notification

[0646] The server scans the acquired work data in real time and detects errors using machine learning models such as TensorFlow. If an error is detected, the worker is immediately notified via their device. The notification also includes appropriate countermeasures.

[0647] Closing and reporting on task completion

[0648] Once a task is completed, the server automatically organizes the work that has been performed and generates a report. The report, which clarifies the next task setting and areas for improvement, is provided to the worker via their terminal. The generated report helps improve overall work efficiency and quality.

[0649] Specific examples

[0650] For example, on a precision assembly line, it operates as follows:

[0651] 1. Task: Attach part A to B of a specific product.

[0652] 2. The server acquires the task data, analyzes it, and generates a work procedure.

[0653] 3. The voice and video data of workers captured through cameras and microphones is analyzed using an emotion analysis engine, and if a state of stress is detected, work procedures are adjusted.

[0654] 4. If an error occurs during work, the server will immediately notify you and provide you with the appropriate solution.

[0655] 5. Once the task is completed, the server automatically generates a report and provides the worker with information on the next task settings and improvements.

[0656] Prompt Sentence Examples

[0657] "A factory work supervision application based on a conference automation system. It includes functions such as specific task setting, optimizing work efficiency through real-time sentiment analysis, error detection, and work report generation."

[0658] This system dramatically improves work efficiency and quality within the factory and reduces the burden on workers. Real-time error detection and immediate response, an improved work environment through emotion analysis, and automatic report generation enable more flexible and efficient work management.

[0659] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0660] Step 1:

[0661] The server receives data from the user to input the purpose and goals of the meeting. The input purpose and goal data is processed within the server as initial data for automatic agenda generation. Here, the server uses natural language processing (NLP) technology to analyze the input data and generate agenda items. The generated agenda is sent to the terminal.

[0662] Step 2:

[0663] The device displays the agenda sent from the server. At this time, the user uploads relevant materials to the server via the device. The server analyzes the uploaded materials and organizes the necessary information. This analysis process uses OCR technology (optical character recognition) and machine learning algorithms to automatically extract useful information from the documents. The organized materials are then sent back to the device and provided to the user.

[0664] Step 3:

[0665] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and uses a calendar API or scheduling algorithm to suggest the best meeting date. The server sends the suggested best date to the device, which notifies the user. Once the user confirms and confirms the final date, the server notifies all participants of the confirmed date.

[0666] Step 4:

[0667] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and simultaneously announces the start of the meeting using speech synthesis technology. The input data are the agenda items and meeting materials, and the output is a visual and audio announcement of the start of the meeting.

[0668] Step 5:

[0669] During a meeting, the device captures the voice and video data of the participants in real time through its equipped camera and microphone. The captured data is sent to a server and analyzed by an emotion analysis engine. This includes, for example, facial expression analysis using OpenCV and voice analysis using NLP technology. If the analysis reveals that the user is feeling stressed, this information is recorded on the server and the progress of the meeting is adjusted.

[0670] Step 6:

[0671] The server analyzes speech in real time during meetings to detect long speeches and position-taking. It uses NLP technology to analyze speech content and detects speeches of a specific length or that focus on a specific topic. The detected information is sent to the device and displayed as a message.

[0672] Step 7:

[0673] At the end of the meeting, the server summarizes the discussion and sets the next task assignment and deadline. The summary and task list are sent from the server to the device, which then provides them to the user. Furthermore, the server generates individual feedback based on the emotional data recognized by the emotion analysis engine and sends it to the device. This provides a detailed meeting report to all participants.

[0674] Prompt Sentence Examples

[0675] "A factory work supervision application based on a conference automation system. It includes functions such as specific task setting, optimizing work efficiency through real-time sentiment analysis, error detection, and work report generation."

[0676] This dramatically improves work efficiency and quality within the factory and reduces the burden on workers. Real-time error detection and immediate response, an improved work environment through emotion analysis, and automatic report generation enable more flexible and efficient work management.

[0677] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0678] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0679] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0680] [Second embodiment]

[0681] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0682] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0683] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0684] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0685] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0686] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0687] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0688] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0689] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0690] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0691] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0692] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0693] The present invention relates to a system for automating a series of processes from advance preparation to progress and closing of a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[0694] Meeting preparation

[0695] Agenda creation

[0696] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[0697] Preparation of meeting materials

[0698] Users upload relevant materials for use in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format. The terminal then provides the converted materials to the user.

[0699] Scheduling and communication

[0700] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[0701] Conference moderator

[0702] Display projection and speech

[0703] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[0704] Speech Management

[0705] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[0706] Closing the meeting

[0707] Summary and reporting

[0708] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, and the terminal provides them to the user. Furthermore, the server sends the summary and task list to all participants using email or notifications.

[0709] Specific examples

[0710] For example, when hosting a project progress meeting, the user inputs the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task" into the system. Based on this, the server automatically generates the following agenda:

[0711] 1. Opening Remarks

[0712] 2. Project X progress report

[0713] 3. Announcement of second half plan

[0714] 4. Question and Answer Session

[0715] 5. Next task setting

[0716] 6. Closing Remarks

[0717] The user then uploads relevant materials (minutes of the previous meeting, progress reports, etc.) to the server, which analyzes them and organizes the necessary information. When the user inputs the list of meeting participants and a range of possible dates (e.g., a weekday morning next week), the server proposes the optimal date and notifies the user via the device.

[0718] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. Speech during the meeting is also analyzed in real time, and messages are displayed to limit long speeches as necessary. Furthermore, the server automatically extracts important speech content and provides it to the user.

[0719] After the meeting, the server summarizes the discussion and sets the next task, its assignment, and deadline. The summary and task list are provided to the user via their terminal and reported to all participants.

[0720] This efficiently automates the entire meeting process, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[0721] The processing flow will be explained below.

[0722] Meeting preparation

[0723] Agenda creation

[0724] Step 1:

[0725] The user inputs the purpose and goals of the meeting using the system's interface.

[0726] Step 2:

[0727] The server receives the input objectives and goals, analyzes the information, and automatically generates an agenda based on the analysis results.

[0728] Step 3:

[0729] The server sends the generated agenda to the terminal.

[0730] Step 4:

[0731] The terminal displays the received agenda to the user, who then checks the displayed agenda.

[0732] Preparation of meeting materials

[0733] Step 1:

[0734] The user uploads the relevant materials to be used in the meeting to the server via the terminal.

[0735] Step 2:

[0736] The server scans the uploaded materials and extracts and organizes the necessary information.

[0737] Step 3:

[0738] The server converts the organized material into a downloadable format.

[0739] Step 4:

[0740] The terminal provides the converted material to the user, who then downloads and checks the provided material.

[0741] Scheduling and communication

[0742] Step 1:

[0743] The user inputs the list of participants and the available dates. The user inputs the list using the system interface.

[0744] Step 2:

[0745] The server receives the entered data and checks the schedules of all participants.

[0746] Step 3:

[0747] The server performs the analysis and suggests the best meeting date.

[0748] Step 4:

[0749] The server sends the proposed itinerary to the terminal.

[0750] Step 5:

[0751] The terminal notifies the user of the proposed schedule, and the user confirms the schedule and makes a final decision.

[0752] Step 6:

[0753] The server will notify all participants of the confirmed schedule.

[0754] Conference moderator

[0755] Display projection and speech

[0756] Step 1:

[0757] The server projects the agenda created at the start of the meeting and the prepared meeting materials onto a display.

[0758] Step 2:

[0759] The server uses voice synthesis technology to announce the start of the conference.

[0760] Speech Management

[0761] Step 1:

[0762] The server analyzes each participant's comments in real time.

[0763] Step 2:

[0764] The server detects position talk and long speech.

[0765] Step 3:

[0766] If a long speech is detected, the server sends that information to the terminal.

[0767] Step 4:

[0768] The terminal displays a message to the user at an appropriate timing to prevent long-term speech.

[0769] Organizing the discussion

[0770] Step 1:

[0771] The server automatically organizes the comments in real time.

[0772] Step 2:

[0773] The server extracts the key points.

[0774] Step 3:

[0775] The server transmits the extracted points to the terminal.

[0776] Step 4:

[0777] The terminal displays the extracted points to the user.

[0778] Closing the meeting

[0779] Summary and reporting

[0780] Step 1:

[0781] The server summarizes the discussion.

[0782] Step 2:

[0783] The server sets the allocation of roles and deadlines for the next task.

[0784] Step 3:

[0785] The server sends the summarized content and task list to the terminal.

[0786] Step 4:

[0787] The terminal provides the summary and task list to the user, who then checks the information provided.

[0788] Step 5:

[0789] The server notifies all participants of the same information.

[0790] Example 1

[0791] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0792] In traditional meeting management, the entire process of preparing, conducting, and closing a meeting is largely manual, requiring time and effort. It can also be difficult for all participants to quickly share necessary information, reducing meeting efficiency. Furthermore, tasks such as managing comments and organizing discussions during a meeting are time-consuming, making it difficult to capture important points.

[0793] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0794] In this invention, the server includes means for inputting the purpose and goals of the meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information using natural language processing, means for providing the organized materials, means for inputting a list of participants and a range of possible dates, means for checking the schedules of participants and proposing the optimal date using a generative AI model, and means for notifying and confirming the proposed date. This efficiently automates a series of processes from advance preparation to progress and closing of the meeting, reducing the burden on the organizer and enabling all participants to quickly share necessary information.

[0795] "Means for inputting the purpose and goals of the meeting" refers to an interface that allows the user to input the purpose of the meeting and the specific goals to be achieved.

[0796] "Means for automatically generating an agenda based on input purpose and goals" refers to a function that analyzes the purpose and goals of a meeting input by the user and automatically creates a meeting schedule based on them.

[0797] "Means for displaying an agenda" refers to a device or interface for visually displaying the automatically generated agenda to a user.

[0798] "Means for uploading related materials" refers to an interface that allows a user to upload documents and materials to be used in a conference to the system.

[0799] "Means for analyzing uploaded materials and organizing necessary information using natural language processing" refers to the function of scanning uploaded materials and using natural language processing technology to extract and organize important information.

[0800] "Means for providing organized material" refers to an interface for providing organized information or transformed material to a user.

[0801] "Means for inputting participant list and possible date range" refers to an interface for inputting into the system a list of people who will be attending the conference and a possible date range for the conference.

[0802] "Means of checking participants' schedules and proposing optimal dates using a generative AI model" refers to a function that collects participants' schedule information and calculates optimal meeting dates using a generative AI model.

[0803] "Means for communicating and confirming proposed dates" refers to the interface or notification system for notifying participants of proposed meeting dates and obtaining confirmation.

[0804] "Means for announcing the start of a conference by voice using voice synthesis technology" refers to a function for announcing the start of a conference by voice using voice synthesis technology.

[0805] "Means of analyzing statements in real time and detecting position-taking and long speech" refers to a function that analyzes statements made during a meeting in real time and identifies specific speech patterns and long speech.

[0806] "Means for displaying a message to suppress long speech" refers to an interface for notifying participants when a long speech is detected.

[0807] "Means for automatically organizing statements and extracting important points" refers to a function that automatically classifies and organizes statements made during a meeting and extracts important points.

[0808] "Means for providing extracted points" refers to an interface for providing automatically extracted important points to a user.

[0809] The present invention relates to a system for automating a series of processes for preparing, conducting, and closing a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[0810] Meeting preparation

[0811] Agenda creation

[0812] Users use their devices to input the purpose and goals of a meeting into the system. This input information is sent to the server, which receives it and stores it in a database. The server then uses a generative AI model to automatically generate an agenda based on the input purpose and goals. The generated agenda is then sent from the server to the device, which displays it to the user.

[0813] Specifically, if a user enters "Project X progress report and second half plan" as the purpose and "Check this month's progress and set next task" as the goal, the server will automatically generate an agenda like the one below.

[0814] 1. Opening Remarks

[0815] 2. Project X progress report

[0816] 3. Announcement of second half plan

[0817] 4. Question and Answer Session

[0818] 5. Next task setting

[0819] 6. Closing Remarks

[0820] Preparation of meeting materials

[0821] Users upload relevant materials to the server via their devices. The server scans the uploaded materials and uses natural language processing technology to extract and organize the necessary information. The server then converts the organized materials into a downloadable format (e.g., PDF or Word file). The server then provides the organized and converted materials to the device, which then displays them to the user.

[0822] Specifically, when a user uploads "minutes of the previous meeting" or "progress report," the server analyzes it, extracts important data such as progress rate and issues, and generates organized documents.

[0823] Scheduling and communication

[0824] The user uses the device to input the participant list and possible date range. This data is also sent to the server, which checks the participants' schedule information. The server then uses the generative AI model to propose the optimal meeting date. The proposed date is sent from the server to the device, which notifies the user. The user reviews the proposed date and finalizes it. The server notifies all participants of the confirmed date.

[0825] Specifically, when a user inputs a weekday morning of the following week as a possible meeting date, the server checks the calendars of the participants and suggests the optimal date that everyone can attend (e.g., 10:00 a.m. next Wednesday).

[0826] Meeting proceedings

[0827] Display projection and audio guide

[0828] When a meeting starts, the server projects a pre-generated agenda and prepared materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[0829] Specifically, at the start of the meeting, an audio announcement will be made saying "The meeting is starting," while the agenda and materials will be displayed on the projector.

[0830] Speech Management

[0831] During the meeting, the server analyzes each participant's speech in real time to detect position-taking and long speech. If detected, the information is sent to the device, which then displays a message prompting the user to moderate their speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are also sent from the server to the device, which then displays them to the user.

[0832] Specifically, when a participant is detected speaking for a long time during a meeting, the server displays the message "Your speech is long" on the terminal.

[0833] Closing the meeting

[0834] Summary and reporting

[0835] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, which provides them to the user. The server then sends the summary and task list to all participants via email or notifications.

[0836] Specifically, after the meeting ends, the server automatically generates a "list of next tasks" and sends it to all participants by email.

[0837] Prompt Sentence Examples

[0838] Enter the purpose of the meeting: "Project X progress report and second half plan"

[0839] Upload related materials: "Minutes of the previous meeting, progress report"

[0840] Enter participant list and date range: "Next weekday morning"

[0841] This system efficiently automates the entire process of a meeting, reducing the burden on organizers and enabling participants to quickly share information.

[0842] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0843] Step 1:

[0844] The user inputs the purpose and goals of the meeting.

[0845] Specifically, the user uses a terminal to input the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task."

[0846] Input: Meeting purpose and goals

[0847] Output: The input data is sent to the server.

[0848] Step 2:

[0849] The server receives the input meeting purpose and goal and stores them in a database.

[0850] Then, a generative AI model is used to auto-generate the agenda.

[0851] Specifically, an agenda is generated that includes items such as "opening remarks" and "setting tasks for the next meeting."

[0852] Input: Meeting purpose and goals

[0853] Output: Auto-generated agenda

[0854] Step 3:

[0855] The server transmits the generated agenda to the terminal.

[0856] The terminal displays the received agenda to the user.

[0857] Specifically, each item on the agenda is listed on the screen.

[0858] Input: Generated agenda

[0859] Output: The agenda is displayed in the terminal.

[0860] Step 4:

[0861] Users upload relevant materials.

[0862] "Minutes of the previous meeting" and "progress reports" are uploaded to the server via the terminal.

[0863] Input: Related materials

[0864] Output: The data is sent to the server.

[0865] Step 5:

[0866] The server receives the uploaded materials and stores them in a database.

[0867] Next, natural language processing technology is used to extract and organize the necessary information.

[0868] Specifically, important data such as progress rate and issues are extracted.

[0869] Input: Uploaded materials

[0870] Output: Organized data

[0871] Step 6:

[0872] The server converts the organized data into a downloadable format.

[0873] The terminal provides this data to the user.

[0874] Specifically, materials can be downloaded in document format (PDF or Word files).

[0875] Input: Organized data

[0876] Output: Downloadable materials

[0877] Step 7:

[0878] The user inputs a list of participants and a range of possible dates.

[0879] Enter a date range such as "morning on a weekday next week" through the terminal.

[0880] Input: Participant list and date range

[0881] Output: The input data is sent to the server

[0882] Step 8:

[0883] The server checks each participant's schedule information and uses a generative AI model to suggest the optimal date.

[0884] The proposed itinerary is again transmitted to the terminal, which notifies the user of this.

[0885] Specifically, it will display the optimal date, such as "10:00 AM next Wednesday."

[0886] Input: Participant list and date range

[0887] Output: Suggested optimal schedule

[0888] Step 9:

[0889] The user reviews and finalizes the proposed itinerary.

[0890] The server will notify all participants of the confirmed schedule.

[0891] Specifically, participants will be notified of the schedule via email or message.

[0892] Input: Proposed optimal dates

[0893] Output: Notification of confirmed schedule

[0894] Step 10:

[0895] At the start of the meeting, the server projects the agenda and meeting materials onto a projector and announces the start of the meeting using voice synthesis technology.

[0896] Specifically, a voice announcement will be made saying "The meeting is about to begin," and the agenda and materials will be displayed on the projector.

[0897] Input: Trigger for starting the meeting (date and time)

[0898] Output: Meeting opening announcement and projection

[0899] Step 11:

[0900] During the meeting, the server analyzes each participant's comments in real time and detects position-taking and long comments.

[0901] If necessary, the device will display a suppression message such as "Your speech is getting long."

[0902] Input: Speech data during the meeting

[0903] Output: Suppression message

[0904] Step 12:

[0905] At the end of the meeting, the server summarizes the discussion and sets the next tasks, assignment of roles, and deadlines.

[0906] The organized summary and task list are sent from the server to the terminal and provided to the user.

[0907] Specifically, a list of upcoming tasks will be displayed on the device and sent to all participants by email.

[0908] Input: Meeting discussion data

[0909] Output: Summary and task list notifications

[0910] This system efficiently automates the entire process of a meeting, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[0911] (Application example 1)

[0912] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0913] The process of preparing, conducting, and closing meetings has traditionally been done manually, requiring time and effort, and is prone to human error and delays. Especially on factory floors, there is a need to streamline these meeting processes and ensure they proceed quickly and accurately. Furthermore, there is a need for effective methods for individual tasks, such as coordinating participant schedules, organizing materials, and managing comments.

[0914] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0915] In this invention, the server includes means for inputting the purpose and goals of a meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information, means for providing the organized materials, means for inputting a participant list and a range of possible dates, means for checking participant schedules and proposing the optimal date, means for notifying and confirming the proposed date, means for analyzing the time and content of speech, means for issuing a warning if the speech time is too long, means for summarizing the main points and automatically setting the next tasks and role allocation, and means for notifying the main points and task information. This automates the entire process of preparing, conducting, and closing a meeting, making it possible to perform it efficiently and accurately.

[0916] The "means for inputting the purpose and goal of the meeting" refers to an input device or interface that allows the user to input the specific purpose of the meeting and the goal that the user wants to achieve into the system.

[0917] A "means for automatically generating an agenda based on input objectives and goals" is an algorithm or software that analyzes the objectives and goals of a meeting and automatically generates an effective agenda.

[0918] A "means for displaying an agenda" is a display device or display interface that allows a user to view the generated agenda.

[0919] The "means for uploading relevant materials" refers to a device or interface for uploading materials required for a conference to the system.

[0920] "Means for analyzing uploaded materials and organizing necessary information" refers to software or algorithms that analyze uploaded materials and extract and organize important information.

[0921] A "means for providing organized material" is an interface or device for providing the analyzed and organized material to a user.

[0922] The "means for inputting a participant list and a possible date range" is an interface or device for inputting a participant list and a possible date range of a conference into the system.

[0923] The "means for checking the schedules of participants and proposing the optimal date" is software or an algorithm that checks the schedules of the participants that have been input and proposes the optimal meeting date.

[0924] A "means for notifying and confirming proposed dates" is an interface or device for notifying participants of proposed meeting dates and receiving confirmation.

[0925] The "means for analyzing the time and content of speech" is software or an algorithm that analyzes the time and content of speech during a meeting in real time.

[0926] The "means for issuing a warning when a speech duration is too long" is an interface or device for detecting an excessively long speech duration and issuing a warning.

[0927] "Means for summarizing key points and automatically setting the next tasks and division of roles" refers to software or an algorithm that summarizes the key points of a meeting and automatically sets the next tasks and division of roles.

[0928] The "means for notifying key points and task information" is an interface or device for notifying all participants of the organized key points and set task information.

[0929] The present invention provides a system for automating the preparation, progress, and closing of a meeting. A specific embodiment of this system will be described below.

[0930] Meeting preparation

[0931] Agenda creation

[0932] First, the user speaks to the robot to input the purpose and goal of the meeting. The robot converts the purpose and goal of the meeting into text data using speech recognition technology and sends it to the server. The server then uses a generative AI model (for example, OpenAI's GPT-3 model) to automatically generate an agenda based on the purpose and goal. This agenda is then displayed on the robot.

[0933] Preparation of meeting materials

[0934] Users upload meeting materials to the robot using a USB memory stick or similar device. The uploaded materials are then sent to a server, which scans and analyzes them, extracting and organizing the necessary information. This process uses document analysis software, and the robot then provides the organized materials to the user.

[0935] Scheduling and communication

[0936] The user inputs a list of participants and a range of possible dates into the robot. The input data is sent to the server, which checks the participants' schedules and automatically proposes the most suitable meeting date. The robot notifies the user of this proposal by voice or on-screen display. The user confirms and confirms the final date, and the server notifies all participants of the confirmed date.

[0937] Conference moderator

[0938] Display projection and speech

[0939] When a meeting starts, the robot retrieves the pre-generated agenda and prepared meeting materials from the server, projects them onto a projector or monitor, and uses voice synthesis technology to announce the start of the meeting.

[0940] Speech Management

[0941] During the meeting, the robot analyzes each participant's speech in real time. It uses speech recognition and natural language processing technology to analyze the content and time of speech and detect long speeches and position-taking. If this is detected, the robot issues a warning and adjusts the speech as necessary. The server also automatically organizes the speech and extracts key points. This information is also provided by the robot.

[0942] Closing the meeting

[0943] Summary and reporting

[0944] At the end of the meeting, the server summarizes the content of the discussion and automatically sets the next tasks, assignment of roles, and deadlines. This summary and task list are provided to the user via the robot, and all participants are notified using the email sending function.

[0945] Specific examples

[0946] For example, to hold a progress meeting for a new product project, the user inputs to the robot, "I would like to hold a progress meeting for the new product project. The purpose is to check the progress and set the next task." The robot sends this information to the server, which generates the following agenda:

[0947] Purpose of the meeting: Check the progress of the new product project

[0948] Goal: Set the next task

[0949] Generate an agenda.

[0950] The final agenda generated by the server looks like this:

[0951] 1. Opening Remarks

[0952] 2. Project progress reports

[0953] 3. Announcement of next plan

[0954] 4. Question and Answer Session

[0955] 5. Next task setting

[0956] 6. Closing Remarks

[0957] This agenda is projected by the robot and announced aloud. The robot also analyzes comments made during the meeting, extracting and displaying important points. After the meeting ends, the server summarizes the main points of the discussion, sets the next tasks and allocation of roles, and notifies all participants.

[0958] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0959] Step 1:

[0960] The user speaks to the robot to input the purpose and goals of the meeting.

[0961] Input: Voice input of the purpose and goals of the meeting

[0962] Output: Meeting objectives and goals as text data

[0963] Data processing and calculation: The robot uses voice recognition technology (e.g., speech_recognition module) to convert voice data into text data.

[0964] Step 2:

[0965] The robot sends the converted text data to a server, which then uses a generative AI model (e.g., GPT-3) to automatically generate an agenda.

[0966] Input: Meeting objectives and goals as text data

[0967] Output: Agenda text data

[0968] Data processing and calculation: The server forms prompt sentences based on the purpose and goal, inputs them into the generative AI model, and generates an agenda.

[0969] Step 3:

[0970] The server sends the generated agenda to the robot, which displays the agenda.

[0971] Input: Agenda text data

[0972] Output: Agenda displayed on the display

[0973] Data processing and calculation: The robot displays the acquired agenda on its built-in display or an external monitor.

[0974] Step 4:

[0975] The user uploads meeting materials to the robot using a USB memory stick, etc. The robot then sends the materials to the server.

[0976] Input: Materials uploaded via USB memory or LAN connection

[0977] Output: Data transferred to the server

[0978] Data processing and calculation: The robot securely transmits the uploaded data to the server.

[0979] Step 5:

[0980] The server scans and analyzes the uploaded materials and organizes the necessary information.

[0981] Input: Uploaded material data

[0982] Output: Organized data

[0983] Data processing and calculation: The server uses document analysis software to scan the documents, extract text, and organize the information.

[0984] Step 6:

[0985] The participant list and possible date range are input to the robot, which then sends this to the server, which then suggests the best meeting date.

[0986] Input: List of participants and possible date range

[0987] Output: Proposed meeting dates

[0988] Data processing and calculation: The server checks the participants' schedules and uses an algorithm to calculate the optimal schedule.

[0989] Step 7:

[0990] The robot will notify the user of the schedule proposed by the server by voice or on-screen display, and the user will confirm it.

[0991] Input: Proposed meeting dates

[0992] Output: Confirmed or revised meeting dates

[0993] Data processing and calculation: The robot presents the proposed schedule and accepts confirmation from the user.

[0994] Step 8:

[0995] At the start of a meeting, the robot retrieves the agenda and prepared meeting materials, projects them onto a projector or monitor, and announces the start of the meeting using voice synthesis technology.

[0996] Input: Meeting agenda and materials data

[0997] Output: Agenda and materials projected on a projector or monitor, audio announcements to start

[0998] Data processing and calculation: The robot visually displays the agenda and materials and announces the start of the meeting by voice.

[0999] Step 9:

[1000] During meetings, the robot analyzes comments in real time and detects long speeches and position-taking.

[1001] Input: Participant's voice

[1002] Output: Parsed speech and warning messages

[1003] Data processing and calculation: The robot uses voice recognition and natural language processing technology to analyze the time and content of speech and display warnings as necessary.

[1004] Step 10:

[1005] At the end of the meeting, the server summarizes the content of the discussion, sets the next tasks, assignment of roles, and deadlines, and notifies all participants of the summary and task list.

[1006] Input: What was said during the meeting and notes

[1007] Output: Summary of discussion and task list

[1008] Data processing and calculation: The server uses natural language processing technology to summarize the discussion content, automatically set the next tasks and role allocation, and notify participants.

[1009] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1010] The present invention combines a system that automates a series of processes from advance preparation to progress and closing of a meeting with an emotion engine that recognizes the user's emotions, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments are described below.

[1011] Meeting preparation

[1012] Agenda creation

[1013] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[1014] Preparation of meeting materials

[1015] Users upload relevant materials to be used in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format and provides them to users via their devices.

[1016] Scheduling and communication

[1017] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[1018] Conference moderator

[1019] Display projection and speech

[1020] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[1021] Speech Management

[1022] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[1023] Additional features of the Emotion Engine

[1024] Furthermore, an emotion engine is built in that analyzes the user's speech and facial expressions to recognize their emotions. The emotion engine analyzes the audio and video data acquired by the server in real time to identify the user's emotions (e.g., joy, anger, sadness, surprise, etc.).

[1025] The server uses the recognized emotions to adjust the progress of the conference. For example, if a user is feeling stressed, the server will adjust the progress of the conference to be more gradual. Also, if a specific emotion is detected, the server will give priority to allowing the corresponding participant to speak.

[1026] Closing the meeting

[1027] Summary and reporting

[1028] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the device, which provides them to the user. The server then sends the summary and task list to all participants via email or notification. It is also possible to provide individual feedback after the meeting based on the emotional data recognized by the emotion engine.

[1029] Specific examples

[1030] For example, when hosting a project progress meeting, the user inputs the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task" into the system. Based on this, the server automatically generates the following agenda:

[1031] 1. Opening Remarks

[1032] 2. Project X progress report

[1033] 3. Announcement of second half plan

[1034] 4. Question and Answer Session

[1035] 5. Next task setting

[1036] 6. Closing Remarks

[1037] The user then uploads relevant materials (minutes of the previous meeting, progress reports, etc.) to the server, which analyzes them and organizes the necessary information. After entering the list of meeting participants and the range of possible dates, the server will suggest the optimal date and notify the user via the device.

[1038] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. Comments made during the meeting are also analyzed in real time, and messages are displayed to discourage long comments as necessary. The server also automatically extracts important comments and provides them to the user.

[1039] Furthermore, the emotion engine analyzes the remarks and facial expressions of meeting participants to recognize their emotions. For example, if a participant is feeling stressed, the server will use that information to adjust the pace of the meeting and prioritize the comments of other participants. After the meeting ends, individual feedback can be provided based on the emotional data.

[1040] This efficiently automates the entire meeting process, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need. In addition, the introduction of an emotion engine enables flexible meeting progression that takes into account the emotional state of participants.

[1041] The processing flow will be explained below.

[1042] Meeting preparation

[1043] Agenda creation

[1044] Step 1:

[1045] The user inputs the purpose and goals of the meeting using the system's interface.

[1046] Step 2:

[1047] The server receives the input objectives and goals, analyzes the information, and automatically generates an agenda based on the analysis results.

[1048] Step 3:

[1049] The server sends the generated agenda to the terminal.

[1050] Step 4:

[1051] The terminal displays the received agenda to the user, who then checks the displayed agenda.

[1052] Preparation of meeting materials

[1053] Step 1:

[1054] The user uploads the relevant materials to be used in the meeting to the server via the terminal.

[1055] Step 2:

[1056] The server scans the uploaded materials and extracts and organizes the necessary information.

[1057] Step 3:

[1058] The server converts the organized material into a downloadable format.

[1059] Step 4:

[1060] The terminal provides the converted material to the user, who then downloads and checks the provided material.

[1061] Scheduling and communication

[1062] Step 1:

[1063] The user inputs the list of participants and the available dates. The user inputs the list using the system interface.

[1064] Step 2:

[1065] The server receives the entered data and checks the schedules of all participants.

[1066] Step 3:

[1067] The server performs the analysis and suggests the best meeting date.

[1068] Step 4:

[1069] The server sends the proposed itinerary to the terminal.

[1070] Step 5:

[1071] The terminal notifies the user of the proposed schedule, and the user confirms the schedule and makes a final decision.

[1072] Step 6:

[1073] The server will notify all participants of the confirmed schedule.

[1074] Conference moderator

[1075] Display projection and speech

[1076] Step 1:

[1077] The server projects the agenda created at the start of the meeting and the prepared meeting materials onto a display.

[1078] Step 2:

[1079] The server uses voice synthesis technology to announce the start of the conference.

[1080] Speech Management

[1081] Step 1:

[1082] The server analyzes each participant's comments in real time.

[1083] Step 2:

[1084] The server detects position talk and long speech.

[1085] Step 3:

[1086] If a long speech is detected, the server sends that information to the terminal.

[1087] Step 4:

[1088] The terminal displays a message to the user at an appropriate timing to prevent long-term speech.

[1089] Organizing the discussion

[1090] Step 1:

[1091] The server automatically organizes the comments in real time.

[1092] Step 2:

[1093] The server extracts the key points.

[1094] Step 3:

[1095] The server transmits the extracted points to the terminal.

[1096] Step 4:

[1097] The terminal displays the extracted points to the user.

[1098] Additional features of the Emotion Engine

[1099] Step 1:

[1100] The server captures audio and video data in real time during the conference.

[1101] Step 2:

[1102] An emotion engine built into the server analyzes the captured data and recognizes the user's emotions.

[1103] Step 3:

[1104] The server acquires the emotion data recognized by the emotion engine and reflects it in the progress of the conference as appropriate.

[1105] Step 4:

[1106] If a user displays strong emotions, the server adjusts the conference progress and allows priority permission to speak as necessary.

[1107] Closing the meeting

[1108] Summary and reporting

[1109] Step 1:

[1110] The server summarizes the discussion at the end of the meeting.

[1111] Step 2:

[1112] The server sets the allocation of roles and deadlines for the next task.

[1113] Step 3:

[1114] The server sends the summarized content and task list to the terminal.

[1115] Step 4:

[1116] The terminal provides the summary and task list to the user, who then checks the information provided.

[1117] Step 5:

[1118] The server notifies all participants of the same information.

[1119] Step 6:

[1120] Based on the emotional data recognized by the emotion engine, the server provides individual feedback after the meeting ends.

[1121] Example 2

[1122] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1123] In conventional conference systems, the process of preparing, conducting, and closing a meeting is done manually, which takes a lot of time and effort. It is also difficult to analyze participants' comments and behavior in real time and provide appropriate feedback. Another problem is that it is difficult to conduct a meeting flexibly while taking into account participants' emotions.

[1124] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting the purpose and goal of the meeting, means for automatically generating an agenda based on the input purpose and goal, means for displaying the agenda, means for uploading materials, means for analyzing the uploaded materials and organizing the necessary information, means for providing the organized materials, means for inputting a participant list and a range of available dates, means for checking the schedules of participants and proposing an optimal date, means for notifying and confirming the proposed date, means for analyzing comments in real time, means for automatically organizing the content of comments and extracting important points, means for providing the extracted points, means for analyzing emotions and adjusting the progress of the meeting, and means for providing individual feedback based on emotion data. This efficiently automates the processes of preparing, conducting, and closing the meeting, enabling flexible meeting progress that takes into account the comments and emotions of participants.

[1125] The "means for inputting the purpose and goal of the meeting" is an interface device that allows the user to input the specific purpose of the meeting and the goal that the user wants to achieve into the system.

[1126] The "means for automatically generating an agenda" is a software module that automatically generates a meeting schedule and agenda based on the input purpose and goals of the meeting.

[1127] A "means for displaying an agenda" is a display device or digital screen that visually displays the generated agenda to a user.

[1128] A "means for uploading materials" is an interface device that allows a user to upload materials or documents to be used in a conference to the system.

[1129] The "means for analyzing uploaded materials and organizing necessary information" is a software module that scans uploaded materials and extracts and classifies necessary information.

[1130] A "means for providing organized material" is an interface or file serving system that provides the organized material in a format that is usable by the user.

[1131] The "means for inputting a participant list and a range of possible dates" is an interface device for inputting participants of a meeting and possible dates for the meeting.

[1132] The "means of checking participants' schedules and proposing the optimal date" is an algorithm that calculates and proposes the optimal meeting date based on the participants' schedules entered.

[1133] A "means for notifying and confirming proposed dates" is a notification system or confirmation interface for notifying participants of proposed meeting dates and obtaining their confirmation.

[1134] "Means for analyzing speech in real time" refers to speech recognition technology that analyzes speech made by participants in a meeting in real time and understands the content.

[1135] The "means for automatically organizing speech content and extracting important points" is a software module that automatically organizes analyzed speech content and extracts important points.

[1136] The "means for providing extracted points" is an interface device for providing the extracted important points to the user.

[1137] The "means for analyzing emotions and adjusting the progress of the meeting" is an algorithm that analyzes participants' emotions from audio and video data and adjusts the progress of the meeting based on their emotional state.

[1138] The "means for providing individual feedback based on emotion data" is an interface device for providing feedback to individual participants based on emotion analysis data.

[1139] This invention combines a system that automates the process of preparing for, conducting, and closing a meeting with an emotion engine that recognizes the user's emotions. This system is implemented mainly through the roles of the server, the terminal, and the user.

[1140] Meeting preparation

[1141] Agenda creation

[1142] The user inputs the purpose and goals of the meeting on their device. This information is sent to the server. The server analyzes the input data and automatically generates an agenda using a generative AI model. The generated agenda is sent from the server to the device, which displays it to the user.

[1143] Preparation of meeting materials

[1144] Users upload relevant materials for meetings from their devices to the server. The server scans the uploaded materials and uses OCR (optical character recognition) technology to extract and organize the necessary information. The server then converts the organized materials into downloadable formats such as PDFs or spreadsheets and provides them to users via their devices.

[1145] Scheduling and communication

[1146] The user inputs the participant list and available dates from their device and sends them to the server. The server checks each participant's schedule and uses an algorithm to propose the optimal meeting date. The proposed optimal date is sent from the server to the device, which notifies the user. The user then confirms and confirms the final date. The server notifies all participants of the confirmed date.

[1147] Conference moderator

[1148] Display projection and speech

[1149] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[1150] Speech Management

[1151] During a meeting, the server uses voice analysis technology to analyze each participant's speech in real time. If the server detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[1152] Additional features of the Emotion Engine

[1153] Furthermore, the server operates an emotion engine that analyzes users' comments and facial expressions to recognize their emotions. The emotion engine analyzes the audio and video data acquired by the server in real time to identify the user's emotions (e.g., joy, anger, sadness, surprise, etc.). The recognized emotions are used to adjust the progress of the meeting. For example, if a user is feeling stressed, the server will adjust the progress of the speech to be more gradual. Furthermore, if a specific emotion is detected, the server will give priority to allowing the corresponding participant to speak.

[1154] Closing the meeting

[1155] Summary and reporting

[1156] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the device, which provides them to the user. The server then sends the summary and task list to all participants via email or notification. It is also possible to provide individual feedback after the meeting based on the emotional data recognized by the emotion engine.

[1157] Specific examples

[1158] For example, when hosting a project progress meeting, the user inputs the purpose "Project X progress report and second half plan" and the goal "Confirm this month's progress and set next tasks" from the terminal. Based on this, the server automatically generates the following agenda:

[1159] Opening remarks

[1160] Project X progress report

[1161] Announcement of second half plan

[1162] Q&A

[1163] Next task settings

[1164] Closing remarks

[1165] Next, the user uploads relevant materials (minutes of the previous meeting, progress reports, etc.) from their device to the server, which then analyzes them and organizes the necessary information. When the user enters the participant list and available dates, the server proposes the optimal dates and notifies them via the device.

[1166] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. During the meeting, the server analyzes the content of comments and emotional data, adjusting the progress as necessary. After the meeting ends, a summary of key points and a task list are provided, and individual feedback based on the emotional data is distributed.

[1167] This system efficiently automates the entire process of a meeting, significantly reducing the burden on the organizer and enabling flexible meeting proceedings that take into account the emotions of participants.

[1168] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1169] Step 1:

[1170] The user enters the purpose and goals of the meeting.

[1171] Input: The user enters the "purpose of the meeting" and "goal of the meeting" into the input form on the terminal.

[1172] Data processing: The data entered by the user is sent from the terminal to the server.

[1173] Output: The server uses the received data for analysis.

[1174] Specific operation: The user inputs "Marketing strategy for new product" as the "Objective of the meeting" on the terminal screen, and "Determine the strategy content" as the "Goal of the meeting."

[1175] Step 2:

[1176] The server receives the objectives and goals and generates the agenda.

[1177] Input: The server receives the "meeting purpose" and "meeting goal" entered by the user.

[1178] Data computation: The server analyzes the input data and automatically generates an agenda using a generative AI model.

[1179] Output: The generated agenda is saved on the server.

[1180] Specific operation: Based on the received data, the server generates an agenda including the flow and content of the meeting.

[1181] Step 3:

[1182] The server sends the agenda to the terminal, which displays it to the user.

[1183] Input: The server has the generated agenda data.

[1184] Data processing: The server sends the agenda data to the terminal.

[1185] Output: The submitted agenda is displayed on the terminal.

[1186] Specific operation: The generated agenda is displayed on the terminal screen to the user. The user confirms it.

[1187] Step 4:

[1188] The user uploads the meeting materials to the server.

[1189] Input: The user selects and uploads relevant materials (PDFs, documents, etc.) to be used in the meeting from their terminal.

[1190] Data processing: The terminal sends the selected materials to the server.

[1191] Output: The server receives the uploaded material.

[1192] Specific operation: The user selects "minutes of the previous meeting" or "progress report" from their device and presses the upload button.

[1193] Step 5:

[1194] The server scans the documents and extracts and organizes the necessary information.

[1195] Input: The server has the uploaded data.

[1196] Data calculation: The server uses OCR technology to scan the text in the document, extract and organize the necessary information.

[1197] Output: The extracted and organized information is saved on the server.

[1198] What it does: The server parses the uploaded PDF and extracts important text data.

[1199] Step 6:

[1200] The server transmits the organized materials to the terminal, which then provides them to the user.

[1201] Input: The server has organized data.

[1202] Data processing: The server converts the organized materials into PDF or spreadsheet format and sends them to the terminal.

[1203] Output: The transmitted material is displayed on the terminal and provided to the user.

[1204] Specific operation: The user can view and download the organized materials displayed on the terminal.

[1205] Step 7:

[1206] The user enters the participant list and date range.

[1207] Input: The user inputs the participant list and possible date range of the meeting from the terminal.

[1208] Data processing: The terminal sends the input information to the server.

[1209] Output: The server uses the received data for analysis.

[1210] Specific operation: The user uploads the participant list in an Excel file and selects and enters the available date range.

[1211] Step 8:

[1212] The server will suggest the best schedule and notify the user via the terminal.

[1213] Input: The server has the attendee list and date range data.

[1214] Data calculation: The server checks the schedules of each participant and uses an algorithm to calculate the optimal meeting date.

[1215] Output: The proposed optimal schedule is stored on the server and sent to the terminal.

[1216] Specific operation: A notification suggesting the best date will be displayed on the user's device.

[1217] Step 9:

[1218] The user checks and confirms the final schedule.

[1219] Input: The user confirms the proposed itinerary on the terminal.

[1220] Data processing: When the user presses the confirm button, the information is sent to the server.

[1221] Output: The confirmed schedule is saved on the server.

[1222] Specific operation: The user checks the proposed schedule and clicks the confirm button.

[1223] Step 10:

[1224] The server will notify all participants of the confirmed schedule.

[1225] Input: The server has the confirmed schedule data.

[1226] Data processing: The server sends the confirmed schedule to all participants via email or notification.

[1227] Output: All participants are notified of the meeting date.

[1228] Specific behavior: A schedule notification will be sent to all participants via email.

[1229] Step 11:

[1230] At the start of the meeting, the server projects the agenda and materials and announces the start of the meeting with a voice message.

[1231] Input: The server has the generated agenda and organized document data.

[1232] Data processing: The server projects the agenda and materials onto a projector and announces the start of the meeting using voice synthesis technology.

[1233] Output: Projected agenda and materials, audio start announcement.

[1234] Specific operation: The meeting agenda and materials are displayed on the projector, and an automated voice announces the start of the meeting.

[1235] Step 12:

[1236] The server analyzes the comments in real time and sends messages to the device as needed.

[1237] Input: Audio data during the conference is sent to the server.

[1238] Data calculation: The server performs real-time voice analysis and determines the content of the speech.

[1239] Output: If position talk or long speech is detected, the information is sent to the device.

[1240] Specific behavior: A warning message will be displayed on the user's device to discourage long-term comments.

[1241] Step 13:

[1242] The server organizes and extracts the content of comments and provides it to the user.

[1243] Input: Speech data analyzed during the meeting is saved on the server.

[1244] Data calculation: The server runs an algorithm to extract important points.

[1245] Output: The extracted key points are stored on the server and sent to the device.

[1246] Specific operation: A summary of important comments is displayed on the user's device.

[1247] Step 14:

[1248] The server uses an emotion engine to analyze and recognize participants' emotions.

[1249] Input: Audio and video data during the conference are sent to the server.

[1250] Data calculation: The emotion engine analyzes data in real time and recognizes emotions.

[1251] Output: The recognized emotion data is stored on the server.

[1252] Specific operation: The server monitors the participants' emotional states based on the emotional data analyzed.

[1253] Step 15:

[1254] The server adjusts the progress of the conference based on the emotion data.

[1255] Input: Emotion data stored on the server.

[1256] Data processing: The server runs an algorithm to adjust the progress of the meeting based on the emotion data.

[1257] Output: Adjusted meeting progress data.

[1258] Specific actions: For example, adjustments may be made such as slowing down the pace for participants who are feeling stressed, or prioritizing the comments of participants who have detected certain emotions.

[1259] Step 16:

[1260] The server organizes the key points and sets tasks and roles.

[1261] Input: Speech and emotion data captured during the meeting.

[1262] Data calculation: The server summarizes the discussion and sets the next task and allocation of roles.

[1263] Output: A concise summary and a task list.

[1264] Specific operation: The server generates summarized discussion points and task lists and compiles them as materials for the next meeting.

[1265] Step 17:

[1266] The server sends the summary and task list to all participants.

[1267] Input: An organized summary and a task list.

[1268] Data processing: The server sends the summary and task list to all participants.

[1269] Output: Summary and task list sent to all participants.

[1270] What happens: A summary and task list will be sent to all participants via email and notifications.

[1271] Step 18:

[1272] After the meeting, the server provides individual feedback based on the emotional data.

[1273] Input: Emotion data collected during the meeting.

[1274] Data calculation: The server creates individual feedback based on the analysis results.

[1275] Output: Individual feedback report.

[1276] Specific operation: The server generates individual feedback for each participant based on the emotional data and provides it via email or notification.

[1277] (Application example 2)

[1278] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1279] In modern factory environments, there is a demand for improved work efficiency and strengthened quality control. However, there is also the issue that emotional factors such as worker stress and fatigue have a significant impact on work efficiency. This has led to issues that traditional work management methods are no longer able to address. Furthermore, when errors occur during work, a system is needed to respond immediately, but traditional systems have issues with responsiveness. Furthermore, creating reports after task completion takes time, so efficient improvement measures are needed.

[1280] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for inputting the purpose and goal of the meeting; a means for automatically generating an agenda based on the input purpose and goal; a means for displaying the agenda; a means for uploading related materials; a means for analyzing the uploaded materials and organizing the necessary information; a means for providing the organized materials; a means for inputting a participant list and a range of possible dates; a means for checking the schedules of participants and proposing an optimal date; a means for notifying and confirming the proposed date; a means for acquiring task data and analyzing it using machine learning technology; a means for acquiring voice and video data of workers in real time; a means for analyzing the acquired data using an emotion analysis engine; a means for detecting errors and notifying workers in real time; and a means for automatically generating a report after task completion. This enables improved work efficiency and enhanced quality control in factories. Furthermore, analyzing and responding to workers' emotional states in real time can also improve the work environment. In addition, immediate response to errors and automatic report generation enable rapid implementation of corrective measures.

[1281] "Meeting objectives and goals" refers to the specific goals that are to be achieved by holding the meeting and the results that are to be achieved at the end of the meeting.

[1282] An "agenda" refers to a list of topics or items set in advance to plan the progress of a meeting or work.

[1283] "Related materials" refers to documents and reference information required for the progress of meetings and work.

[1284] "Upload" refers to the act of sending data from a device to a remote system such as a server or cloud.

[1285] "Machine learning technology" refers to computer programs and algorithms that automatically learn patterns from data and make subsequent predictions or decisions.

[1286] "Emotion analysis engine" refers to software and algorithms that analyze audio and video data and identify human emotional states.

[1287] "Real-time" refers to data or events being processed and analyzed almost immediately after they occur.

[1288] "Error detection" refers to the act of identifying anomalies or errors that occur during a task or system process.

[1289] "Report" refers to a document or digital compilation of the content, progress, and results of a meeting or task.

[1290] MODE FOR CARRYING OUT THE INVENTION

[1291] This invention is a system for improving the work efficiency and quality control of factory robots, which is implemented through the roles of a server, a terminal, and a user. This system is based on a conference automation system and focuses on specific applications in a factory environment.

[1292] Task setting and advance preparation

[1293] The server sends work task data to the factory robots. The task data is analyzed on the server using machine learning technology (such as TensorFlow or PyTorch) to generate specific work procedures. The generated procedures are then sent as instructions to the factory robots.

[1294] Real-time progress management

[1295] The device is equipped with a camera (such as an Intel RealSense or Raspberry Pi camera module) and microphone to capture the worker's voice and video data. The captured data is then sent to a server where it is analyzed in real time by an emotion analysis engine (using OpenCV and NLP technology). This allows the worker's emotional state to be identified and work procedures to be adjusted as necessary.

[1296] Error Detection and Notification

[1297] The server scans the acquired work data in real time and detects errors using machine learning models such as TensorFlow. If an error is detected, the worker is immediately notified via their device. The notification also includes appropriate countermeasures.

[1298] Closing and reporting on task completion

[1299] Once a task is completed, the server automatically organizes the work that has been performed and generates a report. The report, which clarifies the next task setting and areas for improvement, is provided to the worker via their terminal. The generated report helps improve overall work efficiency and quality.

[1300] Specific examples

[1301] For example, on a precision assembly line, it operates as follows:

[1302] 1. Task: Attach part A to B of a specific product.

[1303] 2. The server acquires the task data, analyzes it, and generates a work procedure.

[1304] 3. The voice and video data of workers captured through cameras and microphones is analyzed using an emotion analysis engine, and if a state of stress is detected, work procedures are adjusted.

[1305] 4. If an error occurs during work, the server will immediately notify you and provide you with the appropriate solution.

[1306] 5. Once the task is completed, the server automatically generates a report and provides the worker with information on the next task settings and improvements.

[1307] Prompt Sentence Examples

[1308] "A factory work supervision application based on a conference automation system. It includes functions such as specific task setting, optimizing work efficiency through real-time sentiment analysis, error detection, and work report generation."

[1309] This system dramatically improves work efficiency and quality within the factory and reduces the burden on workers. Real-time error detection and immediate response, an improved work environment through emotion analysis, and automatic report generation enable more flexible and efficient work management.

[1310] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1311] Step 1:

[1312] The server receives data from the user to input the purpose and goals of the meeting. The input purpose and goal data is processed within the server as initial data for automatic agenda generation. Here, the server uses natural language processing (NLP) technology to analyze the input data and generate agenda items. The generated agenda is sent to the terminal.

[1313] Step 2:

[1314] The device displays the agenda sent from the server. At this time, the user uploads relevant materials to the server via the device. The server analyzes the uploaded materials and organizes the necessary information. This analysis process uses OCR technology (optical character recognition) and machine learning algorithms to automatically extract useful information from the documents. The organized materials are then sent back to the device and provided to the user.

[1315] Step 3:

[1316] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and uses a calendar API or scheduling algorithm to suggest the best meeting date. The server sends the suggested best date to the device, which notifies the user. Once the user confirms and confirms the final date, the server notifies all participants of the confirmed date.

[1317] Step 4:

[1318] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and simultaneously announces the start of the meeting using speech synthesis technology. The input data are the agenda items and meeting materials, and the output is a visual and audio announcement of the start of the meeting.

[1319] Step 5:

[1320] During a meeting, the device captures the voice and video data of the participants in real time through its equipped camera and microphone. The captured data is sent to a server and analyzed by an emotion analysis engine. This includes, for example, facial expression analysis using OpenCV and voice analysis using NLP technology. If the analysis reveals that the user is feeling stressed, this information is recorded on the server and the progress of the meeting is adjusted.

[1321] Step 6:

[1322] The server analyzes speech in real time during meetings to detect long speeches and position-taking. It uses NLP technology to analyze speech content and detects speeches of a specific length or that focus on a specific topic. The detected information is sent to the device and displayed as a message.

[1323] Step 7:

[1324] At the end of the meeting, the server summarizes the discussion and sets the next task assignment and deadline. The summary and task list are sent from the server to the device, which then provides them to the user. Furthermore, the server generates individual feedback based on the emotional data recognized by the emotion analysis engine and sends it to the device. This provides a detailed meeting report to all participants.

[1325] Prompt Sentence Examples

[1326] "A factory work supervision application based on a conference automation system. It includes functions such as specific task setting, optimizing work efficiency through real-time sentiment analysis, error detection, and work report generation."

[1327] This dramatically improves work efficiency and quality within the factory and reduces the burden on workers. Real-time error detection and immediate response, an improved work environment through emotion analysis, and automatic report generation enable more flexible and efficient work management.

[1328] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1329] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1330] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1331] [Third embodiment]

[1332] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1333] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1334] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1335] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1336] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1337] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1338] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1339] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1340] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1341] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1342] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1343] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1344] The present invention relates to a system for automating a series of processes from advance preparation to progress and closing of a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[1345] Meeting preparation

[1346] Agenda creation

[1347] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[1348] Preparation of meeting materials

[1349] Users upload relevant materials for use in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format. The terminal then provides the converted materials to the user.

[1350] Scheduling and communication

[1351] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[1352] Conference moderator

[1353] Display projection and speech

[1354] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[1355] Speech Management

[1356] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[1357] Closing the meeting

[1358] Summary and reporting

[1359] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, and the terminal provides them to the user. Furthermore, the server sends the summary and task list to all participants using email or notifications.

[1360] Specific examples

[1361] For example, when hosting a project progress meeting, the user inputs the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task" into the system. Based on this, the server automatically generates the following agenda:

[1362] 1. Opening Remarks

[1363] 2. Project X progress report

[1364] 3. Announcement of second half plan

[1365] 4. Question and Answer Session

[1366] 5. Next task setting

[1367] 6. Closing Remarks

[1368] The user then uploads relevant materials (minutes of the previous meeting, progress reports, etc.) to the server, which analyzes them and organizes the necessary information. When the user inputs the list of meeting participants and a range of possible dates (e.g., a weekday morning next week), the server proposes the optimal date and notifies the user via the device.

[1369] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. Speech during the meeting is also analyzed in real time, and messages are displayed to limit long speeches as necessary. Furthermore, the server automatically extracts important speech content and provides it to the user.

[1370] After the meeting, the server summarizes the discussion and sets the next task, its assignment, and deadline. The summary and task list are provided to the user via their terminal and reported to all participants.

[1371] This efficiently automates the entire meeting process, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[1372] The processing flow will be explained below.

[1373] Meeting preparation

[1374] Agenda creation

[1375] Step 1:

[1376] The user inputs the purpose and goals of the meeting using the system's interface.

[1377] Step 2:

[1378] The server receives the input objectives and goals, analyzes the information, and automatically generates an agenda based on the analysis results.

[1379] Step 3:

[1380] The server sends the generated agenda to the terminal.

[1381] Step 4:

[1382] The terminal displays the received agenda to the user, who then checks the displayed agenda.

[1383] Preparation of meeting materials

[1384] Step 1:

[1385] The user uploads the relevant materials to be used in the meeting to the server via the terminal.

[1386] Step 2:

[1387] The server scans the uploaded materials and extracts and organizes the necessary information.

[1388] Step 3:

[1389] The server converts the organized material into a downloadable format.

[1390] Step 4:

[1391] The terminal provides the converted material to the user, who then downloads and checks the provided material.

[1392] Scheduling and communication

[1393] Step 1:

[1394] The user inputs the list of participants and the available dates. The user inputs the list using the system interface.

[1395] Step 2:

[1396] The server receives the entered data and checks the schedules of all participants.

[1397] Step 3:

[1398] The server performs the analysis and suggests the best meeting date.

[1399] Step 4:

[1400] The server sends the proposed itinerary to the terminal.

[1401] Step 5:

[1402] The terminal notifies the user of the proposed schedule, and the user confirms the schedule and makes a final decision.

[1403] Step 6:

[1404] The server will notify all participants of the confirmed schedule.

[1405] Conference moderator

[1406] Display projection and speech

[1407] Step 1:

[1408] The server projects the agenda created at the start of the meeting and the prepared meeting materials onto a display.

[1409] Step 2:

[1410] The server uses voice synthesis technology to announce the start of the conference.

[1411] Speech Management

[1412] Step 1:

[1413] The server analyzes each participant's comments in real time.

[1414] Step 2:

[1415] The server detects position talk and long speech.

[1416] Step 3:

[1417] If a long speech is detected, the server sends that information to the terminal.

[1418] Step 4:

[1419] The terminal displays a message to the user at an appropriate timing to prevent long-term speech.

[1420] Organizing the discussion

[1421] Step 1:

[1422] The server automatically organizes the comments in real time.

[1423] Step 2:

[1424] The server extracts the key points.

[1425] Step 3:

[1426] The server transmits the extracted points to the terminal.

[1427] Step 4:

[1428] The terminal displays the extracted points to the user.

[1429] Closing the meeting

[1430] Summary and reporting

[1431] Step 1:

[1432] The server summarizes the discussion.

[1433] Step 2:

[1434] The server sets the allocation of roles and deadlines for the next task.

[1435] Step 3:

[1436] The server sends the summarized content and task list to the terminal.

[1437] Step 4:

[1438] The terminal provides the summary and task list to the user, who then checks the information provided.

[1439] Step 5:

[1440] The server notifies all participants of the same information.

[1441] Example 1

[1442] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1443] In traditional meeting management, the entire process of preparing, conducting, and closing a meeting is largely manual, requiring time and effort. It can also be difficult for all participants to quickly share necessary information, reducing meeting efficiency. Furthermore, tasks such as managing comments and organizing discussions during a meeting are time-consuming, making it difficult to capture important points.

[1444] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1445] In this invention, the server includes means for inputting the purpose and goals of the meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information using natural language processing, means for providing the organized materials, means for inputting a list of participants and a range of possible dates, means for checking the schedules of participants and proposing the optimal date using a generative AI model, and means for notifying and confirming the proposed date. This efficiently automates a series of processes from advance preparation to progress and closing of the meeting, reducing the burden on the organizer and enabling all participants to quickly share necessary information.

[1446] "Means for inputting the purpose and goals of the meeting" refers to an interface that allows the user to input the purpose of the meeting and the specific goals to be achieved.

[1447] "Means for automatically generating an agenda based on input purpose and goals" refers to a function that analyzes the purpose and goals of a meeting input by the user and automatically creates a meeting schedule based on them.

[1448] "Means for displaying an agenda" refers to a device or interface for visually displaying the automatically generated agenda to a user.

[1449] "Means for uploading related materials" refers to an interface that allows a user to upload documents and materials to be used in a conference to the system.

[1450] "Means for analyzing uploaded materials and organizing necessary information using natural language processing" refers to the function of scanning uploaded materials and using natural language processing technology to extract and organize important information.

[1451] "Means for providing organized material" refers to an interface for providing organized information or transformed material to a user.

[1452] "Means for inputting participant list and possible date range" refers to an interface for inputting into the system a list of people who will be attending the conference and a possible date range for the conference.

[1453] "Means of checking participants' schedules and proposing optimal dates using a generative AI model" refers to a function that collects participants' schedule information and calculates optimal meeting dates using a generative AI model.

[1454] "Means for communicating and confirming proposed dates" refers to the interface or notification system for notifying participants of proposed meeting dates and obtaining confirmation.

[1455] "Means for announcing the start of a conference by voice using voice synthesis technology" refers to a function for announcing the start of a conference by voice using voice synthesis technology.

[1456] "Means of analyzing statements in real time and detecting position-taking and long speech" refers to a function that analyzes statements made during a meeting in real time and identifies specific speech patterns and long speech.

[1457] "Means for displaying a message to suppress long speech" refers to an interface for notifying participants when a long speech is detected.

[1458] "Means for automatically organizing statements and extracting important points" refers to a function that automatically classifies and organizes statements made during a meeting and extracts important points.

[1459] "Means for providing extracted points" refers to an interface for providing automatically extracted important points to a user.

[1460] The present invention relates to a system for automating a series of processes for preparing, conducting, and closing a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[1461] Meeting preparation

[1462] Agenda creation

[1463] Users use their devices to input the purpose and goals of a meeting into the system. This input information is sent to the server, which receives it and stores it in a database. The server then uses a generative AI model to automatically generate an agenda based on the input purpose and goals. The generated agenda is then sent from the server to the device, which displays it to the user.

[1464] Specifically, if a user enters "Project X progress report and second half plan" as the purpose and "Check this month's progress and set next task" as the goal, the server will automatically generate an agenda like the one below.

[1465] 1. Opening Remarks

[1466] 2. Project X progress report

[1467] 3. Announcement of second half plan

[1468] 4. Question and Answer Session

[1469] 5. Next task setting

[1470] 6. Closing Remarks

[1471] Preparation of meeting materials

[1472] Users upload relevant materials to the server via their devices. The server scans the uploaded materials and uses natural language processing technology to extract and organize the necessary information. The server then converts the organized materials into a downloadable format (e.g., PDF or Word file). The server then provides the organized and converted materials to the device, which then displays them to the user.

[1473] Specifically, when a user uploads "minutes of the previous meeting" or "progress report," the server analyzes it, extracts important data such as progress rate and issues, and generates organized documents.

[1474] Scheduling and communication

[1475] The user uses the device to input the participant list and possible date range. This data is also sent to the server, which checks the participants' schedule information. The server then uses the generative AI model to propose the optimal meeting date. The proposed date is sent from the server to the device, which notifies the user. The user reviews the proposed date and finalizes it. The server notifies all participants of the confirmed date.

[1476] Specifically, when a user inputs a weekday morning of the following week as a possible meeting date, the server checks the calendars of the participants and suggests the optimal date that everyone can attend (e.g., 10:00 a.m. next Wednesday).

[1477] Meeting proceedings

[1478] Display projection and audio guide

[1479] When a meeting starts, the server projects a pre-generated agenda and prepared materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[1480] Specifically, at the start of the meeting, an audio announcement will be made saying "The meeting is starting," while the agenda and materials will be displayed on the projector.

[1481] Speech Management

[1482] During the meeting, the server analyzes each participant's speech in real time to detect position-taking and long speech. If detected, the information is sent to the device, which then displays a message prompting the user to moderate their speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are also sent from the server to the device, which then displays them to the user.

[1483] Specifically, when a participant is detected speaking for a long time during a meeting, the server displays the message "Your speech is long" on the terminal.

[1484] Closing the meeting

[1485] Summary and reporting

[1486] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, which provides them to the user. The server then sends the summary and task list to all participants via email or notifications.

[1487] Specifically, after the meeting ends, the server automatically generates a "list of next tasks" and sends it to all participants by email.

[1488] Prompt Sentence Examples

[1489] Enter the purpose of the meeting: "Project X progress report and second half plan"

[1490] Upload related materials: "Minutes of the previous meeting, progress report"

[1491] Enter participant list and date range: "Next weekday morning"

[1492] This system efficiently automates the entire process of a meeting, reducing the burden on organizers and enabling participants to quickly share information.

[1493] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1494] Step 1:

[1495] The user inputs the purpose and goals of the meeting.

[1496] Specifically, the user uses a terminal to input the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task."

[1497] Input: Meeting purpose and goals

[1498] Output: The input data is sent to the server.

[1499] Step 2:

[1500] The server receives the input meeting purpose and goal and stores them in a database.

[1501] Then, a generative AI model is used to auto-generate the agenda.

[1502] Specifically, an agenda is generated that includes items such as "opening remarks" and "setting tasks for the next meeting."

[1503] Input: Meeting purpose and goals

[1504] Output: Auto-generated agenda

[1505] Step 3:

[1506] The server transmits the generated agenda to the terminal.

[1507] The terminal displays the received agenda to the user.

[1508] Specifically, each item on the agenda is listed on the screen.

[1509] Input: Generated agenda

[1510] Output: The agenda is displayed in the terminal.

[1511] Step 4:

[1512] Users upload relevant materials.

[1513] "Minutes of the previous meeting" and "progress reports" are uploaded to the server via the terminal.

[1514] Input: Related materials

[1515] Output: The data is sent to the server.

[1516] Step 5:

[1517] The server receives the uploaded materials and stores them in a database.

[1518] Next, natural language processing technology is used to extract and organize the necessary information.

[1519] Specifically, important data such as progress rate and issues are extracted.

[1520] Input: Uploaded materials

[1521] Output: Organized data

[1522] Step 6:

[1523] The server converts the organized data into a downloadable format.

[1524] The terminal provides this data to the user.

[1525] Specifically, materials can be downloaded in document format (PDF or Word files).

[1526] Input: Organized data

[1527] Output: Downloadable materials

[1528] Step 7:

[1529] The user inputs a list of participants and a range of possible dates.

[1530] Enter a date range such as "morning on a weekday next week" through the terminal.

[1531] Input: Participant list and date range

[1532] Output: The input data is sent to the server

[1533] Step 8:

[1534] The server checks each participant's schedule information and uses a generative AI model to suggest the optimal date.

[1535] The proposed itinerary is again transmitted to the terminal, which notifies the user of this.

[1536] Specifically, it will display the optimal date, such as "10:00 AM next Wednesday."

[1537] Input: Participant list and date range

[1538] Output: Suggested optimal schedule

[1539] Step 9:

[1540] The user reviews and finalizes the proposed itinerary.

[1541] The server will notify all participants of the confirmed schedule.

[1542] Specifically, participants will be notified of the schedule via email or message.

[1543] Input: Proposed optimal dates

[1544] Output: Notification of confirmed schedule

[1545] Step 10:

[1546] At the start of the meeting, the server projects the agenda and meeting materials onto a projector and announces the start of the meeting using voice synthesis technology.

[1547] Specifically, a voice announcement will be made saying "The meeting is about to begin," and the agenda and materials will be displayed on the projector.

[1548] Input: Trigger for starting the meeting (date and time)

[1549] Output: Meeting opening announcement and projection

[1550] Step 11:

[1551] During the meeting, the server analyzes each participant's comments in real time and detects position-taking and long comments.

[1552] If necessary, the device will display a suppression message such as "Your speech is getting long."

[1553] Input: Speech data during the meeting

[1554] Output: Suppression message

[1555] Step 12:

[1556] At the end of the meeting, the server summarizes the discussion and sets the next tasks, assignment of roles, and deadlines.

[1557] The organized summary and task list are sent from the server to the terminal and provided to the user.

[1558] Specifically, a list of upcoming tasks will be displayed on the device and sent to all participants by email.

[1559] Input: Meeting discussion data

[1560] Output: Summary and task list notifications

[1561] This system efficiently automates the entire process of a meeting, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[1562] (Application example 1)

[1563] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1564] The process of preparing, conducting, and closing meetings has traditionally been done manually, requiring time and effort, and is prone to human error and delays. Especially on factory floors, there is a need to streamline these meeting processes and ensure they proceed quickly and accurately. Furthermore, there is a need for effective methods for individual tasks, such as coordinating participant schedules, organizing materials, and managing comments.

[1565] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1566] In this invention, the server includes means for inputting the purpose and goals of a meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information, means for providing the organized materials, means for inputting a participant list and a range of possible dates, means for checking participant schedules and proposing the optimal date, means for notifying and confirming the proposed date, means for analyzing the time and content of speech, means for issuing a warning if the speech time is too long, means for summarizing the main points and automatically setting the next tasks and role allocation, and means for notifying the main points and task information. This automates the entire process of preparing, conducting, and closing a meeting, making it possible to perform it efficiently and accurately.

[1567] The "means for inputting the purpose and goal of the meeting" refers to an input device or interface that allows the user to input the specific purpose of the meeting and the goal that the user wants to achieve into the system.

[1568] A "means for automatically generating an agenda based on input objectives and goals" is an algorithm or software that analyzes the objectives and goals of a meeting and automatically generates an effective agenda.

[1569] A "means for displaying an agenda" is a display device or display interface that allows a user to view the generated agenda.

[1570] The "means for uploading relevant materials" refers to a device or interface for uploading materials required for a conference to the system.

[1571] "Means for analyzing uploaded materials and organizing necessary information" refers to software or algorithms that analyze uploaded materials and extract and organize important information.

[1572] A "means for providing organized material" is an interface or device for providing the analyzed and organized material to a user.

[1573] The "means for inputting a participant list and a possible date range" is an interface or device for inputting a participant list and a possible date range of a conference into the system.

[1574] The "means for checking the schedules of participants and proposing the optimal date" is software or an algorithm that checks the schedules of the participants that have been input and proposes the optimal meeting date.

[1575] A "means for notifying and confirming proposed dates" is an interface or device for notifying participants of proposed meeting dates and receiving confirmation.

[1576] The "means for analyzing the time and content of speech" is software or an algorithm that analyzes the time and content of speech during a meeting in real time.

[1577] The "means for issuing a warning when a speech duration is too long" is an interface or device for detecting an excessively long speech duration and issuing a warning.

[1578] "Means for summarizing key points and automatically setting the next tasks and division of roles" refers to software or an algorithm that summarizes the key points of a meeting and automatically sets the next tasks and division of roles.

[1579] The "means for notifying key points and task information" is an interface or device for notifying all participants of the organized key points and set task information.

[1580] The present invention provides a system for automating the preparation, progress, and closing of a meeting. A specific embodiment of this system will be described below.

[1581] Meeting preparation

[1582] Agenda creation

[1583] First, the user speaks to the robot to input the purpose and goal of the meeting. The robot converts the purpose and goal of the meeting into text data using speech recognition technology and sends it to the server. The server then uses a generative AI model (for example, OpenAI's GPT-3 model) to automatically generate an agenda based on the purpose and goal. This agenda is then displayed on the robot.

[1584] Preparation of meeting materials

[1585] Users upload meeting materials to the robot using a USB memory stick or similar device. The uploaded materials are then sent to a server, which scans and analyzes them, extracting and organizing the necessary information. This process uses document analysis software, and the robot then provides the organized materials to the user.

[1586] Scheduling and communication

[1587] The user inputs a list of participants and a range of possible dates into the robot. The input data is sent to the server, which checks the participants' schedules and automatically proposes the most suitable meeting date. The robot notifies the user of this proposal by voice or on-screen display. The user confirms and confirms the final date, and the server notifies all participants of the confirmed date.

[1588] Conference moderator

[1589] Display projection and speech

[1590] When a meeting starts, the robot retrieves the pre-generated agenda and prepared meeting materials from the server, projects them onto a projector or monitor, and uses voice synthesis technology to announce the start of the meeting.

[1591] Speech Management

[1592] During the meeting, the robot analyzes each participant's speech in real time. It uses speech recognition and natural language processing technology to analyze the content and time of speech and detect long speeches and position-taking. If this is detected, the robot issues a warning and adjusts the speech as necessary. The server also automatically organizes the speech and extracts key points. This information is also provided by the robot.

[1593] Closing the meeting

[1594] Summary and reporting

[1595] At the end of the meeting, the server summarizes the content of the discussion and automatically sets the next tasks, assignment of roles, and deadlines. This summary and task list are provided to the user via the robot, and all participants are notified using the email sending function.

[1596] Specific examples

[1597] For example, to hold a progress meeting for a new product project, the user inputs to the robot, "I would like to hold a progress meeting for the new product project. The purpose is to check the progress and set the next task." The robot sends this information to the server, which generates the following agenda:

[1598] Purpose of the meeting: Check the progress of the new product project

[1599] Goal: Set the next task

[1600] Generate an agenda.

[1601] The final agenda generated by the server looks like this:

[1602] 1. Opening Remarks

[1603] 2. Project progress reports

[1604] 3. Announcement of next plan

[1605] 4. Question and Answer Session

[1606] 5. Next task setting

[1607] 6. Closing Remarks

[1608] This agenda is projected by the robot and announced aloud. The robot also analyzes comments made during the meeting, extracting and displaying important points. After the meeting ends, the server summarizes the main points of the discussion, sets the next tasks and allocation of roles, and notifies all participants.

[1609] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1610] Step 1:

[1611] The user speaks to the robot to input the purpose and goals of the meeting.

[1612] Input: Voice input of the purpose and goals of the meeting

[1613] Output: Meeting objectives and goals as text data

[1614] Data processing and calculation: The robot uses voice recognition technology (e.g., speech_recognition module) to convert voice data into text data.

[1615] Step 2:

[1616] The robot sends the converted text data to a server, which then uses a generative AI model (e.g., GPT-3) to automatically generate an agenda.

[1617] Input: Meeting objectives and goals as text data

[1618] Output: Agenda text data

[1619] Data processing and calculation: The server forms prompt sentences based on the purpose and goal, inputs them into the generative AI model, and generates an agenda.

[1620] Step 3:

[1621] The server sends the generated agenda to the robot, which displays the agenda.

[1622] Input: Agenda text data

[1623] Output: Agenda displayed on the display

[1624] Data processing and calculation: The robot displays the acquired agenda on its built-in display or an external monitor.

[1625] Step 4:

[1626] The user uploads meeting materials to the robot using a USB memory stick, etc. The robot then sends the materials to the server.

[1627] Input: Materials uploaded via USB memory or LAN connection

[1628] Output: Data transferred to the server

[1629] Data processing and calculation: The robot securely transmits the uploaded data to the server.

[1630] Step 5:

[1631] The server scans and analyzes the uploaded materials and organizes the necessary information.

[1632] Input: Uploaded material data

[1633] Output: Organized data

[1634] Data processing and calculation: The server uses document analysis software to scan the documents, extract text, and organize the information.

[1635] Step 6:

[1636] The participant list and possible date range are input to the robot, which then sends this to the server, which then suggests the best meeting date.

[1637] Input: List of participants and possible date range

[1638] Output: Proposed meeting dates

[1639] Data processing and calculation: The server checks the participants' schedules and uses an algorithm to calculate the optimal schedule.

[1640] Step 7:

[1641] The robot will notify the user of the schedule proposed by the server by voice or on-screen display, and the user will confirm it.

[1642] Input: Proposed meeting dates

[1643] Output: Confirmed or revised meeting dates

[1644] Data processing and calculation: The robot presents the proposed schedule and accepts confirmation from the user.

[1645] Step 8:

[1646] At the start of a meeting, the robot retrieves the agenda and prepared meeting materials, projects them onto a projector or monitor, and announces the start of the meeting using voice synthesis technology.

[1647] Input: Meeting agenda and materials data

[1648] Output: Agenda and materials projected on a projector or monitor, audio announcements to start

[1649] Data processing and calculation: The robot visually displays the agenda and materials and announces the start of the meeting by voice.

[1650] Step 9:

[1651] During meetings, the robot analyzes comments in real time and detects long speeches and position-taking.

[1652] Input: Participant's voice

[1653] Output: Parsed speech and warning messages

[1654] Data processing and calculation: The robot uses voice recognition and natural language processing technology to analyze the time and content of speech and display warnings as necessary.

[1655] Step 10:

[1656] At the end of the meeting, the server summarizes the content of the discussion, sets the next tasks, assignment of roles, and deadlines, and notifies all participants of the summary and task list.

[1657] Input: What was said during the meeting and notes

[1658] Output: Summary of discussion and task list

[1659] Data processing and calculation: The server uses natural language processing technology to summarize the discussion content, automatically set the next tasks and role allocation, and notify participants.

[1660] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1661] The present invention combines a system that automates a series of processes from advance preparation to progress and closing of a meeting with an emotion engine that recognizes the user's emotions, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments are described below.

[1662] Meeting preparation

[1663] Agenda creation

[1664] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[1665] Preparation of meeting materials

[1666] Users upload relevant materials to be used in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format and provides them to users via their devices.

[1667] Scheduling and communication

[1668] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[1669] Conference moderator

[1670] Display projection and speech

[1671] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[1672] Speech Management

[1673] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[1674] Additional features of the Emotion Engine

[1675] Furthermore, an emotion engine is built in that analyzes the user's speech and facial expressions to recognize their emotions. The emotion engine analyzes the audio and video data acquired by the server in real time to identify the user's emotions (e.g., joy, anger, sadness, surprise, etc.).

[1676] The server uses the recognized emotions to adjust the progress of the conference. For example, if a user is feeling stressed, the server will adjust the progress of the conference to be more gradual. Also, if a specific emotion is detected, the server will give priority to allowing the corresponding participant to speak.

[1677] Closing the meeting

[1678] Summary and reporting

[1679] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the device, which provides them to the user. The server then sends the summary and task list to all participants via email or notification. It is also possible to provide individual feedback after the meeting based on the emotional data recognized by the emotion engine.

[1680] Specific examples

[1681] For example, when hosting a project progress meeting, the user inputs the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task" into the system. Based on this, the server automatically generates the following agenda:

[1682] 1. Opening Remarks

[1683] 2. Project X progress report

[1684] 3. Announcement of second half plan

[1685] 4. Question and Answer Session

[1686] 5. Next task setting

[1687] 6. Closing Remarks

[1688] The user then uploads relevant materials (minutes of the previous meeting, progress reports, etc.) to the server, which analyzes them and organizes the necessary information. After entering the list of meeting participants and the range of possible dates, the server will suggest the optimal date and notify the user via the device.

[1689] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. Comments made during the meeting are also analyzed in real time, and messages are displayed to discourage long comments as necessary. The server also automatically extracts important comments and provides them to the user.

[1690] Furthermore, the emotion engine analyzes the remarks and facial expressions of meeting participants to recognize their emotions. For example, if a participant is feeling stressed, the server will use that information to adjust the pace of the meeting and prioritize the comments of other participants. After the meeting ends, individual feedback can be provided based on the emotional data.

[1691] This efficiently automates the entire meeting process, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need. In addition, the introduction of an emotion engine enables flexible meeting progression that takes into account the emotional state of participants.

[1692] The processing flow will be explained below.

[1693] Meeting preparation

[1694] Agenda creation

[1695] Step 1:

[1696] The user inputs the purpose and goals of the meeting using the system's interface.

[1697] Step 2:

[1698] The server receives the input objectives and goals, analyzes the information, and automatically generates an agenda based on the analysis results.

[1699] Step 3:

[1700] The server sends the generated agenda to the terminal.

[1701] Step 4:

[1702] The terminal displays the received agenda to the user, who then checks the displayed agenda.

[1703] Preparation of meeting materials

[1704] Step 1:

[1705] The user uploads the relevant materials to be used in the meeting to the server via the terminal.

[1706] Step 2:

[1707] The server scans the uploaded materials and extracts and organizes the necessary information.

[1708] Step 3:

[1709] The server converts the organized material into a downloadable format.

[1710] Step 4:

[1711] The terminal provides the converted material to the user, who then downloads and checks the provided material.

[1712] Scheduling and communication

[1713] Step 1:

[1714] The user inputs the list of participants and the available dates. The user inputs the list using the system interface.

[1715] Step 2:

[1716] The server receives the entered data and checks the schedules of all participants.

[1717] Step 3:

[1718] The server performs the analysis and suggests the best meeting date.

[1719] Step 4:

[1720] The server sends the proposed itinerary to the terminal.

[1721] Step 5:

[1722] The terminal notifies the user of the proposed schedule, and the user confirms the schedule and makes a final decision.

[1723] Step 6:

[1724] The server will notify all participants of the confirmed schedule.

[1725] Conference moderator

[1726] Display projection and speech

[1727] Step 1:

[1728] The server projects the agenda created at the start of the meeting and the prepared meeting materials onto a display.

[1729] Step 2:

[1730] The server uses voice synthesis technology to announce the start of the conference.

[1731] Speech Management

[1732] Step 1:

[1733] The server analyzes each participant's comments in real time.

[1734] Step 2:

[1735] The server detects position talk and long speech.

[1736] Step 3:

[1737] If a long speech is detected, the server sends that information to the terminal.

[1738] Step 4:

[1739] The terminal displays a message to the user at an appropriate timing to prevent long-term speech.

[1740] Organizing the discussion

[1741] Step 1:

[1742] The server automatically organizes the comments in real time.

[1743] Step 2:

[1744] The server extracts the key points.

[1745] Step 3:

[1746] The server transmits the extracted points to the terminal.

[1747] Step 4:

[1748] The terminal displays the extracted points to the user.

[1749] Additional features of the Emotion Engine

[1750] Step 1:

[1751] The server captures audio and video data in real time during the conference.

[1752] Step 2:

[1753] An emotion engine built into the server analyzes the captured data and recognizes the user's emotions.

[1754] Step 3:

[1755] The server acquires the emotion data recognized by the emotion engine and reflects it in the progress of the conference as appropriate.

[1756] Step 4:

[1757] If a user displays strong emotions, the server adjusts the conference progress and allows priority permission to speak as necessary.

[1758] Closing the meeting

[1759] Summary and reporting

[1760] Step 1:

[1761] The server summarizes the discussion at the end of the meeting.

[1762] Step 2:

[1763] The server sets the allocation of roles and deadlines for the next task.

[1764] Step 3:

[1765] The server sends the summarized content and task list to the terminal.

[1766] Step 4:

[1767] The terminal provides the summary and task list to the user, who then checks the information provided.

[1768] Step 5:

[1769] The server notifies all participants of the same information.

[1770] Step 6:

[1771] Based on the emotional data recognized by the emotion engine, the server provides individual feedback after the meeting ends.

[1772] Example 2

[1773] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1774] In conventional conference systems, the process of preparing, conducting, and closing a meeting is done manually, which takes a lot of time and effort. It is also difficult to analyze participants' comments and behavior in real time and provide appropriate feedback. Another problem is that it is difficult to conduct a meeting flexibly while taking into account participants' emotions.

[1775] The specific processing by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for inputting the purpose and goal of the meeting, means for automatically generating an agenda based on the input purpose and goal, means for displaying the agenda, means for uploading materials, means for analyzing the uploaded materials and organizing the necessary information, means for providing the organized materials, means for inputting a participant list and a range of available dates, means for checking the schedules of participants and proposing an optimal date, means for notifying and confirming the proposed date, means for analyzing comments in real time, means for automatically organizing the content of comments and extracting important points, means for providing the extracted points, means for analyzing emotions and adjusting the progress of the meeting, and means for providing individual feedback based on emotion data. This efficiently automates the processes of preparing, conducting, and closing the meeting, enabling flexible meeting progress that takes into account the comments and emotions of participants.

[1776] The "means for inputting the purpose and goal of the meeting" is an interface device that allows the user to input the specific purpose of the meeting and the goal that the user wants to achieve into the system.

[1777] The "means for automatically generating an agenda" is a software module that automatically generates a meeting schedule and agenda based on the input purpose and goals of the meeting.

[1778] A "means for displaying an agenda" is a display device or digital screen that visually displays the generated agenda to a user.

[1779] A "means for uploading materials" is an interface device that allows a user to upload materials or documents to be used in a conference to the system.

[1780] The "means for analyzing uploaded materials and organizing necessary information" is a software module that scans uploaded materials and extracts and classifies necessary information.

[1781] A "means for providing organized material" is an interface or file serving system that provides the organized material in a format that is usable by the user.

[1782] The "means for inputting a participant list and a range of possible dates" is an interface device for inputting participants of a meeting and possible dates for the meeting.

[1783] The "means of checking participants' schedules and proposing the optimal date" is an algorithm that calculates and proposes the optimal meeting date based on the participants' schedules entered.

[1784] A "means for notifying and confirming proposed dates" is a notification system or confirmation interface for notifying participants of proposed meeting dates and obtaining their confirmation.

[1785] "Means for analyzing speech in real time" refers to speech recognition technology that analyzes speech made by participants in a meeting in real time and understands the content.

[1786] The "means for automatically organizing speech content and extracting important points" is a software module that automatically organizes analyzed speech content and extracts important points.

[1787] The "means for providing extracted points" is an interface device for providing the extracted important points to the user.

[1788] The "means for analyzing emotions and adjusting the progress of the meeting" is an algorithm that analyzes participants' emotions from audio and video data and adjusts the progress of the meeting based on their emotional state.

[1789] The "means for providing individual feedback based on emotion data" is an interface device for providing feedback to individual participants based on emotion analysis data.

[1790] This invention combines a system that automates the process of preparing for, conducting, and closing a meeting with an emotion engine that recognizes the user's emotions. This system is implemented mainly through the roles of the server, the terminal, and the user.

[1791] Meeting preparation

[1792] Agenda creation

[1793] The user inputs the purpose and goals of the meeting on their device. This information is sent to the server. The server analyzes the input data and automatically generates an agenda using a generative AI model. The generated agenda is sent from the server to the device, which displays it to the user.

[1794] Preparation of meeting materials

[1795] Users upload relevant materials for meetings from their devices to the server. The server scans the uploaded materials and uses OCR (optical character recognition) technology to extract and organize the necessary information. The server then converts the organized materials into downloadable formats such as PDFs or spreadsheets and provides them to users via their devices.

[1796] Scheduling and communication

[1797] The user inputs the participant list and available dates from their device and sends them to the server. The server checks each participant's schedule and uses an algorithm to propose the optimal meeting date. The proposed optimal date is sent from the server to the device, which notifies the user. The user then confirms and confirms the final date. The server notifies all participants of the confirmed date.

[1798] Conference moderator

[1799] Display projection and speech

[1800] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[1801] Speech Management

[1802] During a meeting, the server uses voice analysis technology to analyze each participant's speech in real time. If the server detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[1803] Additional features of the Emotion Engine

[1804] Furthermore, the server operates an emotion engine that analyzes users' comments and facial expressions to recognize their emotions. The emotion engine analyzes the audio and video data acquired by the server in real time to identify the user's emotions (e.g., joy, anger, sadness, surprise, etc.). The recognized emotions are used to adjust the progress of the meeting. For example, if a user is feeling stressed, the server will adjust the progress of the speech to be more gradual. Furthermore, if a specific emotion is detected, the server will give priority to allowing the corresponding participant to speak.

[1805] Closing the meeting

[1806] Summary and reporting

[1807] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the device, which provides them to the user. The server then sends the summary and task list to all participants via email or notification. It is also possible to provide individual feedback after the meeting based on the emotional data recognized by the emotion engine.

[1808] Specific examples

[1809] For example, when hosting a project progress meeting, the user inputs the purpose "Project X progress report and second half plan" and the goal "Confirm this month's progress and set next tasks" from the terminal. Based on this, the server automatically generates the following agenda:

[1810] Opening remarks

[1811] Project X progress report

[1812] Announcement of second half plan

[1813] Q&A

[1814] Next task settings

[1815] Closing remarks

[1816] Next, the user uploads relevant materials (minutes of the previous meeting, progress reports, etc.) from their device to the server, which then analyzes them and organizes the necessary information. When the user enters the participant list and available dates, the server proposes the optimal dates and notifies them via the device.

[1817] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. During the meeting, the server analyzes the content of comments and emotional data, adjusting the progress as necessary. After the meeting ends, a summary of key points and a task list are provided, and individual feedback based on the emotional data is distributed.

[1818] This system efficiently automates the entire process of a meeting, significantly reducing the burden on the organizer and enabling flexible meeting proceedings that take into account the emotions of participants.

[1819] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1820] Step 1:

[1821] The user enters the purpose and goals of the meeting.

[1822] Input: The user enters the "purpose of the meeting" and "goal of the meeting" into the input form on the terminal.

[1823] Data processing: The data entered by the user is sent from the terminal to the server.

[1824] Output: The server uses the received data for analysis.

[1825] Specific operation: The user inputs "Marketing strategy for new product" as the "Objective of the meeting" on the terminal screen, and "Determine the strategy content" as the "Goal of the meeting."

[1826] Step 2:

[1827] The server receives the objectives and goals and generates the agenda.

[1828] Input: The server receives the "meeting purpose" and "meeting goal" entered by the user.

[1829] Data computation: The server analyzes the input data and automatically generates an agenda using a generative AI model.

[1830] Output: The generated agenda is saved on the server.

[1831] Specific operation: Based on the received data, the server generates an agenda including the flow and content of the meeting.

[1832] Step 3:

[1833] The server sends the agenda to the terminal, which displays it to the user.

[1834] Input: The server has the generated agenda data.

[1835] Data processing: The server sends the agenda data to the terminal.

[1836] Output: The submitted agenda is displayed on the terminal.

[1837] Specific operation: The generated agenda is displayed on the terminal screen to the user. The user confirms it.

[1838] Step 4:

[1839] The user uploads the meeting materials to the server.

[1840] Input: The user selects and uploads relevant materials (PDFs, documents, etc.) to be used in the meeting from their terminal.

[1841] Data processing: The terminal sends the selected materials to the server.

[1842] Output: The server receives the uploaded material.

[1843] Specific operation: The user selects "minutes of the previous meeting" or "progress report" from their device and presses the upload button.

[1844] Step 5:

[1845] The server scans the documents and extracts and organizes the necessary information.

[1846] Input: The server has the uploaded data.

[1847] Data calculation: The server uses OCR technology to scan the text in the document, extract and organize the necessary information.

[1848] Output: The extracted and organized information is saved on the server.

[1849] What it does: The server parses the uploaded PDF and extracts important text data.

[1850] Step 6:

[1851] The server transmits the organized materials to the terminal, which then provides them to the user.

[1852] Input: The server has organized data.

[1853] Data processing: The server converts the organized materials into PDF or spreadsheet format and sends them to the terminal.

[1854] Output: The transmitted material is displayed on the terminal and provided to the user.

[1855] Specific operation: The user can view and download the organized materials displayed on the terminal.

[1856] Step 7:

[1857] The user enters the participant list and date range.

[1858] Input: The user inputs the participant list and possible date range of the meeting from the terminal.

[1859] Data processing: The terminal sends the input information to the server.

[1860] Output: The server uses the received data for analysis.

[1861] Specific operation: The user uploads the participant list in an Excel file and selects and enters the available date range.

[1862] Step 8:

[1863] The server will suggest the best schedule and notify the user via the terminal.

[1864] Input: The server has the attendee list and date range data.

[1865] Data calculation: The server checks the schedules of each participant and uses an algorithm to calculate the optimal meeting date.

[1866] Output: The proposed optimal schedule is stored on the server and sent to the terminal.

[1867] Specific operation: A notification suggesting the best date will be displayed on the user's device.

[1868] Step 9:

[1869] The user checks and confirms the final schedule.

[1870] Input: The user confirms the proposed itinerary on the terminal.

[1871] Data processing: When the user presses the confirm button, the information is sent to the server.

[1872] Output: The confirmed schedule is saved on the server.

[1873] Specific operation: The user checks the proposed schedule and clicks the confirm button.

[1874] Step 10:

[1875] The server will notify all participants of the confirmed schedule.

[1876] Input: The server has the confirmed schedule data.

[1877] Data processing: The server sends the confirmed schedule to all participants via email or notification.

[1878] Output: All participants are notified of the meeting date.

[1879] Specific behavior: A schedule notification will be sent to all participants via email.

[1880] Step 11:

[1881] At the start of the meeting, the server projects the agenda and materials and announces the start of the meeting with a voice message.

[1882] Input: The server has the generated agenda and organized document data.

[1883] Data processing: The server projects the agenda and materials onto a projector and announces the start of the meeting using voice synthesis technology.

[1884] Output: Projected agenda and materials, audio start announcement.

[1885] Specific operation: The meeting agenda and materials are displayed on the projector, and an automated voice announces the start of the meeting.

[1886] Step 12:

[1887] The server analyzes the comments in real time and sends messages to the device as needed.

[1888] Input: Audio data during the conference is sent to the server.

[1889] Data calculation: The server performs real-time voice analysis and determines the content of the speech.

[1890] Output: If position talk or long speech is detected, the information is sent to the device.

[1891] Specific behavior: A warning message will be displayed on the user's device to discourage long-term comments.

[1892] Step 13:

[1893] The server organizes and extracts the content of comments and provides it to the user.

[1894] Input: Speech data analyzed during the meeting is saved on the server.

[1895] Data calculation: The server runs an algorithm to extract important points.

[1896] Output: The extracted key points are stored on the server and sent to the device.

[1897] Specific operation: A summary of important comments is displayed on the user's device.

[1898] Step 14:

[1899] The server uses an emotion engine to analyze and recognize participants' emotions.

[1900] Input: Audio and video data during the conference are sent to the server.

[1901] Data calculation: The emotion engine analyzes data in real time and recognizes emotions.

[1902] Output: The recognized emotion data is stored on the server.

[1903] Specific operation: The server monitors the participants' emotional states based on the emotional data analyzed.

[1904] Step 15:

[1905] The server adjusts the progress of the conference based on the emotion data.

[1906] Input: Emotion data stored on the server.

[1907] Data processing: The server runs an algorithm to adjust the progress of the meeting based on the emotion data.

[1908] Output: Adjusted meeting progress data.

[1909] Specific actions: For example, adjustments may be made such as slowing down the pace for participants who are feeling stressed, or prioritizing the comments of participants who have detected certain emotions.

[1910] Step 16:

[1911] The server organizes the key points and sets tasks and roles.

[1912] Input: Speech and emotion data captured during the meeting.

[1913] Data calculation: The server summarizes the discussion and sets the next task and allocation of roles.

[1914] Output: A concise summary and a task list.

[1915] Specific operation: The server generates summarized discussion points and task lists and compiles them as materials for the next meeting.

[1916] Step 17:

[1917] The server sends the summary and task list to all participants.

[1918] Input: An organized summary and a task list.

[1919] Data processing: The server sends the summary and task list to all participants.

[1920] Output: Summary and task list sent to all participants.

[1921] What happens: A summary and task list will be sent to all participants via email and notifications.

[1922] Step 18:

[1923] After the meeting, the server provides individual feedback based on the emotional data.

[1924] Input: Emotion data collected during the meeting.

[1925] Data calculation: The server creates individual feedback based on the analysis results.

[1926] Output: Individual feedback report.

[1927] Specific operation: The server generates individual feedback for each participant based on the emotional data and provides it via email or notification.

[1928] (Application example 2)

[1929] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1930] In modern factory environments, there is a demand for improved work efficiency and strengthened quality control. However, there is also the issue that emotional factors such as worker stress and fatigue have a significant impact on work efficiency. This has led to issues that traditional work management methods are no longer able to address. Furthermore, when errors occur during work, a system is needed to respond immediately, but traditional systems have issues with responsiveness. Furthermore, creating reports after task completion takes time, so efficient improvement measures are needed.

[1931] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes: a means for inputting the purpose and goal of the meeting; a means for automatically generating an agenda based on the input purpose and goal; a means for displaying the agenda; a means for uploading related materials; a means for analyzing the uploaded materials and organizing the necessary information; a means for providing the organized materials; a means for inputting a participant list and a range of possible dates; a means for checking the schedules of participants and proposing an optimal date; a means for notifying and confirming the proposed date; a means for acquiring task data and analyzing it using machine learning technology; a means for acquiring voice and video data of workers in real time; a means for analyzing the acquired data using an emotion analysis engine; a means for detecting errors and notifying workers in real time; and a means for automatically generating a report after task completion. This enables improved work efficiency and enhanced quality control in factories. Furthermore, analyzing and responding to workers' emotional states in real time can also improve the work environment. In addition, immediate response to errors and automatic report generation enable rapid implementation of corrective measures.

[1932] "Meeting objectives and goals" refers to the specific goals that are to be achieved by holding the meeting and the results that are to be achieved at the end of the meeting.

[1933] An "agenda" refers to a list of topics or items set in advance to plan the progress of a meeting or work.

[1934] "Related materials" refers to documents and reference information required for the progress of meetings and work.

[1935] "Upload" refers to the act of sending data from a device to a remote system such as a server or cloud.

[1936] "Machine learning technology" refers to computer programs and algorithms that automatically learn patterns from data and make subsequent predictions or decisions.

[1937] "Emotion analysis engine" refers to software and algorithms that analyze audio and video data and identify human emotional states.

[1938] "Real-time" refers to data or events being processed and analyzed almost immediately after they occur.

[1939] "Error detection" refers to the act of identifying anomalies or errors that occur during a task or system process.

[1940] "Report" refers to a document or digital compilation of the content, progress, and results of a meeting or task.

[1941] MODE FOR CARRYING OUT THE INVENTION

[1942] This invention is a system for improving the work efficiency and quality control of factory robots, which is implemented through the roles of a server, a terminal, and a user. This system is based on a conference automation system and focuses on specific applications in a factory environment.

[1943] Task setting and advance preparation

[1944] The server sends work task data to the factory robots. The task data is analyzed on the server using machine learning technology (such as TensorFlow or PyTorch) to generate specific work procedures. The generated procedures are then sent as instructions to the factory robots.

[1945] Real-time progress management

[1946] The device is equipped with a camera (such as an Intel RealSense or Raspberry Pi camera module) and microphone to capture the worker's voice and video data. The captured data is then sent to a server where it is analyzed in real time by an emotion analysis engine (using OpenCV and NLP technology). This allows the worker's emotional state to be identified and work procedures to be adjusted as necessary.

[1947] Error Detection and Notification

[1948] The server scans the acquired work data in real time and detects errors using machine learning models such as TensorFlow. If an error is detected, the worker is immediately notified via their device. The notification also includes appropriate countermeasures.

[1949] Closing and reporting on task completion

[1950] Once a task is completed, the server automatically organizes the work that has been performed and generates a report. The report, which clarifies the next task setting and areas for improvement, is provided to the worker via their terminal. The generated report helps improve overall work efficiency and quality.

[1951] Specific examples

[1952] For example, on a precision assembly line, it operates as follows:

[1953] 1. Task: Attach part A to B of a specific product.

[1954] 2. The server acquires the task data, analyzes it, and generates a work procedure.

[1955] 3. The voice and video data of workers captured through cameras and microphones is analyzed using an emotion analysis engine, and if a state of stress is detected, work procedures are adjusted.

[1956] 4. If an error occurs during work, the server will immediately notify you and provide you with the appropriate solution.

[1957] 5. Once the task is completed, the server automatically generates a report and provides the worker with information on the next task settings and improvements.

[1958] Prompt Sentence Examples

[1959] "A factory work supervision application based on a conference automation system. It includes functions such as specific task setting, optimizing work efficiency through real-time sentiment analysis, error detection, and work report generation."

[1960] This system dramatically improves work efficiency and quality within the factory and reduces the burden on workers. Real-time error detection and immediate response, an improved work environment through emotion analysis, and automatic report generation enable more flexible and efficient work management.

[1961] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1962] Step 1:

[1963] The server receives data from the user to input the purpose and goals of the meeting. The input purpose and goal data is processed within the server as initial data for automatic agenda generation. Here, the server uses natural language processing (NLP) technology to analyze the input data and generate agenda items. The generated agenda is sent to the terminal.

[1964] Step 2:

[1965] The device displays the agenda sent from the server. At this time, the user uploads relevant materials to the server via the device. The server analyzes the uploaded materials and organizes the necessary information. This analysis process uses OCR technology (optical character recognition) and machine learning algorithms to automatically extract useful information from the documents. The organized materials are then sent back to the device and provided to the user.

[1966] Step 3:

[1967] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and uses a calendar API or scheduling algorithm to suggest the best meeting date. The server sends the suggested best date to the device, which notifies the user. Once the user confirms and confirms the final date, the server notifies all participants of the confirmed date.

[1968] Step 4:

[1969] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and simultaneously announces the start of the meeting using speech synthesis technology. The input data are the agenda items and meeting materials, and the output is a visual and audio announcement of the start of the meeting.

[1970] Step 5:

[1971] During a meeting, the device captures the voice and video data of the participants in real time through its equipped camera and microphone. The captured data is sent to a server and analyzed by an emotion analysis engine. This includes, for example, facial expression analysis using OpenCV and voice analysis using NLP technology. If the analysis reveals that the user is feeling stressed, this information is recorded on the server and the progress of the meeting is adjusted.

[1972] Step 6:

[1973] The server analyzes speech in real time during meetings to detect long speeches and position-taking. It uses NLP technology to analyze speech content and detects speeches of a specific length or that focus on a specific topic. The detected information is sent to the device and displayed as a message.

[1974] Step 7:

[1975] At the end of the meeting, the server summarizes the discussion and sets the next task assignment and deadline. The summary and task list are sent from the server to the device, which then provides them to the user. Furthermore, the server generates individual feedback based on the emotional data recognized by the emotion analysis engine and sends it to the device. This provides a detailed meeting report to all participants.

[1976] Prompt Sentence Examples

[1977] "A factory work supervision application based on a conference automation system. It includes functions such as specific task setting, optimizing work efficiency through real-time sentiment analysis, error detection, and work report generation."

[1978] This dramatically improves work efficiency and quality within the factory and reduces the burden on workers. Real-time error detection and immediate response, an improved work environment through emotion analysis, and automatic report generation enable more flexible and efficient work management.

[1979] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1980] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1981] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1982] [Fourth embodiment]

[1983] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1984] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1985] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1986] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1987] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1988] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1989] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1990] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1991] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1992] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1993] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1994] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1995] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1996] The present invention relates to a system for automating a series of processes from advance preparation to progress and closing of a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[1997] Meeting preparation

[1998] Agenda creation

[1999] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[2000] Preparation of meeting materials

[2001] Users upload relevant materials for use in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format. The terminal then provides the converted materials to the user.

[2002] Scheduling and communication

[2003] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[2004] Conference moderator

[2005] Display projection and speech

[2006] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[2007] Speech Management

[2008] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[2009] Closing the meeting

[2010] Summary and reporting

[2011] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, and the terminal provides them to the user. Furthermore, the server sends the summary and task list to all participants using email or notifications.

[2012] Specific examples

[2013] For example, when hosting a project progress meeting, the user inputs the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task" into the system. Based on this, the server automatically generates the following agenda:

[2014] 1. Opening Remarks

[2015] 2. Project X progress report

[2016] 3. Announcement of second half plan

[2017] 4. Question and Answer Session

[2018] 5. Next task setting

[2019] 6. Closing Remarks

[2020] The user then uploads relevant materials (minutes of the previous meeting, progress reports, etc.) to the server, which analyzes them and organizes the necessary information. When the user inputs the list of meeting participants and a range of possible dates (e.g., a weekday morning next week), the server proposes the optimal date and notifies the user via the device.

[2021] On the day of the meeting, the server projects the agenda and organized materials onto a projector and announces the start of the meeting with a voice message. Speech during the meeting is also analyzed in real time, and messages are displayed to limit long speeches as necessary. Furthermore, the server automatically extracts important speech content and provides it to the user.

[2022] After the meeting, the server summarizes the discussion and sets the next task, its assignment, and deadline. The summary and task list are provided to the user via their terminal and reported to all participants.

[2023] This efficiently automates the entire meeting process, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[2024] The processing flow will be explained below.

[2025] Meeting preparation

[2026] Agenda creation

[2027] Step 1:

[2028] The user inputs the purpose and goals of the meeting using the system's interface.

[2029] Step 2:

[2030] The server receives the input objectives and goals, analyzes the information, and automatically generates an agenda based on the analysis results.

[2031] Step 3:

[2032] The server sends the generated agenda to the terminal.

[2033] Step 4:

[2034] The terminal displays the received agenda to the user, who then checks the displayed agenda.

[2035] Preparation of meeting materials

[2036] Step 1:

[2037] The user uploads the relevant materials to be used in the meeting to the server via the terminal.

[2038] Step 2:

[2039] The server scans the uploaded materials and extracts and organizes the necessary information.

[2040] Step 3:

[2041] The server converts the organized material into a downloadable format.

[2042] Step 4:

[2043] The terminal provides the converted material to the user, who then downloads and checks the provided material.

[2044] Scheduling and communication

[2045] Step 1:

[2046] The user inputs the list of participants and the available dates. The user inputs the list using the system interface.

[2047] Step 2:

[2048] The server receives the entered data and checks the schedules of all participants.

[2049] Step 3:

[2050] The server performs the analysis and suggests the best meeting date.

[2051] Step 4:

[2052] The server sends the proposed itinerary to the terminal.

[2053] Step 5:

[2054] The terminal notifies the user of the proposed schedule, and the user confirms the schedule and makes a final decision.

[2055] Step 6:

[2056] The server will notify all participants of the confirmed schedule.

[2057] Conference moderator

[2058] Display projection and speech

[2059] Step 1:

[2060] The server projects the agenda created at the start of the meeting and the prepared meeting materials onto a display.

[2061] Step 2:

[2062] The server uses voice synthesis technology to announce the start of the conference.

[2063] Speech Management

[2064] Step 1:

[2065] The server analyzes each participant's comments in real time.

[2066] Step 2:

[2067] The server detects position talk and long speech.

[2068] Step 3:

[2069] If a long speech is detected, the server sends that information to the terminal.

[2070] Step 4:

[2071] The terminal displays a message to the user at an appropriate timing to prevent long-term speech.

[2072] Organizing the discussion

[2073] Step 1:

[2074] The server automatically organizes the comments in real time.

[2075] Step 2:

[2076] The server extracts the key points.

[2077] Step 3:

[2078] The server transmits the extracted points to the terminal.

[2079] Step 4:

[2080] The terminal displays the extracted points to the user.

[2081] Closing the meeting

[2082] Summary and reporting

[2083] Step 1:

[2084] The server summarizes the discussion.

[2085] Step 2:

[2086] The server sets the allocation of roles and deadlines for the next task.

[2087] Step 3:

[2088] The server sends the summarized content and task list to the terminal.

[2089] Step 4:

[2090] The terminal provides the summary and task list to the user, who then checks the information provided.

[2091] Step 5:

[2092] The server notifies all participants of the same information.

[2093] Example 1

[2094] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2095] In traditional meeting management, the entire process of preparing, conducting, and closing a meeting is largely manual, requiring time and effort. It can also be difficult for all participants to quickly share necessary information, reducing meeting efficiency. Furthermore, tasks such as managing comments and organizing discussions during a meeting are time-consuming, making it difficult to capture important points.

[2096] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[2097] In this invention, the server includes means for inputting the purpose and goals of the meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information using natural language processing, means for providing the organized materials, means for inputting a list of participants and a range of possible dates, means for checking the schedules of participants and proposing the optimal date using a generative AI model, and means for notifying and confirming the proposed date. This efficiently automates a series of processes from advance preparation to progress and closing of the meeting, reducing the burden on the organizer and enabling all participants to quickly share necessary information.

[2098] "Means for inputting the purpose and goals of the meeting" refers to an interface that allows the user to input the purpose of the meeting and the specific goals to be achieved.

[2099] "Means for automatically generating an agenda based on input purpose and goals" refers to a function that analyzes the purpose and goals of a meeting input by the user and automatically creates a meeting schedule based on them.

[2100] "Means for displaying an agenda" refers to a device or interface for visually displaying the automatically generated agenda to a user.

[2101] "Means for uploading related materials" refers to an interface that allows a user to upload documents and materials to be used in a conference to the system.

[2102] "Means for analyzing uploaded materials and organizing necessary information using natural language processing" refers to the function of scanning uploaded materials and using natural language processing technology to extract and organize important information.

[2103] "Means for providing organized material" refers to an interface for providing organized information or transformed material to a user.

[2104] "Means for inputting participant list and possible date range" refers to an interface for inputting into the system a list of people who will be attending the conference and a possible date range for the conference.

[2105] "Means of checking participants' schedules and proposing optimal dates using a generative AI model" refers to a function that collects participants' schedule information and calculates optimal meeting dates using a generative AI model.

[2106] "Means for communicating and confirming proposed dates" refers to the interface or notification system for notifying participants of proposed meeting dates and obtaining confirmation.

[2107] "Means for announcing the start of a conference by voice using voice synthesis technology" refers to a function for announcing the start of a conference by voice using voice synthesis technology.

[2108] "Means of analyzing statements in real time and detecting position-taking and long speech" refers to a function that analyzes statements made during a meeting in real time and identifies specific speech patterns and long speech.

[2109] "Means for displaying a message to suppress long speech" refers to an interface for notifying participants when a long speech is detected.

[2110] "Means for automatically organizing statements and extracting important points" refers to a function that automatically classifies and organizes statements made during a meeting and extracts important points.

[2111] "Means for providing extracted points" refers to an interface for providing automatically extracted important points to a user.

[2112] The present invention relates to a system for automating a series of processes for preparing, conducting, and closing a conference, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments will be described below.

[2113] Meeting preparation

[2114] Agenda creation

[2115] Users use their devices to input the purpose and goals of a meeting into the system. This input information is sent to the server, which receives it and stores it in a database. The server then uses a generative AI model to automatically generate an agenda based on the input purpose and goals. The generated agenda is then sent from the server to the device, which displays it to the user.

[2116] Specifically, if a user enters "Project X progress report and second half plan" as the purpose and "Check this month's progress and set next task" as the goal, the server will automatically generate an agenda like the one below.

[2117] 1. Opening Remarks

[2118] 2. Project X progress report

[2119] 3. Announcement of second half plan

[2120] 4. Question and Answer Session

[2121] 5. Next task setting

[2122] 6. Closing Remarks

[2123] Preparation of meeting materials

[2124] Users upload relevant materials to the server via their devices. The server scans the uploaded materials and uses natural language processing technology to extract and organize the necessary information. The server then converts the organized materials into a downloadable format (e.g., PDF or Word file). The server then provides the organized and converted materials to the device, which then displays them to the user.

[2125] Specifically, when a user uploads "minutes of the previous meeting" or "progress report," the server analyzes it, extracts important data such as progress rate and issues, and generates organized documents.

[2126] Scheduling and communication

[2127] The user uses the device to input the participant list and possible date range. This data is also sent to the server, which checks the participants' schedule information. The server then uses the generative AI model to propose the optimal meeting date. The proposed date is sent from the server to the device, which notifies the user. The user reviews the proposed date and finalizes it. The server notifies all participants of the confirmed date.

[2128] Specifically, when a user inputs a weekday morning of the following week as a possible meeting date, the server checks the calendars of the participants and suggests the optimal date that everyone can attend (e.g., 10:00 a.m. next Wednesday).

[2129] Meeting proceedings

[2130] Display projection and audio guide

[2131] When a meeting starts, the server projects a pre-generated agenda and prepared materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[2132] Specifically, at the start of the meeting, an audio announcement will be made saying "The meeting is starting," while the agenda and materials will be displayed on the projector.

[2133] Speech Management

[2134] During the meeting, the server analyzes each participant's speech in real time to detect position-taking and long speech. If detected, the information is sent to the device, which then displays a message prompting the user to moderate their speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are also sent from the server to the device, which then displays them to the user.

[2135] Specifically, when a participant is detected speaking for a long time during a meeting, the server displays the message "Your speech is long" on the terminal.

[2136] Closing the meeting

[2137] Summary and reporting

[2138] At the end of the meeting, the server summarizes the discussion and sets the next task, role assignment, and deadline. The summary and task list are sent from the server to the terminal, which provides them to the user. The server then sends the summary and task list to all participants via email or notifications.

[2139] Specifically, after the meeting ends, the server automatically generates a "list of next tasks" and sends it to all participants by email.

[2140] Prompt Sentence Examples

[2141] Enter the purpose of the meeting: "Project X progress report and second half plan"

[2142] Upload related materials: "Minutes of the previous meeting, progress report"

[2143] Enter participant list and date range: "Next weekday morning"

[2144] This system efficiently automates the entire process of a meeting, reducing the burden on organizers and enabling participants to quickly share information.

[2145] The flow of the identification process in the first embodiment will be described with reference to FIG.

[2146] Step 1:

[2147] The user inputs the purpose and goals of the meeting.

[2148] Specifically, the user uses a terminal to input the purpose of "reporting the progress of Project X and planning for the second half of the year" and the goal of "checking this month's progress and setting the next task."

[2149] Input: Meeting purpose and goals

[2150] Output: The input data is sent to the server.

[2151] Step 2:

[2152] The server receives the input meeting purpose and goal and stores them in a database.

[2153] Then, a generative AI model is used to auto-generate the agenda.

[2154] Specifically, an agenda is generated that includes items such as "opening remarks" and "setting tasks for the next meeting."

[2155] Input: Meeting purpose and goals

[2156] Output: Auto-generated agenda

[2157] Step 3:

[2158] The server transmits the generated agenda to the terminal.

[2159] The terminal displays the received agenda to the user.

[2160] Specifically, each item on the agenda is listed on the screen.

[2161] Input: Generated agenda

[2162] Output: The agenda is displayed in the terminal.

[2163] Step 4:

[2164] Users upload relevant materials.

[2165] "Minutes of the previous meeting" and "progress reports" are uploaded to the server via the terminal.

[2166] Input: Related materials

[2167] Output: The data is sent to the server.

[2168] Step 5:

[2169] The server receives the uploaded materials and stores them in a database.

[2170] Next, natural language processing technology is used to extract and organize the necessary information.

[2171] Specifically, important data such as progress rate and issues are extracted.

[2172] Input: Uploaded materials

[2173] Output: Organized data

[2174] Step 6:

[2175] The server converts the organized data into a downloadable format.

[2176] The terminal provides this data to the user.

[2177] Specifically, materials can be downloaded in document format (PDF or Word files).

[2178] Input: Organized data

[2179] Output: Downloadable materials

[2180] Step 7:

[2181] The user inputs a list of participants and a range of possible dates.

[2182] Enter a date range such as "morning on a weekday next week" through the terminal.

[2183] Input: Participant list and date range

[2184] Output: The input data is sent to the server

[2185] Step 8:

[2186] The server checks each participant's schedule information and uses a generative AI model to suggest the optimal date.

[2187] The proposed itinerary is again transmitted to the terminal, which notifies the user of this.

[2188] Specifically, it will display the optimal date, such as "10:00 AM next Wednesday."

[2189] Input: Participant list and date range

[2190] Output: Suggested optimal schedule

[2191] Step 9:

[2192] The user reviews and finalizes the proposed itinerary.

[2193] The server will notify all participants of the confirmed schedule.

[2194] Specifically, participants will be notified of the schedule via email or message.

[2195] Input: Proposed optimal dates

[2196] Output: Notification of confirmed schedule

[2197] Step 10:

[2198] At the start of the meeting, the server projects the agenda and meeting materials onto a projector and announces the start of the meeting using voice synthesis technology.

[2199] Specifically, a voice announcement will be made saying "The meeting is about to begin," and the agenda and materials will be displayed on the projector.

[2200] Input: Trigger for starting the meeting (date and time)

[2201] Output: Meeting opening announcement and projection

[2202] Step 11:

[2203] During the meeting, the server analyzes each participant's comments in real time and detects position-taking and long comments.

[2204] If necessary, the device will display a suppression message such as "Your speech is getting long."

[2205] Input: Speech data during the meeting

[2206] Output: Suppression message

[2207] Step 12:

[2208] At the end of the meeting, the server summarizes the discussion and sets the next tasks, assignment of roles, and deadlines.

[2209] The organized summary and task list are sent from the server to the terminal and provided to the user.

[2210] Specifically, a list of upcoming tasks will be displayed on the device and sent to all participants by email.

[2211] Input: Meeting discussion data

[2212] Output: Summary and task list notifications

[2213] This system efficiently automates the entire process of a meeting, significantly reducing the burden on the organizer and enabling all participants to quickly share the information they need.

[2214] (Application example 1)

[2215] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2216] The process of preparing, conducting, and closing meetings has traditionally been done manually, requiring time and effort, and is prone to human error and delays. Especially on factory floors, there is a need to streamline these meeting processes and ensure they proceed quickly and accurately. Furthermore, there is a need for effective methods for individual tasks, such as coordinating participant schedules, organizing materials, and managing comments.

[2217] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[2218] In this invention, the server includes means for inputting the purpose and goals of a meeting, means for automatically generating an agenda based on the input purpose and goals, means for displaying the agenda, means for uploading related materials, means for analyzing the uploaded materials and organizing the necessary information, means for providing the organized materials, means for inputting a participant list and a range of possible dates, means for checking participant schedules and proposing the optimal date, means for notifying and confirming the proposed date, means for analyzing the time and content of speech, means for issuing a warning if the speech time is too long, means for summarizing the main points and automatically setting the next tasks and role allocation, and means for notifying the main points and task information. This automates the entire process of preparing, conducting, and closing a meeting, making it possible to perform it efficiently and accurately.

[2219] The "means for inputting the purpose and goal of the meeting" refers to an input device or interface that allows the user to input the specific purpose of the meeting and the goal that the user wants to achieve into the system.

[2220] A "means for automatically generating an agenda based on input objectives and goals" is an algorithm or software that analyzes the objectives and goals of a meeting and automatically generates an effective agenda.

[2221] A "means for displaying an agenda" is a display device or display interface that allows a user to view the generated agenda.

[2222] The "means for uploading relevant materials" refers to a device or interface for uploading materials required for a conference to the system.

[2223] "Means for analyzing uploaded materials and organizing necessary information" refers to software or algorithms that analyze uploaded materials and extract and organize important information.

[2224] A "means for providing organized material" is an interface or device for providing the analyzed and organized material to a user.

[2225] The "means for inputting a participant list and a possible date range" is an interface or device for inputting a participant list and a possible date range of a conference into the system.

[2226] The "means for checking the schedules of participants and proposing the optimal date" is software or an algorithm that checks the schedules of the participants that have been input and proposes the optimal meeting date.

[2227] A "means for notifying and confirming proposed dates" is an interface or device for notifying participants of proposed meeting dates and receiving confirmation.

[2228] The "means for analyzing the time and content of speech" is software or an algorithm that analyzes the time and content of speech during a meeting in real time.

[2229] The "means for issuing a warning when a speech duration is too long" is an interface or device for detecting an excessively long speech duration and issuing a warning.

[2230] "Means for summarizing key points and automatically setting the next tasks and division of roles" refers to software or an algorithm that summarizes the key points of a meeting and automatically sets the next tasks and division of roles.

[2231] The "means for notifying key points and task information" is an interface or device for notifying all participants of the organized key points and set task information.

[2232] The present invention provides a system for automating the preparation, progress, and closing of a meeting. A specific embodiment of this system will be described below.

[2233] Meeting preparation

[2234] Agenda creation

[2235] First, the user speaks to the robot to input the purpose and goal of the meeting. The robot converts the purpose and goal of the meeting into text data using speech recognition technology and sends it to the server. The server then uses a generative AI model (for example, OpenAI's GPT-3 model) to automatically generate an agenda based on the purpose and goal. This agenda is then displayed on the robot.

[2236] Preparation of meeting materials

[2237] Users upload meeting materials to the robot using a USB memory stick or similar device. The uploaded materials are then sent to a server, which scans and analyzes them, extracting and organizing the necessary information. This process uses document analysis software, and the robot then provides the organized materials to the user.

[2238] Scheduling and communication

[2239] The user inputs a list of participants and a range of possible dates into the robot. The input data is sent to the server, which checks the participants' schedules and automatically proposes the most suitable meeting date. The robot notifies the user of this proposal by voice or on-screen display. The user confirms and confirms the final date, and the server notifies all participants of the confirmed date.

[2240] Conference moderator

[2241] Display projection and speech

[2242] When a meeting starts, the robot retrieves the pre-generated agenda and prepared meeting materials from the server, projects them onto a projector or monitor, and uses voice synthesis technology to announce the start of the meeting.

[2243] Speech Management

[2244] During the meeting, the robot analyzes each participant's speech in real time. It uses speech recognition and natural language processing technology to analyze the content and time of speech and detect long speeches and position-taking. If this is detected, the robot issues a warning and adjusts the speech as necessary. The server also automatically organizes the speech and extracts key points. This information is also provided by the robot.

[2245] Closing the meeting

[2246] Summary and reporting

[2247] At the end of the meeting, the server summarizes the content of the discussion and automatically sets the next tasks, assignment of roles, and deadlines. This summary and task list are provided to the user via the robot, and all participants are notified using the email sending function.

[2248] Specific examples

[2249] For example, to hold a progress meeting for a new product project, the user inputs to the robot, "I would like to hold a progress meeting for the new product project. The purpose is to check the progress and set the next task." The robot sends this information to the server, which generates the following agenda:

[2250] Purpose of the meeting: Check the progress of the new product project

[2251] Goal: Set the next task

[2252] Generate an agenda.

[2253] The final agenda generated by the server looks like this:

[2254] 1. Opening Remarks

[2255] 2. Project progress reports

[2256] 3. Announcement of next plan

[2257] 4. Question and Answer Session

[2258] 5. Next task setting

[2259] 6. Closing Remarks

[2260] This agenda is projected by the robot and announced aloud. The robot also analyzes comments made during the meeting, extracting and displaying important points. After the meeting ends, the server summarizes the main points of the discussion, sets the next tasks and allocation of roles, and notifies all participants.

[2261] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[2262] Step 1:

[2263] The user speaks to the robot to input the purpose and goals of the meeting.

[2264] Input: Voice input of the purpose and goals of the meeting

[2265] Output: Meeting objectives and goals as text data

[2266] Data processing and calculation: The robot uses voice recognition technology (e.g., speech_recognition module) to convert voice data into text data.

[2267] Step 2:

[2268] The robot sends the converted text data to a server, which then uses a generative AI model (e.g., GPT-3) to automatically generate an agenda.

[2269] Input: Meeting objectives and goals as text data

[2270] Output: Agenda text data

[2271] Data processing and calculation: The server forms prompt sentences based on the purpose and goal, inputs them into the generative AI model, and generates an agenda.

[2272] Step 3:

[2273] The server sends the generated agenda to the robot, which displays the agenda.

[2274] Input: Agenda text data

[2275] Output: Agenda displayed on the display

[2276] Data processing and calculation: The robot displays the acquired agenda on its built-in display or an external monitor.

[2277] Step 4:

[2278] The user uploads meeting materials to the robot using a USB memory stick, etc. The robot then sends the materials to the server.

[2279] Input: Materials uploaded via USB memory or LAN connection

[2280] Output: Data transferred to the server

[2281] Data processing and calculation: The robot securely transmits the uploaded data to the server.

[2282] Step 5:

[2283] The server scans and analyzes the uploaded materials and organizes the necessary information.

[2284] Input: Uploaded material data

[2285] Output: Organized data

[2286] Data processing and calculation: The server uses document analysis software to scan the documents, extract text, and organize the information.

[2287] Step 6:

[2288] The participant list and possible date range are input to the robot, which then sends this to the server, which then suggests the best meeting date.

[2289] Input: List of participants and possible date range

[2290] Output: Proposed meeting dates

[2291] Data processing and calculation: The server checks the participants' schedules and uses an algorithm to calculate the optimal schedule.

[2292] Step 7:

[2293] The robot will notify the user of the schedule proposed by the server by voice or on-screen display, and the user will confirm it.

[2294] Input: Proposed meeting dates

[2295] Output: Confirmed or revised meeting dates

[2296] Data processing and calculation: The robot presents the proposed schedule and accepts confirmation from the user.

[2297] Step 8:

[2298] At the start of a meeting, the robot retrieves the agenda and prepared meeting materials, projects them onto a projector or monitor, and announces the start of the meeting using voice synthesis technology.

[2299] Input: Meeting agenda and materials data

[2300] Output: Agenda and materials projected on a projector or monitor, audio announcements to start

[2301] Data processing and calculation: The robot visually displays the agenda and materials and announces the start of the meeting by voice.

[2302] Step 9:

[2303] During meetings, the robot analyzes comments in real time and detects long speeches and position-taking.

[2304] Input: Participant's voice

[2305] Output: Parsed speech and warning messages

[2306] Data processing and calculation: The robot uses voice recognition and natural language processing technology to analyze the time and content of speech and display warnings as necessary.

[2307] Step 10:

[2308] At the end of the meeting, the server summarizes the content of the discussion, sets the next tasks, assignment of roles, and deadlines, and notifies all participants of the summary and task list.

[2309] Input: What was said during the meeting and notes

[2310] Output: Summary of discussion and task list

[2311] Data processing and calculation: The server uses natural language processing technology to summarize the discussion content, automatically set the next tasks and role allocation, and notify participants.

[2312] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[2313] The present invention combines a system that automates a series of processes from advance preparation to progress and closing of a meeting with an emotion engine that recognizes the user's emotions, and is implemented mainly through the roles of a server, a terminal, and a user. Specific embodiments are described below.

[2314] Meeting preparation

[2315] Agenda creation

[2316] First, the user inputs the purpose and goals of the meeting. This input is received by the server. The server analyzes the received data and automatically generates an agenda based on the results. The generated agenda is sent from the server to the terminal, and the terminal displays the agenda to the user.

[2317] Preparation of meeting materials

[2318] Users upload relevant materials to be used in meetings to the server. The server scans the uploaded materials, extracts and organizes the necessary information, and then converts the organized materials into a downloadable format and provides them to users via their devices.

[2319] Scheduling and communication

[2320] The user inputs a list of participants and a range of possible dates. This input data is also sent to the server. The server checks each participant's schedule and proposes the optimal meeting date. The proposed optimal date is sent from the server to the terminal, which notifies the user. The user confirms and confirms the final date. The server notifies all participants of the confirmed date.

[2321] Conference moderator

[2322] Display projection and speech

[2323] When a meeting starts, the server projects a pre-generated agenda and prepared meeting materials onto a projector or monitor, and also uses speech synthesis technology to announce the start of the meeting.

[2324] Speech Management

[2325] During the meeting, the server analyzes each participant's speech in real time. If it detects position-taking or long speech, it sends the information to the device, which then displays a message to the user to curb the long speech at the appropriate time. The server also automatically organizes the speech and extracts key points. These extracted points are sent to the device, which then displays them to the user.

[2326] Additional features of the Emotion Engine

[2327] Furthermore, an emotion engine is built in that analyzes the user's speech and facial expressions to recognize their emotions. The emotion engine analyzes the audio and video data acquired b...

Claims

1. A means for inputting the purpose and goals of the meeting; A means for automatically generating an agenda based on input objectives and goals; a means for displaying the agenda; a means for uploading relevant materials; A means of analyzing uploaded materials and organizing necessary information, A means for providing organized materials; a means for inputting a list of participants and a range of possible dates; A way to check participants' schedules and suggest the best dates, A system that includes a means for notifying and confirming proposed dates.

2. A means to project the agenda and meeting materials at the start of the meeting and announce the start of the meeting by voice; A means of analyzing statements in real time to detect position-taking and long statements, measures to curb long speeches; A means to automatically organize the content of comments and extract important points, 10. The system of claim 1, further comprising: means for providing the extracted points.

3. A way to summarize the discussion and set the next task assignment and deadline. a means for providing summaries and task lists; 10. The system of claim 1, further comprising means for reporting the summarized content and task list to the participants.

Images