System

Abstract

A system is provided.SOLUTION: A system comprising: means for receiving meeting configuration information and generating a meeting schedule; means for performing facial recognition of meeting participants and updating attendance status; means for generating meeting progress scripts and preparing materials for use during a meeting; means for receiving participant questions during a meeting and generating answers to the questions; means for recording meeting content and categorizing issues discussed; and means for generating and reporting suggestions for issue resolution.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Traditional meeting management requires a lot of time and effort to prepare, conduct, and prepare materials, making it difficult to manage efficiently. Another problem is that participants' questions cannot be answered promptly and appropriately during the meeting, resulting in a decline in meeting quality. Furthermore, post-meeting review, issue organization, and analysis are all done manually, resulting in low work efficiency and a long time required to properly resolve issues. [Means for solving the problem]

[0005] The present invention solves the conventional problems with a system that includes a means for receiving meeting setting information and generating a meeting schedule, a means for performing facial recognition of meeting participants and updating their attendance status, a means for generating a meeting progress script and preparing materials to be used during the meeting, a means for accepting questions from participants during the meeting and generating answers to those questions, a means for recording the meeting content and categorizing the issues discussed, and a means for generating and reporting proposals for solving the issues. This enables the integrated management of meeting preparation, progress, and responses, efficient review, and proposals for solving the issues.

[0006] "Meeting setting information" refers to information such as the date and time of the meeting, the participant list, and the agenda.

[0007] A "meeting schedule" indicates a plan such as the start time, end time, and order of agenda items for a meeting.

[0008] "Facial recognition" refers to technology that uses a camera to identify the faces of meeting participants and identify them as individuals.

[0009] "Attendance status" indicates information on people who are attending the meeting and people who are absent.

[0010] A "progress script" is a set of instructions that shows how the announcer AI should proceed as the meeting progresses.

[0011] "Materials" refers to information such as documents, images, and videos used during a meeting.

[0012] A "question" refers to a statement made by a meeting participant requesting confirmation or opinion.

[0013] "Answer" refers to providing appropriate information or explanation in response to a question.

[0014] "Recording" refers to the digital preservation of discussions and statements made during a meeting.

[0015] "Issues" refer to problems or topics that are discussed during a meeting and need to be resolved.

[0016] "Categorization" refers to the process of classifying the issues discussed by field or content.

[0017] A "proposal" is a solution or action plan for a specific problem.

[0018] "Report" refers to documents and data used to communicate proposals and meeting results to meeting participants. [Brief explanation of the drawings]

[0019] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0020] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0021] First, the terms used in the following description will be explained.

[0022] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0023] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0024] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0025] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0026] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0027] [First embodiment]

[0028] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0029] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0030] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0031] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0032] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0033] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0034] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0035] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0036] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0037] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0038] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0039] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0040] This invention relates to a system that supports efficient progress of meetings and problem-solving. This system receives and manages meeting setting information, identifies participants using face recognition technology, prepares meeting scripts and materials, provides prompt answers to questions from participants, and organizes issues and proposes solutions after the meeting.

[0041] (System configuration)

[0042] The system includes the following major components:

[0043] 1. Server

[0044] The server controls the entire system and is responsible for managing and processing data.

[0045] Receives meeting setting information and generates and saves the meeting schedule.

[0046] Processes facial recognition data of meeting participants and updates their attendance status.

[0047] Generate progress scripts and materials and send them to the terminal.

[0048] It searches for relevant information and generates answers to questions from participants.

[0049] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[0050] 2. Terminal

[0051] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[0052] Meeting participants are identified using a camera and facial recognition function.

[0053] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[0054] Participants' questions are recognized by voice and sent to the server as text data.

[0055] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[0056] Discussions during meetings are recorded in real time and sent to a server.

[0057] 3. Users

[0058] The user enters the conference setting information and registers it in the system.

[0059] Ask questions and give opinions during the meeting and use the system interface.

[0060] (Specific operation example)

[0061] As an example, consider holding a kickoff meeting for a new product.

[0062] 1. A user schedules a meeting

[0063] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[0064] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[0065] 2. Pre-meeting preparation

[0066] The server creates a meeting script and prepares and stores materials related to the agenda (e.g., market research data, new product specifications, etc.) in cloud storage.

[0067] The terminal downloads this data from the server and prepares to start the meeting.

[0068] 3. Participant facial recognition

[0069] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[0070] The server updates attendance status based on facial recognition data and also records information on absentees.

[0071] 4. Meeting Management

[0072] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[0073] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[0074] 5. Questions and Answers

[0075] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[0076] The device recognizes the question and sends it to the server as text data.

[0077] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[0078] The device displays the answer in document format, and an AI announcer explains it aloud.

[0079] 6. Recording and reviewing the meeting

[0080] The device records the content of discussions during the meeting in real time and sends it to the server.

[0081] After the meeting, the server organizes the recordings and categorizes each topic.

[0082] The server performs the analysis, proposes solutions to each problem, and reports them to the user via the terminal.

[0083] Using this system, you can consistently and efficiently prepare for meetings, conduct them, answer questions, and review and resolve issues after the meeting.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[0087] Step 2:

[0088] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[0089] Step 3:

[0090] At least a few hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[0091] Step 4:

[0092] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[0093] Step 5:

[0094] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data is then sent to the server.

[0095] Step 6:

[0096] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[0097] Step 7:

[0098] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[0099] Step 8:

[0100] The terminal displays materials based on the progress script, and an AI announcer provides explanations.

[0101] Step 9:

[0102] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[0103] Step 10:

[0104] The terminal sends the text data of the question to the server.

[0105] Step 11:

[0106] The server searches the database for relevant information and generates answers to the questions, which are then sent to the device.

[0107] Step 12:

[0108] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[0109] Step 13:

[0110] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[0111] Step 14:

[0112] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[0113] Step 15:

[0114] The server performs a detailed analysis of each problem and generates a solution proposal, including an action plan and predicted outcomes.

[0115] Step 16:

[0116] The terminal reports the proposed solution to the user and provides it in an exportable format.

[0117] Step 17:

[0118] The user receives the report and develops an action plan as needed.

[0119] Example 1

[0120] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0121] Conventional meeting management has the problem of being difficult to manage consistently from setting up the meeting to proceeding, answering questions, and reviewing and resolving issues after the meeting. This has led to problems such as the inability to efficiently organize the information necessary for the meeting, confirm attendance, record the meeting content in real time, and process the post-meeting information.

[0122] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0123] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for recognizing characteristics of conference participants and updating attendance status, means for generating a conference progress script and preparing information to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and classifying the issues discussed, and means for generating and reporting proposals for solving the issues. This makes it possible to efficiently and consistently manage the entire conference process, and quickly organize information, confirm attendance, record in real time, and organize issues and propose solutions after the conference.

[0124] "Meeting setting information" refers to basic information required to hold a meeting, such as the date and time of the meeting, location, participants, and agenda.

[0125] "Means for generating a meeting schedule" refers to a function that automatically creates and saves specific meeting schedules based on received meeting setting information.

[0126] "Feature recognition of meeting participants" refers to technology that uses facial recognition technology to identify meeting participants and confirm their attendance status.

[0127] "Means for updating attendance status" refers to a function that updates the attendee list in real time based on the results of feature recognition of meeting participants and manages attendance and absence status.

[0128] "Means for generating a meeting progress script" refers to a function that automatically creates a series of steps that compile the instructions and information necessary to progress the meeting.

[0129] "Means for preparing information to be used during a meeting" refers to the function of collecting and organizing necessary materials and data based on the meeting progress script and making them available during the meeting.

[0130] "Means for accepting questions and generating answers to those questions" refers to a function for receiving questions submitted by participants during a meeting and generating answers to address those questions immediately.

[0131] "A means of recording meeting content and classifying the issues discussed" refers to a function that records what is said and discussed during a meeting in real time, and then organizes and classifies the content by issue.

[0132] "Means for generating and reporting proposals for resolving issues" refers to the function of analyzing the recorded meeting content, proposing specific solutions to identified issues, and reporting them to the relevant parties.

[0133] The system of the present invention is designed to support efficient progress of meetings and problem-solving. This system is composed of a server, terminals, and users.

[0134] server

[0135] The server controls the entire system and is responsible for managing and processing data. Specifically, the server receives meeting setting information from users, generates a meeting schedule, stores it in a database, and then automatically sends invitation emails to each participant.

[0136] The server uses facial recognition technology to identify meeting participants and update their attendance status.The server uses a built-in database system (e.g., MySQL) and cloud storage (e.g., Amazon S3) to generate and store the meeting script and related materials.

[0137] During the meeting, the system accepts questions from participants, searches for relevant information in its database, and generates answers. The answers are sent to the terminal in real time. It also records the contents of the meeting, categorizes the issues after the meeting, and generates and reports proposed solutions.

[0138] Terminal

[0139] The device functions as a user interface and receives instructions from the server to proceed with the conference. The device is equipped with a camera and facial recognition function, which identifies participants at the start of the conference and sends that data to the server.

[0140] The AI ​​announcer on the device will conduct the meeting based on the script and materials received from the server, outputting instructions such as "First, I'll give an overview of the new product."

[0141] During the meeting, the device recognizes participants' voice questions and sends them as text data to the server. Answers and materials received from the server are displayed on the device's screen, and an AI announcer provides audible explanations.

[0142] The device records the meeting contents in real time and sends the data to a server, allowing participants to review the meeting contents later and use them as reference material for problem-solving.

[0143] User

[0144] The user enters meeting setting information and registers it in the system. For example, the user enters information such as "The kickoff meeting for the new product is scheduled for May 15th at 2:00 PM."

[0145] During the meeting, users can ask questions and give opinions and use the system's interface to help the meeting proceed.

[0146] Specific examples

[0147] As an example, consider the case of holding a kickoff meeting for a new product. The user sets up the meeting, and the server generates a schedule and notifies participants. At the start of the meeting, the device performs facial recognition and reports attendance status to the server. The server sends a progress script to the device, and an announcer AI leads the meeting. Questions from participants are sent to the server via the device, and answers are instantly generated and displayed. After the meeting ends, the server organizes the recorded content, categorizes issues, and proposes solutions.

[0148] Example prompts for generative AI models

[0149] "Please explain in detail the process for generating the meeting script and related materials."

[0150] "Please explain the specific flow of questions and responses during a meeting."

[0151] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[0152] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0153] Step 1:

[0154] Enter and register meeting setting information

[0155] The user inputs the meeting setting information (e.g., the name of the meeting, the date and time, the location, the participant list, and the details of the agenda) and registers it in the system. The input data is sent to the server.

[0156] Input: Meeting settings information (meeting name, date, time, location, participant list, agenda details)

[0157] Data processing / calculation: The server receives the input information, generates a meeting schedule and participant list, stores them in a database, and generates and sends invitation emails to each participant.

[0158] Output: Meeting schedule, participant list, invitation email

[0159] Step 2:

[0160] Preparing the script and materials

[0161] The server generates a progress script and related materials (e.g., market research data, specifications for new products, etc.) required for the conference.

[0162] Input: Meeting setting information, past meeting data, reference materials

[0163] Data processing / calculation: Based on this information, the server automatically generates a progress script and uploads related materials to cloud storage.

[0164] Output: Progress script, related materials (uploaded to cloud storage)

[0165] Step 3:

[0166] Meeting start and participant recognition

[0167] At the start of a conference, the device uses a camera to identify participants and sends that data to the server.

[0168] Input: Participant's face data (obtained from camera)

[0169] Data processing / calculation: The device uses facial recognition technology to identify participants and sends data to the server to update their attendance status.

[0170] Output: Updated attendance data (sent to server)

[0171] Step 4:

[0172] Meeting proceedings

[0173] The server sends the generated script to the device, and the device's announcer AI runs the conference based on the script.

[0174] Input: Progression script

[0175] Data processing / calculation: Based on the progress script, the terminal uses an announcer AI to conduct the meeting by voice and display the necessary materials.

[0176] Output: Ongoing voice instructions, display of materials

[0177] Step 5:

[0178] Questions and Answers

[0179] During a meeting, a user asks a question. The device recognizes the question and sends it to the server as text data.

[0180] Input: Participant's question (audio data)

[0181] Data processing / calculation: The terminal converts the voice data into text, and the server receives the text data, searches for relevant information from a database, and generates an answer.

[0182] Output: Text data question (sent to server), answer (text format, sent to terminal)

[0183] Step 6:

[0184] Meeting minutes and review

[0185] The device records the content of discussions during the meeting in real time and sends the data to a server. After the meeting ends, the server organizes the recorded content, categorizes the issues, and proposes solutions.

[0186] Input: Real-time discussion content (audio data)

[0187] Data processing / calculation: The terminal converts the voice data into text, and the server organizes and classifies the content of the discussion based on the text data, generating solutions to the problems.

[0188] Output: Record contents, issue classification, solution method (report to user)

[0189] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[0190] (Application example 1)

[0191] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0192] There is a need for a system in factories that can efficiently issue work instructions and conduct meetings, quickly respond to worker questions, and allow users to review recorded meeting content and work content later. It is also necessary to utilize facial recognition and voice recognition technology to accurately grasp the attendance status of workers and quickly provide necessary information. The present invention aims to provide a system that solves these problems.

[0193] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0194] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and factory workers and updating attendance status, means for generating a conference progress script and preparing and displaying materials to be used during the conference and work instruction conference, means for acquiring questions from participants and workers as text data using voice recognition technology during the conference and during work and generating answers to those questions, means for recording the contents of the conference and work, categorizing the issues discussed, and means for generating and reporting proposals for solving the issues. This allows for smooth progress of conferences and work instruction discussions in the factory and for quick responses to worker questions. Furthermore, the recorded contents of the conference and work can be reviewed later, leading to efficient business improvement and optimization.

[0195] "Meeting setting information" refers to detailed information required to hold a meeting, such as the date, time, location, participants, and agenda of the meeting.

[0196] A "meeting schedule" is a plan of how a meeting will proceed, including the start time, end time, break times, and the order of each agenda item.

[0197] "Facial recognition" is a technology that uses a camera to identify a person's face and identify the person in question.

[0198] "Attendance status" indicates information on those who actually attended a meeting or task and those who were absent.

[0199] A "meeting progress script" is a script for smoothly running a meeting, and includes the procedure for proceeding with each agenda item and the order in which speakers will speak.

[0200] "Materials" refers to information media such as documents, charts, presentation materials, etc. used during meetings and work instruction meetings.

[0201] A "question" is a question or confirmation that a meeting participant or factory worker submits during a meeting or during work instructions.

[0202] An "answer" is an explanation or answer to a question posed.

[0203] "Speech recognition technology" is a technology that analyzes speech and converts it into text data.

[0204] "Recording" refers to the act of saving statements and discussions made during meetings or work instruction meetings as digital data.

[0205] "Issues" are problems or areas for improvement that were discussed during meetings or work instruction meetings.

[0206] "Categorization" is the process of classifying and organizing recorded tasks.

[0207] A "proposal" is a proposal that shows a solution or improvement to a specific problem.

[0208] "Factory workers" are employees or workers who actually perform work in a factory.

[0209] This invention relates to a system that supports efficient progress of meetings and work instructions in factories and problem-solving. This system receives and manages setting information for meetings and work instructions, identifies participants and workers using face recognition technology, prepares progress scripts and materials, provides quick answers to questions from participants and workers, and organizes issues and proposes solutions after meetings and work.

[0210] The system consists of the following main components:

[0211] 1. Server

[0212] The server controls the entire system and is responsible for managing and processing data.

[0213] The server receives the setting information of the meeting and the work instruction, and generates and saves the meeting schedule.

[0214] The server processes face recognition data of meeting and work order participants and updates their attendance status.

[0215] The server generates a progress script and materials and sends them to the terminal.

[0216] The server searches for relevant information in response to questions from participants and workers and generates answers.

[0217] The server records the meeting and work instructions and stores them in a format that can be accessed later.

[0218] 2. Terminal

[0219] The terminal functions as a user interface and receives instructions from the server to proceed with the conference and work instructions.

[0220] The device uses a camera and facial recognition function to identify participants and workers.

[0221] The terminal has the function of assisting the progress based on the progress script and materials received from the server.

[0222] The terminal recognizes the questions of participants and workers through voice recognition and transmits them to the server as text data.

[0223] The terminal displays the answers and materials received from the server to participants in the meeting and at the work instruction site, and provides audio explanations.

[0224] The terminal records discussions during meetings and work instructions in real time and transmits them to the server.

[0225] 3. Users

[0226] The user inputs the setting information for the meeting and work instructions and registers it in the system.

[0227] Users can ask questions and give opinions during meetings and work instructions, and use the system's interface.

[0228] The system uses the following hardware and software:

[0229] Camera: A device for performing facial recognition.

[0230] Microphone: A device for voice recognition.

[0231] Server: A computer that performs data management and processing.

[0232] AI assistant: Software that uses voice recognition technology and document generation, etc.

[0233] Uses the Python libraries: facial_recognition, audio_processing, and document_processing.

[0234] As a concrete example, consider the case of holding a meeting to explain new work procedures. In this case, the robot will start the meeting at a set date and time, recognize the faces of the participants, and take attendance. It will then present materials based on the procedure explanation script and answer safety-related questions in real time. After the meeting, it will automatically generate minutes and provide feedback to the relevant parties.

[0235] Example prompt sentence:

[0236] "Please schedule a meeting to explain the new procedures. The date and time is October 15th at 2 PM, and the participants are the engineering and quality control teams."

[0237] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0238] Step 1:

[0239] The user inputs the setting information for the meeting or work instruction and sends it to the server.

[0240] Input: Meeting or work order setup information such as date, time, location, participants, and agenda.

[0241] Specific operation: The user inputs the configuration information through the interface and clicks the "Submit" button, which sends the input information to the server.

[0242] Output: The server has received the configuration information and is ready to proceed to the next processing step.

[0243] Step 2:

[0244] The server generates a conference schedule based on the received setting information and transmits it to the terminal.

[0245] Input: Meeting and work order setup information received from users.

[0246] Specific operation: The server determines the timetable for each agenda item based on the setting information, generates a meeting schedule, and sends the generated schedule to the terminal.

[0247] Output: Meeting schedule.

[0248] Step 3:

[0249] The server generates a progress script and prepares the materials necessary for meetings and work instructions.

[0250] Input: Meeting schedule and related configuration information.

[0251] Specific operation: The server generates a progress script based on the schedule and stores appropriate materials (documents, diagrams, presentations, etc.) in cloud storage.

[0252] Output: Progress script and materials.

[0253] Step 4:

[0254] The device uses a camera to recognize the faces of participants and workers and sends the results to the server.

[0255] Input: Camera video data.

[0256] Specific operation: Images are captured in real time using the device's built-in camera, and participants and workers are identified using facial recognition technology. The identified facial recognition data is sent to the server.

[0257] Output: Recognized participant and worker data.

[0258] Step 5:

[0259] The server updates attendance status based on the facial recognition data and sends the attendee list to the terminal.

[0260] Input: facial recognition data.

[0261] Specific operation: The server verifies the sent facial recognition data and updates the attendee list, which is then sent to the terminal and displayed on the terminal during the meeting or work order.

[0262] Output: Attendees list.

[0263] Step 6:

[0264] The terminals conduct meetings and work based on the progress script and display materials.

[0265] Input: Progression script and materials.

[0266] Specific operation: The terminal reads out the progress script and displays materials on the screen while proceeding with the meeting and work instructions. It provides necessary information to participants and workers according to the progress instructions.

[0267] Output: Progress of meetings and work instructions.

[0268] Step 7:

[0269] Participants and workers ask questions by voice, and the device recognizes the voice and sends it to the server.

[0270] Input: Participant or worker voice question.

[0271] Specific operation: The device uses a microphone to capture the voice of the question, converts it into text data using voice recognition technology, and sends the converted data to the server.

[0272] Output: Question data converted from speech to text.

[0273] Step 8:

[0274] The server generates an answer to the question and sends it to the terminal.

[0275] Input: Speech-to-text question data.

[0276] Specific operation: The server accesses the database, searches for relevant information, generates an answer, and sends the answer to the terminal.

[0277] Output: Response data.

[0278] Step 9:

[0279] The device displays the answer received from the server and explains it aloud.

[0280] Input: Response data.

[0281] Specific operation: The terminal displays the received response data on the screen and uses voice synthesis technology to communicate the response to the participant or worker aloud.

[0282] Output: Providing answers to participants or workers.

[0283] Step 10:

[0284] The terminal records the contents of meetings and work instructions and sends them to the server.

[0285] Input: Discussions and progress during meetings and work instructions.

[0286] Specific operation: The device records the contents of meetings and work instructions in real time and periodically sends the data to the server.

[0287] Output: Recorded meetings and work.

[0288] Step 11:

[0289] After the meeting or work instruction is completed, the server organizes the recorded content and stores it in a format that can be accessed later.

[0290] Input: Recorded meeting and work data.

[0291] Specific operation: The server analyzes the recorded data, categorizes each issue, generates analysis results and proposals, and stores them in cloud storage or an internal database.

[0292] Output: Saved meetings and work, as well as analysis results and recommendations.

[0293] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0294] This invention combines an emotion engine with a system that supports efficient meeting progress and problem-solving. This system receives and manages meeting setting information, identifies participants using facial recognition technology, prepares meeting scripts and materials, provides quick answers to questions from participants, and organizes issues and proposes solutions after the meeting. It also improves the quality of meetings by recognizing and utilizing participants' emotions in real time.

[0295] (System configuration)

[0296] The system includes the following major components:

[0297] 1. Server

[0298] The server controls the entire system and is responsible for managing and processing data.

[0299] Receives meeting setting information and generates and saves the meeting schedule.

[0300] Processes facial recognition data of meeting participants and updates their attendance status.

[0301] Generate progress scripts and materials and send them to the terminal.

[0302] It searches for relevant information and generates answers to questions from participants.

[0303] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[0304] Uses an emotion engine to analyze and store emotional data from meeting participants' facial expressions and tone of voice.

[0305] 2. Terminal

[0306] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[0307] Meeting participants are identified using a camera and facial recognition function.

[0308] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[0309] Participants' questions are recognized by voice and sent to the server as text data.

[0310] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[0311] Discussions during meetings are recorded in real time and sent to a server.

[0312] Using a camera and microphone, participants' facial expressions and tone of voice are analyzed and emotional data is sent to a server.

[0313] 3. Users

[0314] The user enters the conference setting information and registers it in the system.

[0315] Ask questions and give opinions during the meeting and use the system interface.

[0316] (Specific operation example)

[0317] For example, consider a case where you are holding a kickoff meeting for a new product and want to recognize participants' emotions in real time and reflect them in the progress of the meeting.

[0318] 1. A user schedules a meeting

[0319] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[0320] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[0321] 2. Pre-meeting preparation

[0322] The server creates a meeting script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[0323] The terminal downloads this data from the server and prepares to start the meeting.

[0324] 3. Participant facial and emotion recognition

[0325] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[0326] The server updates attendance status based on facial recognition data and also records information on absentees.

[0327] At the same time, the device uses a camera and microphone to analyze the participants' facial expressions and tone of voice, and transmits the emotional data to the server.

[0328] 4. Meeting management and emotional feedback

[0329] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[0330] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[0331] The server uses an emotion engine to adjust the announcer AI's tone and method of proceeding according to the participant's emotional state. For example, if a participant shows dissatisfaction or confusion, the announcer AI will prompt questions or provide additional explanations.

[0332] 5. Questions and Answers

[0333] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[0334] The device recognizes the question and sends it to the server as text data.

[0335] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[0336] The device displays the answer in document form, and an AI announcer provides the answer aloud.

[0337] 6. Organizing meeting records and emotion data

[0338] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[0339] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[0340] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[0341] 7. Proposal for solving the problem

[0342] The proposed solution also includes feedback based on emotional data.

[0343] For example, you can provide particularly detailed solutions to issues that participants complained about during the meeting.

[0344] The terminal reports the proposed solution to the user and provides it in an exportable format.

[0345] By using this system, it is possible to consistently and efficiently carry out everything from preparing for meetings to conducting them, answering questions, and post-meeting reflection and problem-solving. It also makes it possible to further improve the quality of meetings by utilizing feedback from an emotion engine.

[0346] The processing flow will be explained below.

[0347] Step 1:

[0348] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[0349] Step 2:

[0350] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[0351] Step 3:

[0352] At least several hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, specifications for a new product, etc.) and stores them in cloud storage.

[0353] Step 4:

[0354] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[0355] Step 5:

[0356] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data and attendee information are then sent to the server.

[0357] Step 6:

[0358] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[0359] Step 7:

[0360] The device uses a camera and microphone to analyze the facial expressions and tone of voice of meeting participants, obtain emotional data, and send it to the server.

[0361] Step 8:

[0362] The server analyzes the emotional data using an emotion engine and records the emotional state of the participants.

[0363] Step 9:

[0364] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[0365] Step 10:

[0366] The terminal displays materials based on the progress script, and the announcer AI gives an explanation. The server uses emotional data to adjust the announcer AI's tone and progress according to the participants' reactions.

[0367] Step 11:

[0368] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[0369] Step 12:

[0370] The terminal sends the text data of the question to the server.

[0371] Step 13:

[0372] The server searches the database for relevant information, generates an answer to the question, and sends the generated information and answer to the terminal.

[0373] Step 14:

[0374] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[0375] Step 15:

[0376] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[0377] Step 16:

[0378] After the meeting, the server organizes the recorded data and categorizes the issues discussed.

[0379] Step 17:

[0380] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[0381] Step 18:

[0382] The terminal reports the proposed solution to the user and provides it in an exportable format.

[0383] Example 2

[0384] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0385] Conventional conference systems not only lack efficiency in meeting progress and problem-solving, but also tend to lower the quality of meetings because they cannot take into account the emotions of participants.In addition, there are issues with the overall process being complicated and time-consuming, as it requires a lot of manual work, such as setting up meetings, confirming participants, preparing materials, answering questions, and following up after meetings.

[0386] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0387] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for solving the issues, and means for recognizing the emotions of conference participants in real time and reflecting them in the conference progress. This enables consistent and efficient processing of the conference, from preparation to progress, question and answer session, and post-conference issue organization and solution proposals. Furthermore, recognizing participants' emotions in real time and reflecting them in the conference progress can improve the quality of the conference.

[0388] 1. "Meeting setting information" refers to the detailed information necessary to hold a meeting, such as the date and time of the meeting, participants, location, and agenda.

[0389] 2. "Means for generating a conference schedule" refers to a device or program that has the function of automatically creating and saving a schedule, such as the date and time, location, and participant list of a conference, based on the conference setting information.

[0390] 3. "Means for recognizing the faces of meeting participants and updating attendance status" refers to devices or programs that use cameras and facial recognition technology to identify the faces of meeting participants, confirm attendance, and update logs.

[0391] 4. A "meeting script" is a document or instruction manual that describes the procedures and steps for conducting a meeting, and serves as a guideline for ensuring that the meeting proceeds on time.

[0392] 5. "Materials used during the meeting" refers to information that supports the progress of the meeting, such as data, reports, and presentation materials related to the meeting agenda.

[0393] 6. "Means for accepting questions from participants during the conference and generating answers to those questions" refers to devices or programs that have the function of receiving questions from participants, analyzing their content, and providing appropriate answers.

[0394] 7. "Means for recording meeting content and categorizing the issues discussed" refers to devices or programs that have the function of recording the content of discussions during meetings and categorizing and organizing that content.

[0395] 8. "Means for generating and reporting proposals for solving problems" refers to devices or programs that have the function of analyzing the problems discussed in the meeting, coming up with solutions, and compiling them into documents or reports.

[0396] 9. "Means of recognizing the emotions of meeting participants in real time and reflecting them in the progress of the meeting" refers to devices or programs that use cameras and microphones to analyze the facial expressions and tone of voice of meeting participants and adjust the way the meeting is conducted based on that emotional data.

[0397] The present invention combines an emotion engine with a system that supports efficient meeting progress and problem solving, and includes the following main components:

[0398] System configuration

[0399] This system is composed of server, terminal, and user components, and by clearly defining the roles of each component, efficient conference management is achieved.

[0400] server

[0401] Managing conference settings information

[0402] The server receives the meeting setting information and generates a meeting schedule. For example, specific setting information such as "A kickoff meeting for a new product is scheduled for May 15th at 2:00 PM. Participants include the general manager, project manager, and engineering team" is entered. The meeting schedule is then saved in the database and invitation emails are sent to participants.

[0403] Facial recognition and attendance updates

[0404] The server processes the facial recognition data of the conference participants and updates the attendance status. Specifically, it analyzes the facial recognition data sent from the terminals and updates the attendee list.

[0405] Preparing the script and materials

[0406] The server generates a meeting progress script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage and prepared for transmission to the terminals.

[0407] Q&A and problem identification

[0408] The server accepts questions from participants and searches a database to generate answers to those questions. It also includes a means to record the meeting content and categorize the issues discussed.

[0409] Analysis by emotion engine

[0410] The server analyzes the emotional data sent from the devices and reflects the feedback in the progress script. The emotion engine analyzes the participants' facial expressions and tone of voice and makes adjustments according to the progress of the meeting.

[0411] Terminal

[0412] User Interface

[0413] The device functions as a user interface and receives instructions from the server to conduct the conference. It uses a camera and facial recognition function to recognize the faces of participants and transmits the data to the server.

[0414] Announcer AI

[0415] The AI ​​announcer on the device will conduct the meeting by voice, displaying the necessary materials based on the meeting script, and will recognize the user's questions and send them to the server as text data.

[0416] Real-time recording and analysis of emotion data

[0417] The device records the content of discussions in real time during the meeting and sends it to the server. It also uses a camera and microphone to analyze participants' facial expressions and tone of voice, and sends emotional data to the server.

[0418] User

[0419] Meeting setup and questions

[0420] Users enter meeting setting information and register it in the system. During the meeting, they can ask questions and give opinions and use the system's interface.

[0421] Specific examples

[0422] Consider a case where a kickoff meeting for a new product is being held and participants' emotions are recognized in real time and reflected in the progress of the meeting. The user sets up the meeting, and the server generates a meeting schedule and notifies participants. The device receives the prepared materials and performs facial recognition at the start of the meeting. The server sends instructions to the device based on the progress script, and the device's announcer AI follows these instructions to proceed with the meeting. During the meeting, the user asks a question, and the device recognizes the question through voice recognition and sends it to the server. The server generates an answer, which the device displays and explains in audio. After the meeting ends, the server organizes the recorded data and generates proposals for solving problems based on the emotional data. This improves the efficiency and quality of meetings.

[0423] The advantage of this system is that it can efficiently handle everything from preparing for meetings to conducting them, answering questions, and sorting out issues and proposing solutions after the meeting. Furthermore, it can improve the quality of meetings by utilizing feedback from an emotion engine.

[0424] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0425] Step 1: Receiving meeting settings

[0426] The user enters meeting setting information. For example, the user might enter "New product kickoff meeting scheduled for May 15th at 2:00 PM. Participants will be the general manager, project manager, and engineering team."

[0427] Input: Conference setup information from the user.

[0428] Data processing: The server receives this information and analyzes and organizes the conference setting information.

[0429] Output: Structured conference setup data.

[0430] Specific operation: The server saves the conference setting information in a database and sends invitation emails to participants as needed.

[0431] Step 2: Generate and save the meeting schedule

[0432] The server generates a conference schedule based on the received setting information.

[0433] Input: Conference setup information.

[0434] Data processing: Generate schedules based on meeting dates, times, locations, and participant lists.

[0435] Output: Meeting schedule information.

[0436] Specific operation: The server saves the generated schedule in the database and sends invitation emails to participants.

[0437] Step 3: Prepare the script and materials

[0438] The server creates a progress script for the conference and prepares the necessary materials.

[0439] Input: Meeting setup information and related materials (market research data, new product specifications, etc.).

[0440] Data processing: Organizing and generating procedures and materials.

[0441] Output: Progress script and preparation materials.

[0442] Specific operation: The material is stored in cloud storage and prepared to be sent to the device.

[0443] Step 4: Participant facial and emotion recognition

[0444] The device uses a camera to recognize the faces of participants at the start of the meeting.

[0445] Input: Face images of meeting participants.

[0446] Data processing: Data processing using facial recognition algorithms.

[0447] Output: Participant's facial recognition data.

[0448] Specific operation: The device sends facial recognition data to the server, which then updates the attendance status. At the same time, the device uses the camera and microphone to analyze emotion data and send it to the server.

[0449] Step 5: Conducting the meeting

[0450] The server sends instructions to the terminal based on the progress script.

[0451] Input: Progress script.

[0452] Data processing: Analyzes the instructions and sends them to the terminal.

[0453] Output: Progression instructions data.

[0454] How it works: The AI ​​announcer on the device conducts the meeting by voice while displaying the necessary materials. The emotion engine on the server analyzes the emotions of the participants and sends feedback to the device to adjust the way the meeting is conducted.

[0455] Step 6: Receiving questions and providing answers

[0456] The user (participant) asks a question by voice. For example, "Tell me about your new product launch plan."

[0457] Input: Participant's question audio data.

[0458] Data processing: Performs voice recognition and converts it into text data.

[0459] Output: Question text data.

[0460] How it works: The device recognizes the question and sends it as text data to the server. The server searches for relevant information in a database and generates an answer using a generative AI model. The device displays the answer in a document format, and the announcer AI provides the answer via voice.

[0461] Step 7: Record the meeting and organize the emotional data

[0462] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[0463] Input: Meeting discussion content and participants' emotional data.

[0464] Data processing: Converting discussion content into text and analyzing emotional data.

[0465] Output: Recorded data and emotion data.

[0466] What it does: The server organizes the recorded data and categorizes the issues discussed.

[0467] Step 8: Generate problem-solving proposals

[0468] The server generates problem-solving proposals based on the recorded data.

[0469] Input: Recorded data and emotion data.

[0470] Data processing: Data analysis and generation of proposals using generative AI models.

[0471] Output: A document proposing a solution to the problem.

[0472] How it works: The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[0473] Step 9: Report the proposed solution

[0474] The server documents the generated proposals and reports them to the user.

[0475] Input: A proposal document for solving the problem.

[0476] Data processing: Proposal documentation and format conversion.

[0477] Output: Proposal report.

[0478] Specific operation: The server exports the proposed solution in document format, and the device displays and reports it to the user.

[0479] (Application example 2)

[0480] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0481] Although systems exist to support efficient meeting progress, there is a problem in the industrial sector where communication and discussions between workers are not sufficiently efficient. Furthermore, because there is no technology to analyze emotional data in real time and reflect it in the progress of meetings, it is difficult to respond quickly when participants have complaints or questions. This can lead to a decline in the quality and productivity of meetings and delays in problem-solving.

[0482] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0483] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for resolving the issues, means for analyzing participants' facial expressions and voice data in real time and saving the data as emotion data, and means installed on industrial terminals for adjusting the progress script based on the emotion data during the conference. This makes it possible to improve the efficiency and quality of industrial conferences and to analyze and respond to participants' emotions in real time.

[0484] definition statement

[0485] "Meeting setting information" is information that includes details of the date, time, location, participants and agenda of the meeting.

[0486] The "conference schedule" is a schedule generated based on the setting information of the conference, and includes the date and time of the conference, the order of proceedings on the agenda, and the duration of each agenda item.

[0487] A "conference participant" is someone who is scheduled to attend a conference.

[0488] "Facial recognition" is a technology that identifies a person's face from image data and identifies that person.

[0489] "Attendance status" is information indicating the attendance of conference participants.

[0490] A "meeting progress script" is a script created to support the progress of a meeting, and describes in detail the explanation of each agenda item and the progress procedure.

[0491] "Materials" are files containing documents, presentations, graphs, and other information prepared for use during a meeting.

[0492] "Questions" refer to questions or comments submitted by meeting participants during the meeting.

[0493] "Response" refers to an answer or explanation to a submitted question.

[0494] "Meeting content" is information including the topics discussed, statements, and decisions made during a meeting.

[0495] An "issue" refers to a problem to be discussed or resolved during a meeting.

[0496] "Categorization" refers to classifying the issues discussed into specific categories.

[0497] A "proposal" refers to a specific action plan or measure offered to solve a problem.

[0498] "Facial expression data" is information indicating the emotional state obtained from a person's face.

[0499] "Voice data" refers to acoustic information obtained from a person's voice, and is also used to analyze emotional states.

[0500] "Emotion data" is information indicating an emotional state based on facial expression data and voice data analyzed in real time.

[0501] An "industrial terminal" is a computer terminal used in factories and manufacturing sites, and is typically a rugged, durable piece of equipment.

[0502] "Adjusting the progress script" means changing the meeting progress script based on real-time emotional data and responding according to the participants' situations.

[0503] MODE FOR CARRYING OUT THE INVENTION

[0504] This invention is built around a conference support system installed on industrial terminals, and provides consistent support from receiving conference setting information to proposing solutions to problems after the conference. The system is composed of multiple components, including a server, terminals, and an emotion analysis engine.

[0505] The server operates as follows:

[0506] 1. Receives conference setting information and generates and saves a conference schedule. The schedule is created based on the conference date, time, location, participant information, etc. set by the user.

[0507] 2. Uses facial recognition technology to identify meeting participants and update their attendance status. The server manages this information and updates the meeting attendee list in real time.

[0508] 3. Generate a meeting script, prepare materials to be used during the meeting, save them in cloud storage, and then download these materials to the device.

[0509] 4. During the conference, participants can ask questions, search for answers in the database, and generate answers. Questions are converted into text using speech recognition technology.

[0510] 5. Record the meeting and categorize the issues discussed. The recorded data will be organized and categorized for later analysis.

[0511] 6. The emotion engine analyzes participants' facial expressions and voice data in real time and stores them as emotion data. This information is used to adjust the script as the meeting progresses.

[0512] 7. After the meeting, a proposal for solving the problem is generated based on the recorded meeting content and emotional data, and reported to the device. This proposal is provided in an accessible format for later use.

[0513] The terminal operates as follows:

[0514] 1. Prepare for the meeting based on the meeting schedule, progress script, and materials downloaded from the server.

[0515] 2. Use face recognition to identify participants and send their attendance status to the server. Use a camera and face recognition technology to identify and confirm participants' faces.

[0516] 3. During the meeting, the server displays the progress script and presents materials to attendees. It also recognizes participants' questions through a microphone and sends them to the server.

[0517] 4. The AI ​​announcer provides audible support for the meeting, adjusting the script as needed based on emotional data. If participants show confusion or dissatisfaction, the AI ​​announcer will provide additional explanations to help participants understand.

[0518] 5. The contents of the meeting discussion are recorded in real time and sent to the server along with emotion data.

[0519] The user uses it as follows:

[0520] 1. Enter and register the meeting setting information on the server. For example, you could set a new product kickoff meeting for May 15th at 2:00 PM, with the department head, project manager, and engineering team as participants.

[0521] 2. Ask questions or make comments during the meeting and send them to the system. For example, "Tell me about your plans to bring new products to market."

[0522] 3. After the meeting, the solution to the problem proposed by the server is confirmed and implemented.

[0523] The main hardware and software used are as follows:

[0524] Face recognition technology: OpenCV

[0525] Speech recognition technology: Google Speech-to-Text API

[0526] Emotion Recognition Engine: TensorFlow-based model

[0527] Meeting Script Generation: Custom Script Generation Tool

[0528] Announcer AI: Voice generation using TTS (Text-to-Speech) (e.g., pyttsx3)

[0529] The following prompt sentences are concrete examples of practical applications of this system:

[0530] "There are five participants in the meeting. Please show the script and provide additional explanations when participants show signs of dissatisfaction."

[0531] In this way, this invention supports the conduct of meetings within factories and helps achieve more effective and efficient communication by analyzing participants' emotions in real time.

[0532] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0533] Processing flow

[0534] Step 1: Receiving conference setting information and generating a conference schedule

[0535] The server receives meeting setting information (date and time, location, participant information, etc.) entered by the user. It then generates a meeting schedule based on this information and stores it in a database. The generated meeting schedule includes the date and time of the meeting, the order of agenda items, and the duration of each agenda item.

[0536] Input: Conference setup information from the user

[0537] Data processing / calculation: Analysis and scheduling of setting information

[0538] Output: Meeting schedule

[0539] Step 2: Face recognition and attendance updates for meeting participants

[0540] At the start of a meeting, the device uses a camera to capture the faces of participants and sends the data to the server, which then uses facial recognition technology to identify participants and update their attendance status, which is then saved in a database in real time.

[0541] Input: Facial image data from the camera

[0542] Data processing / calculation: Facial recognition analysis, attendance status update

[0543] Output: Attendees list

[0544] Step 3: Prepare the meeting script and materials

[0545] The server generates a progress script based on the meeting setting information and past meeting data, and saves the materials to be used during the meeting in cloud storage. The server then sends the progress script and materials to the device, which then downloads them and prepares for the meeting.

[0546] Input: Meeting setting information, past meeting data

[0547] Data processing / calculation: Script generation and document creation

[0548] Output: Progress script, meeting materials

[0549] Step 4: Real-time emotion recognition and script adjustment

[0550] During the meeting, the device uses a camera and microphone to analyze participants' facial expressions and voices and transmits the data to a server. The server then uses a generative AI model (such as TensorFlow) to analyze the emotional data in real time and adjust the meeting script as needed. For example, if a participant shows dissatisfaction or confusion, the server will change the meeting script to provide additional explanation.

[0551] Input: facial expression and voice data from camera and microphone

[0552] Data processing / calculation: Emotion analysis, script adjustment

[0553] Output: Adjusted progress script

[0554] Step 5: Speech recognition of questions and answer generation

[0555] When a participant asks a question during a meeting, the device captures the voice through the microphone and converts it into text data using the Google Speech-to-Text API. This text data is sent to a server, which then searches a database for relevant information, generates an appropriate answer, and sends it to the device. The device then uses announcer AI to provide the answer via voice.

[0556] Input: Audio data from microphone (question)

[0557] Data processing / calculation: speech recognition, database search, answer generation

[0558] Output: Answer to question (text and audio)

[0559] Step 6: Record meeting content and emotional data

[0560] The device records what is said and the key points of discussion during the meeting in real time and sends the data to a server. The server also stores participants' emotional data. The server then organizes this data and provides it in a format that can be accessed later.

[0561] Input: Contents of remarks made during meetings, emotional data

[0562] Data processing / calculation: Data recording and organization

[0563] Output: Recorded meeting content, organized emotion data

[0564] Step 7: Generate problem-solving proposals

[0565] After the meeting, the server analyzes the recorded content and emotional data, categorizes the issues discussed, and then proposes specific solutions for each issue and sends them to the device. These proposals are then provided in a format that can be accessed later.

[0566] Input: Recorded meeting content, emotion data

[0567] Data processing / calculation: Data analysis, categorization, and proposal generation

[0568] Output: Problem-solving proposal

[0569] The implementation of this system will enable meetings within factories to proceed efficiently and effectively, and will achieve better communication by reflecting the emotions of participants in real time.

[0570] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0571] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0572] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0573] [Second embodiment]

[0574] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0575] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0576] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0577] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0578] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0579] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0580] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0581] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0582] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0583] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0584] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0585] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0586] This invention relates to a system that supports efficient progress of meetings and problem-solving. This system receives and manages meeting setting information, identifies participants using face recognition technology, prepares meeting scripts and materials, provides prompt answers to questions from participants, and organizes issues and proposes solutions after the meeting.

[0587] (System configuration)

[0588] The system includes the following major components:

[0589] 1. Server

[0590] The server controls the entire system and is responsible for managing and processing data.

[0591] Receives meeting setting information and generates and saves the meeting schedule.

[0592] Processes facial recognition data of meeting participants and updates their attendance status.

[0593] Generate progress scripts and materials and send them to the terminal.

[0594] It searches for relevant information and generates answers to questions from participants.

[0595] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[0596] 2. Terminal

[0597] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[0598] Meeting participants are identified using a camera and facial recognition function.

[0599] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[0600] Participants' questions are recognized by voice and sent to the server as text data.

[0601] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[0602] Discussions during meetings are recorded in real time and sent to a server.

[0603] 3. Users

[0604] The user enters the conference setting information and registers it in the system.

[0605] Ask questions and give opinions during the meeting and use the system interface.

[0606] (Specific operation example)

[0607] As an example, consider holding a kickoff meeting for a new product.

[0608] 1. A user schedules a meeting

[0609] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[0610] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[0611] 2. Pre-meeting preparation

[0612] The server creates a meeting script and prepares and stores materials related to the agenda (e.g., market research data, new product specifications, etc.) in cloud storage.

[0613] The terminal downloads this data from the server and prepares to start the meeting.

[0614] 3. Participant facial recognition

[0615] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[0616] The server updates attendance status based on facial recognition data and also records information on absentees.

[0617] 4. Meeting Management

[0618] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[0619] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[0620] 5. Questions and Answers

[0621] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[0622] The device recognizes the question and sends it to the server as text data.

[0623] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[0624] The device displays the answer in document format, and an AI announcer explains it aloud.

[0625] 6. Recording and reviewing the meeting

[0626] The device records the content of discussions during the meeting in real time and sends it to the server.

[0627] After the meeting, the server organizes the recordings and categorizes each topic.

[0628] The server performs the analysis, proposes solutions to each problem, and reports them to the user via the terminal.

[0629] Using this system, you can consistently and efficiently prepare for meetings, conduct them, answer questions, and review and resolve issues after the meeting.

[0630] The processing flow will be explained below.

[0631] Step 1:

[0632] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[0633] Step 2:

[0634] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[0635] Step 3:

[0636] At least a few hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[0637] Step 4:

[0638] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[0639] Step 5:

[0640] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data is then sent to the server.

[0641] Step 6:

[0642] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[0643] Step 7:

[0644] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[0645] Step 8:

[0646] The terminal displays materials based on the progress script, and an AI announcer provides explanations.

[0647] Step 9:

[0648] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[0649] Step 10:

[0650] The terminal sends the text data of the question to the server.

[0651] Step 11:

[0652] The server searches the database for relevant information and generates answers to the questions, which are then sent to the device.

[0653] Step 12:

[0654] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[0655] Step 13:

[0656] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[0657] Step 14:

[0658] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[0659] Step 15:

[0660] The server performs a detailed analysis of each problem and generates a solution proposal, including an action plan and predicted outcomes.

[0661] Step 16:

[0662] The terminal reports the proposed solution to the user and provides it in an exportable format.

[0663] Step 17:

[0664] The user receives the report and develops an action plan as needed.

[0665] Example 1

[0666] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0667] Conventional meeting management has the problem of being difficult to manage consistently from setting up the meeting to proceeding, answering questions, and reviewing and resolving issues after the meeting. This has led to problems such as the inability to efficiently organize the information necessary for the meeting, confirm attendance, record the meeting content in real time, and process the post-meeting information.

[0668] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0669] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for recognizing characteristics of conference participants and updating attendance status, means for generating a conference progress script and preparing information to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and classifying the issues discussed, and means for generating and reporting proposals for solving the issues. This makes it possible to efficiently and consistently manage the entire conference process, and quickly organize information, confirm attendance, record in real time, and organize issues and propose solutions after the conference.

[0670] "Meeting setting information" refers to basic information required to hold a meeting, such as the date and time of the meeting, location, participants, and agenda.

[0671] "Means for generating a meeting schedule" refers to a function that automatically creates and saves specific meeting schedules based on received meeting setting information.

[0672] "Feature recognition of meeting participants" refers to technology that uses facial recognition technology to identify meeting participants and confirm their attendance status.

[0673] "Means for updating attendance status" refers to a function that updates the attendee list in real time based on the results of feature recognition of meeting participants and manages attendance and absence status.

[0674] "Means for generating a meeting progress script" refers to a function that automatically creates a series of steps that compile the instructions and information necessary to progress the meeting.

[0675] "Means for preparing information to be used during a meeting" refers to the function of collecting and organizing necessary materials and data based on the meeting progress script and making them available during the meeting.

[0676] "Means for accepting questions and generating answers to those questions" refers to a function for receiving questions submitted by participants during a meeting and generating answers to address those questions immediately.

[0677] "A means of recording meeting content and classifying the issues discussed" refers to a function that records what is said and discussed during a meeting in real time, and then organizes and classifies the content by issue.

[0678] "Means for generating and reporting proposals for resolving issues" refers to the function of analyzing the recorded meeting content, proposing specific solutions to identified issues, and reporting them to the relevant parties.

[0679] The system of the present invention is designed to support efficient progress of meetings and problem-solving. This system is composed of a server, terminals, and users.

[0680] server

[0681] The server controls the entire system and is responsible for managing and processing data. Specifically, the server receives meeting setting information from users, generates a meeting schedule, stores it in a database, and then automatically sends invitation emails to each participant.

[0682] The server uses facial recognition technology to identify meeting participants and update their attendance status.The server uses a built-in database system (e.g., MySQL) and cloud storage (e.g., Amazon S3) to generate and store the meeting script and related materials.

[0683] During the meeting, the system accepts questions from participants, searches for relevant information in its database, and generates answers. The answers are sent to the terminal in real time. It also records the contents of the meeting, categorizes the issues after the meeting, and generates and reports proposed solutions.

[0684] Terminal

[0685] The device functions as a user interface and receives instructions from the server to proceed with the conference. The device is equipped with a camera and facial recognition function, which identifies participants at the start of the conference and sends that data to the server.

[0686] The AI ​​announcer on the device will conduct the meeting based on the script and materials received from the server, outputting instructions such as "First, I'll give an overview of the new product."

[0687] During the meeting, the device recognizes participants' voice questions and sends them as text data to the server. Answers and materials received from the server are displayed on the device's screen, and an AI announcer provides audible explanations.

[0688] The device records the meeting contents in real time and sends the data to a server, allowing participants to review the meeting contents later and use them as reference material for problem-solving.

[0689] User

[0690] The user enters meeting setting information and registers it in the system. For example, the user enters information such as "The kickoff meeting for the new product is scheduled for May 15th at 2:00 PM."

[0691] During the meeting, users can ask questions and give opinions and use the system's interface to help the meeting proceed.

[0692] Specific examples

[0693] As an example, consider the case of holding a kickoff meeting for a new product. The user sets up the meeting, and the server generates a schedule and notifies participants. At the start of the meeting, the device performs facial recognition and reports attendance status to the server. The server sends a progress script to the device, and an announcer AI leads the meeting. Questions from participants are sent to the server via the device, and answers are instantly generated and displayed. After the meeting ends, the server organizes the recorded content, categorizes issues, and proposes solutions.

[0694] Example prompts for generative AI models

[0695] "Please explain in detail the process for generating the meeting script and related materials."

[0696] "Please explain the specific flow of questions and responses during a meeting."

[0697] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[0698] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0699] Step 1:

[0700] Enter and register meeting setting information

[0701] The user inputs the meeting setting information (e.g., the name of the meeting, the date and time, the location, the participant list, and the details of the agenda) and registers it in the system. The input data is sent to the server.

[0702] Input: Meeting settings information (meeting name, date, time, location, participant list, agenda details)

[0703] Data processing / calculation: The server receives the input information, generates a meeting schedule and participant list, stores them in a database, and generates and sends invitation emails to each participant.

[0704] Output: Meeting schedule, participant list, invitation email

[0705] Step 2:

[0706] Preparing the script and materials

[0707] The server generates a progress script and related materials (e.g., market research data, specifications for new products, etc.) required for the conference.

[0708] Input: Meeting setting information, past meeting data, reference materials

[0709] Data processing / calculation: Based on this information, the server automatically generates a progress script and uploads related materials to cloud storage.

[0710] Output: Progress script, related materials (uploaded to cloud storage)

[0711] Step 3:

[0712] Meeting start and participant recognition

[0713] At the start of a conference, the device uses a camera to identify participants and sends that data to the server.

[0714] Input: Participant's face data (obtained from camera)

[0715] Data processing / calculation: The device uses facial recognition technology to identify participants and sends data to the server to update their attendance status.

[0716] Output: Updated attendance data (sent to server)

[0717] Step 4:

[0718] Meeting proceedings

[0719] The server sends the generated script to the device, and the device's announcer AI runs the conference based on the script.

[0720] Input: Progression script

[0721] Data processing / calculation: Based on the progress script, the terminal uses an announcer AI to conduct the meeting by voice and display the necessary materials.

[0722] Output: Ongoing voice instructions, display of materials

[0723] Step 5:

[0724] Questions and Answers

[0725] During a meeting, a user asks a question. The device recognizes the question and sends it to the server as text data.

[0726] Input: Participant's question (audio data)

[0727] Data processing / calculation: The terminal converts the voice data into text, and the server receives the text data, searches for relevant information from a database, and generates an answer.

[0728] Output: Text data question (sent to server), answer (text format, sent to terminal)

[0729] Step 6:

[0730] Meeting minutes and review

[0731] The device records the content of discussions during the meeting in real time and sends the data to a server. After the meeting ends, the server organizes the recorded content, categorizes the issues, and proposes solutions.

[0732] Input: Real-time discussion content (audio data)

[0733] Data processing / calculation: The terminal converts the voice data into text, and the server organizes and classifies the content of the discussion based on the text data, generating solutions to the problems.

[0734] Output: Record contents, issue classification, solution method (report to user)

[0735] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[0736] (Application example 1)

[0737] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0738] There is a need for a system in factories that can efficiently issue work instructions and conduct meetings, quickly respond to worker questions, and allow users to review recorded meeting content and work content later. It is also necessary to utilize facial recognition and voice recognition technology to accurately grasp the attendance status of workers and quickly provide necessary information. The present invention aims to provide a system that solves these problems.

[0739] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0740] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and factory workers and updating attendance status, means for generating a conference progress script and preparing and displaying materials to be used during the conference and work instruction conference, means for acquiring questions from participants and workers as text data using voice recognition technology during the conference and during work and generating answers to those questions, means for recording the contents of the conference and work, categorizing the issues discussed, and means for generating and reporting proposals for solving the issues. This allows for smooth progress of conferences and work instruction discussions in the factory and for quick responses to worker questions. Furthermore, the recorded contents of the conference and work can be reviewed later, leading to efficient business improvement and optimization.

[0741] "Meeting setting information" refers to detailed information required to hold a meeting, such as the date, time, location, participants, and agenda of the meeting.

[0742] A "meeting schedule" is a plan of how a meeting will proceed, including the start time, end time, break times, and the order of each agenda item.

[0743] "Facial recognition" is a technology that uses a camera to identify a person's face and identify the person in question.

[0744] "Attendance status" indicates information on those who actually attended a meeting or task and those who were absent.

[0745] A "meeting progress script" is a script for smoothly running a meeting, and includes the procedure for proceeding with each agenda item and the order in which speakers will speak.

[0746] "Materials" refers to information media such as documents, charts, presentation materials, etc. used during meetings and work instruction meetings.

[0747] A "question" is a question or confirmation that a meeting participant or factory worker submits during a meeting or during work instructions.

[0748] An "answer" is an explanation or answer to a question posed.

[0749] "Speech recognition technology" is a technology that analyzes speech and converts it into text data.

[0750] "Recording" refers to the act of saving statements and discussions made during meetings or work instruction meetings as digital data.

[0751] "Issues" are problems or areas for improvement that were discussed during meetings or work instruction meetings.

[0752] "Categorization" is the process of classifying and organizing recorded tasks.

[0753] A "proposal" is a proposal that shows a solution or improvement to a specific problem.

[0754] "Factory workers" are employees or workers who actually perform work in a factory.

[0755] This invention relates to a system that supports efficient progress of meetings and work instructions in factories and problem-solving. This system receives and manages setting information for meetings and work instructions, identifies participants and workers using face recognition technology, prepares progress scripts and materials, provides quick answers to questions from participants and workers, and organizes issues and proposes solutions after meetings and work.

[0756] The system consists of the following main components:

[0757] 1. Server

[0758] The server controls the entire system and is responsible for managing and processing data.

[0759] The server receives the setting information of the meeting and the work instruction, and generates and saves the meeting schedule.

[0760] The server processes face recognition data of meeting and work order participants and updates their attendance status.

[0761] The server generates a progress script and materials and sends them to the terminal.

[0762] The server searches for relevant information in response to questions from participants and workers and generates answers.

[0763] The server records the meeting and work instructions and stores them in a format that can be accessed later.

[0764] 2. Terminal

[0765] The terminal functions as a user interface and receives instructions from the server to proceed with the conference and work instructions.

[0766] The device uses a camera and facial recognition function to identify participants and workers.

[0767] The terminal has the function of assisting the progress based on the progress script and materials received from the server.

[0768] The terminal recognizes the questions of participants and workers through voice recognition and transmits them to the server as text data.

[0769] The terminal displays the answers and materials received from the server to participants in the meeting and at the work instruction site, and provides audio explanations.

[0770] The terminal records discussions during meetings and work instructions in real time and transmits them to the server.

[0771] 3. Users

[0772] The user inputs the setting information for the meeting and work instructions and registers it in the system.

[0773] Users can ask questions and give opinions during meetings and work instructions, and use the system's interface.

[0774] The system uses the following hardware and software:

[0775] Camera: A device for performing facial recognition.

[0776] Microphone: A device for voice recognition.

[0777] Server: A computer that performs data management and processing.

[0778] AI assistant: Software that uses voice recognition technology and document generation, etc.

[0779] Uses the Python libraries: facial_recognition, audio_processing, and document_processing.

[0780] As a concrete example, consider the case of holding a meeting to explain new work procedures. In this case, the robot will start the meeting at a set date and time, recognize the faces of the participants, and take attendance. It will then present materials based on the procedure explanation script and answer safety-related questions in real time. After the meeting, it will automatically generate minutes and provide feedback to the relevant parties.

[0781] Example prompt sentence:

[0782] "Please schedule a meeting to explain the new procedures. The date and time is October 15th at 2 PM, and the participants are the engineering and quality control teams."

[0783] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0784] Step 1:

[0785] The user inputs the setting information for the meeting or work instruction and sends it to the server.

[0786] Input: Meeting or work order setup information such as date, time, location, participants, and agenda.

[0787] Specific operation: The user inputs the configuration information through the interface and clicks the "Submit" button, which sends the input information to the server.

[0788] Output: The server has received the configuration information and is ready to proceed to the next processing step.

[0789] Step 2:

[0790] The server generates a conference schedule based on the received setting information and transmits it to the terminal.

[0791] Input: Meeting and work order setup information received from users.

[0792] Specific operation: The server determines the timetable for each agenda item based on the setting information, generates a meeting schedule, and sends the generated schedule to the terminal.

[0793] Output: Meeting schedule.

[0794] Step 3:

[0795] The server generates a progress script and prepares the materials necessary for meetings and work instructions.

[0796] Input: Meeting schedule and related configuration information.

[0797] Specific operation: The server generates a progress script based on the schedule and stores appropriate materials (documents, diagrams, presentations, etc.) in cloud storage.

[0798] Output: Progress script and materials.

[0799] Step 4:

[0800] The device uses a camera to recognize the faces of participants and workers and sends the results to the server.

[0801] Input: Camera video data.

[0802] Specific operation: Images are captured in real time using the device's built-in camera, and participants and workers are identified using facial recognition technology. The identified facial recognition data is sent to the server.

[0803] Output: Recognized participant and worker data.

[0804] Step 5:

[0805] The server updates attendance status based on the facial recognition data and sends the attendee list to the terminal.

[0806] Input: facial recognition data.

[0807] Specific operation: The server verifies the sent facial recognition data and updates the attendee list, which is then sent to the terminal and displayed on the terminal during the meeting or work order.

[0808] Output: Attendees list.

[0809] Step 6:

[0810] The terminals conduct meetings and work based on the progress script and display materials.

[0811] Input: Progression script and materials.

[0812] Specific operation: The terminal reads out the progress script and displays materials on the screen while proceeding with the meeting and work instructions. It provides necessary information to participants and workers according to the progress instructions.

[0813] Output: Progress of meetings and work instructions.

[0814] Step 7:

[0815] Participants and workers ask questions by voice, and the device recognizes the voice and sends it to the server.

[0816] Input: Participant or worker voice question.

[0817] Specific operation: The device uses a microphone to capture the voice of the question, converts it into text data using voice recognition technology, and sends the converted data to the server.

[0818] Output: Question data converted from speech to text.

[0819] Step 8:

[0820] The server generates an answer to the question and sends it to the terminal.

[0821] Input: Speech-to-text question data.

[0822] Specific operation: The server accesses the database, searches for relevant information, generates an answer, and sends the answer to the terminal.

[0823] Output: Response data.

[0824] Step 9:

[0825] The device displays the answer received from the server and explains it aloud.

[0826] Input: Response data.

[0827] Specific operation: The terminal displays the received response data on the screen and uses voice synthesis technology to communicate the response to the participant or worker aloud.

[0828] Output: Providing answers to participants or workers.

[0829] Step 10:

[0830] The terminal records the contents of meetings and work instructions and sends them to the server.

[0831] Input: Discussions and progress during meetings and work instructions.

[0832] Specific operation: The device records the contents of meetings and work instructions in real time and periodically sends the data to the server.

[0833] Output: Recorded meetings and work.

[0834] Step 11:

[0835] After the meeting or work instruction is completed, the server organizes the recorded content and stores it in a format that can be accessed later.

[0836] Input: Recorded meeting and work data.

[0837] Specific operation: The server analyzes the recorded data, categorizes each issue, generates analysis results and proposals, and stores them in cloud storage or an internal database.

[0838] Output: Saved meetings and work, as well as analysis results and recommendations.

[0839] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0840] This invention combines an emotion engine with a system that supports efficient meeting progress and problem-solving. This system receives and manages meeting setting information, identifies participants using facial recognition technology, prepares meeting scripts and materials, provides quick answers to questions from participants, and organizes issues and proposes solutions after the meeting. It also improves the quality of meetings by recognizing and utilizing participants' emotions in real time.

[0841] (System configuration)

[0842] The system includes the following major components:

[0843] 1. Server

[0844] The server controls the entire system and is responsible for managing and processing data.

[0845] Receives meeting setting information and generates and saves the meeting schedule.

[0846] Processes facial recognition data of meeting participants and updates their attendance status.

[0847] Generate progress scripts and materials and send them to the terminal.

[0848] It searches for relevant information and generates answers to questions from participants.

[0849] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[0850] Uses an emotion engine to analyze and store emotional data from meeting participants' facial expressions and tone of voice.

[0851] 2. Terminal

[0852] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[0853] Meeting participants are identified using a camera and facial recognition function.

[0854] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[0855] Participants' questions are recognized by voice and sent to the server as text data.

[0856] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[0857] Discussions during meetings are recorded in real time and sent to a server.

[0858] Using a camera and microphone, participants' facial expressions and tone of voice are analyzed and emotional data is sent to a server.

[0859] 3. Users

[0860] The user enters the conference setting information and registers it in the system.

[0861] Ask questions and give opinions during the meeting and use the system interface.

[0862] (Specific operation example)

[0863] For example, consider a case where you are holding a kickoff meeting for a new product and want to recognize participants' emotions in real time and reflect them in the progress of the meeting.

[0864] 1. A user schedules a meeting

[0865] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[0866] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[0867] 2. Pre-meeting preparation

[0868] The server creates a meeting script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[0869] The terminal downloads this data from the server and prepares to start the meeting.

[0870] 3. Participant facial and emotion recognition

[0871] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[0872] The server updates attendance status based on facial recognition data and also records information on absentees.

[0873] At the same time, the device uses a camera and microphone to analyze the participants' facial expressions and tone of voice, and transmits the emotional data to the server.

[0874] 4. Meeting management and emotional feedback

[0875] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[0876] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[0877] The server uses an emotion engine to adjust the announcer AI's tone and method of proceeding according to the participant's emotional state. For example, if a participant shows dissatisfaction or confusion, the announcer AI will prompt questions or provide additional explanations.

[0878] 5. Questions and Answers

[0879] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[0880] The device recognizes the question and sends it to the server as text data.

[0881] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[0882] The device displays the answer in document form, and an AI announcer provides the answer aloud.

[0883] 6. Organizing meeting records and emotion data

[0884] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[0885] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[0886] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[0887] 7. Proposal for solving the problem

[0888] The proposed solution also includes feedback based on emotional data.

[0889] For example, you can provide particularly detailed solutions to issues that participants complained about during the meeting.

[0890] The terminal reports the proposed solution to the user and provides it in an exportable format.

[0891] By using this system, it is possible to consistently and efficiently carry out everything from preparing for meetings to conducting them, answering questions, and post-meeting reflection and problem-solving. It also makes it possible to further improve the quality of meetings by utilizing feedback from the emotion engine.

[0892] The processing flow will be explained below.

[0893] Step 1:

[0894] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[0895] Step 2:

[0896] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[0897] Step 3:

[0898] At least several hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, specifications for a new product, etc.) and stores them in cloud storage.

[0899] Step 4:

[0900] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[0901] Step 5:

[0902] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data and attendee information are then sent to the server.

[0903] Step 6:

[0904] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[0905] Step 7:

[0906] The device uses a camera and microphone to analyze the facial expressions and tone of voice of meeting participants, obtain emotional data, and send it to the server.

[0907] Step 8:

[0908] The server analyzes the emotional data using an emotion engine and records the emotional state of the participants.

[0909] Step 9:

[0910] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[0911] Step 10:

[0912] The terminal displays materials based on the progress script, and the announcer AI gives an explanation. The server uses emotional data to adjust the announcer AI's tone and progress according to the participants' reactions.

[0913] Step 11:

[0914] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[0915] Step 12:

[0916] The terminal sends the text data of the question to the server.

[0917] Step 13:

[0918] The server searches the database for relevant information, generates an answer to the question, and sends the generated information and answer to the terminal.

[0919] Step 14:

[0920] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[0921] Step 15:

[0922] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[0923] Step 16:

[0924] After the meeting, the server organizes the recorded data and categorizes the issues discussed.

[0925] Step 17:

[0926] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[0927] Step 18:

[0928] The terminal reports the proposed solution to the user and provides it in an exportable format.

[0929] Example 2

[0930] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0931] Conventional conference systems not only lack efficiency in meeting progress and problem-solving, but also tend to lower the quality of meetings because they cannot take into account the emotions of participants.In addition, there are issues with the overall process being complicated and time-consuming, as it requires a lot of manual work, such as setting up meetings, confirming participants, preparing materials, answering questions, and following up after meetings.

[0932] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0933] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for solving the issues, and means for recognizing the emotions of conference participants in real time and reflecting them in the conference progress. This enables consistent and efficient processing of the conference, from preparation to progress, question and answer session, and post-conference issue organization and solution proposals. Furthermore, recognizing participants' emotions in real time and reflecting them in the conference progress can improve the quality of the conference.

[0934] 1. "Meeting setting information" refers to the detailed information necessary to hold a meeting, such as the date and time of the meeting, participants, location, and agenda.

[0935] 2. "Means for generating a conference schedule" refers to a device or program that has the function of automatically creating and saving a schedule, such as the date and time, location, and participant list of a conference, based on the conference setting information.

[0936] 3. "Means for recognizing the faces of meeting participants and updating attendance status" refers to devices or programs that use cameras and facial recognition technology to identify the faces of meeting participants, confirm attendance, and update logs.

[0937] 4. A "meeting script" is a document or instruction manual that describes the procedures and steps for conducting a meeting, and serves as a guideline for ensuring that the meeting proceeds on time.

[0938] 5. "Materials used during the meeting" refers to information that supports the progress of the meeting, such as data, reports, and presentation materials related to the meeting agenda.

[0939] 6. "Means for accepting questions from participants during the conference and generating answers to those questions" refers to devices or programs that have the function of receiving questions from participants, analyzing their content, and providing appropriate answers.

[0940] 7. "Means for recording meeting content and categorizing the issues discussed" refers to devices or programs that have the function of recording the content of discussions during meetings and categorizing and organizing that content.

[0941] 8. "Means for generating and reporting proposals for solving problems" refers to devices or programs that have the function of analyzing the problems discussed in the meeting, coming up with solutions, and compiling them into documents or reports.

[0942] 9. "Means of recognizing the emotions of meeting participants in real time and reflecting them in the progress of the meeting" refers to devices or programs that use cameras and microphones to analyze the facial expressions and tone of voice of meeting participants and adjust the way the meeting is conducted based on that emotional data.

[0943] The present invention combines an emotion engine with a system that supports efficient meeting progress and problem solving, and includes the following main components:

[0944] System configuration

[0945] This system is composed of server, terminal, and user components, and by clearly defining the roles of each component, efficient conference management is achieved.

[0946] server

[0947] Managing conference settings information

[0948] The server receives the meeting setting information and generates a meeting schedule. For example, specific setting information such as "A kickoff meeting for a new product is scheduled for May 15th at 2:00 PM. Participants include the general manager, project manager, and engineering team" is entered. The meeting schedule is then saved in the database and invitation emails are sent to participants.

[0949] Facial recognition and attendance updates

[0950] The server processes the facial recognition data of the conference participants and updates the attendance status. Specifically, it analyzes the facial recognition data sent from the terminals and updates the attendee list.

[0951] Preparing the script and materials

[0952] The server generates a meeting progress script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage and prepared for transmission to the terminals.

[0953] Q&A and problem identification

[0954] The server accepts questions from participants and searches a database to generate answers to those questions. It also includes a means to record the meeting content and categorize the issues discussed.

[0955] Analysis by emotion engine

[0956] The server analyzes the emotional data sent from the devices and reflects the feedback in the progress script. The emotion engine analyzes the participants' facial expressions and tone of voice and makes adjustments according to the progress of the meeting.

[0957] Terminal

[0958] User Interface

[0959] The device functions as a user interface and receives instructions from the server to conduct the conference. It uses a camera and facial recognition function to recognize the faces of participants and transmits the data to the server.

[0960] Announcer AI

[0961] The AI ​​announcer on the device will conduct the meeting by voice, displaying the necessary materials based on the meeting script, and will recognize the user's questions and send them to the server as text data.

[0962] Real-time recording and analysis of emotion data

[0963] The device records the content of discussions in real time during the meeting and sends it to the server. It also uses a camera and microphone to analyze participants' facial expressions and tone of voice, and sends emotional data to the server.

[0964] User

[0965] Meeting setup and questions

[0966] Users enter meeting setting information and register it in the system. During the meeting, they can ask questions and give opinions and use the system's interface.

[0967] Specific examples

[0968] Consider a case where a kickoff meeting for a new product is being held and participants' emotions are recognized in real time and reflected in the progress of the meeting. The user sets up the meeting, and the server generates a meeting schedule and notifies participants. The device receives the prepared materials and performs facial recognition at the start of the meeting. The server sends instructions to the device based on the progress script, and the device's announcer AI follows these instructions to proceed with the meeting. During the meeting, the user asks a question, and the device recognizes the question through voice recognition and sends it to the server. The server generates an answer, which the device displays and explains in audio. After the meeting ends, the server organizes the recorded data and generates proposals for solving problems based on the emotional data. This improves the efficiency and quality of meetings.

[0969] The advantage of this system is that it can efficiently handle everything from preparing for meetings to conducting them, answering questions, and sorting out issues and proposing solutions after the meeting. Furthermore, it can improve the quality of meetings by utilizing feedback from an emotion engine.

[0970] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0971] Step 1: Receiving meeting settings

[0972] The user enters meeting setting information. For example, the user might enter "New product kickoff meeting scheduled for May 15th at 2:00 PM. Participants will be the general manager, project manager, and engineering team."

[0973] Input: Conference setup information from the user.

[0974] Data processing: The server receives this information and analyzes and organizes the conference setting information.

[0975] Output: Structured conference setup data.

[0976] Specific operation: The server saves the conference setting information in a database and sends invitation emails to participants as needed.

[0977] Step 2: Generate and save the meeting schedule

[0978] The server generates a conference schedule based on the received setting information.

[0979] Input: Conference setup information.

[0980] Data processing: Generate schedules based on meeting dates, times, locations, and participant lists.

[0981] Output: Meeting schedule information.

[0982] Specific operation: The server saves the generated schedule in the database and sends invitation emails to participants.

[0983] Step 3: Prepare the script and materials

[0984] The server creates a progress script for the conference and prepares the necessary materials.

[0985] Input: Meeting setup information and related materials (market research data, new product specifications, etc.).

[0986] Data processing: Organizing and generating procedures and materials.

[0987] Output: Progress script and preparation materials.

[0988] Specific operation: The material is stored in cloud storage and prepared to be sent to the device.

[0989] Step 4: Participant facial and emotion recognition

[0990] The device uses a camera to recognize the faces of participants at the start of the meeting.

[0991] Input: Face images of meeting participants.

[0992] Data processing: Data processing using facial recognition algorithms.

[0993] Output: Participant's facial recognition data.

[0994] Specific operation: The device sends facial recognition data to the server, which then updates the attendance status. At the same time, the device uses the camera and microphone to analyze emotion data and send it to the server.

[0995] Step 5: Conducting the meeting

[0996] The server sends instructions to the terminal based on the progress script.

[0997] Input: Progress script.

[0998] Data processing: Analyzes the instructions and sends them to the terminal.

[0999] Output: Progression instructions data.

[1000] How it works: The AI ​​announcer on the device conducts the meeting by voice while displaying the necessary materials. The emotion engine on the server analyzes the emotions of the participants and sends feedback to the device to adjust the way the meeting is conducted.

[1001] Step 6: Receiving questions and providing answers

[1002] The user (participant) asks a question by voice. For example, "Tell me about your new product launch plan."

[1003] Input: Participant's question audio data.

[1004] Data processing: Performs voice recognition and converts it into text data.

[1005] Output: Question text data.

[1006] How it works: The device recognizes the question and sends it as text data to the server. The server searches for relevant information in a database and generates an answer using a generative AI model. The device displays the answer in a document format, and the announcer AI provides the answer via voice.

[1007] Step 7: Record the meeting and organize the emotional data

[1008] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[1009] Input: Meeting discussion content and participants' emotional data.

[1010] Data processing: Converting discussion content into text and analyzing emotional data.

[1011] Output: Recorded data and emotion data.

[1012] What it does: The server organizes the recorded data and categorizes the issues discussed.

[1013] Step 8: Generate problem-solving proposals

[1014] The server generates problem-solving proposals based on the recorded data.

[1015] Input: Recorded data and emotion data.

[1016] Data processing: Data analysis and generation of proposals using generative AI models.

[1017] Output: A document proposing a solution to the problem.

[1018] How it works: The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[1019] Step 9: Report the proposed solution

[1020] The server documents the generated proposals and reports them to the user.

[1021] Input: A proposal document for solving the problem.

[1022] Data processing: Proposal documentation and format conversion.

[1023] Output: Proposal report.

[1024] Specific operation: The server exports the proposed solution in document format, and the device displays and reports it to the user.

[1025] (Application example 2)

[1026] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1027] Although systems exist to support efficient meeting progress, there is a problem in the industrial sector where communication and discussions between workers are not sufficiently efficient. Furthermore, because there is no technology to analyze emotional data in real time and reflect it in the progress of meetings, it is difficult to respond quickly when participants have complaints or questions. This can lead to a decline in the quality and productivity of meetings and delays in problem-solving.

[1028] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1029] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for resolving the issues, means for analyzing participants' facial expressions and voice data in real time and saving the data as emotion data, and means installed on industrial terminals for adjusting the progress script based on the emotion data during the conference. This makes it possible to improve the efficiency and quality of industrial conferences and to analyze and respond to participants' emotions in real time.

[1030] definition statement

[1031] "Meeting setting information" is information that includes details of the date, time, location, participants and agenda of the meeting.

[1032] The "conference schedule" is a schedule generated based on the setting information of the conference, and includes the date and time of the conference, the order of proceedings on the agenda, and the duration of each agenda item.

[1033] A "conference participant" is someone who is scheduled to attend a conference.

[1034] "Facial recognition" is a technology that identifies a person's face from image data and identifies that person.

[1035] "Attendance status" is information indicating the attendance of conference participants.

[1036] A "meeting progress script" is a script created to support the progress of a meeting, and describes in detail the explanation of each agenda item and the progress procedure.

[1037] "Materials" are files containing documents, presentations, graphs, and other information prepared for use during a meeting.

[1038] "Questions" refer to questions or comments submitted by meeting participants during the meeting.

[1039] "Response" refers to an answer or explanation to a submitted question.

[1040] "Meeting content" is information including the topics discussed, statements, and decisions made during a meeting.

[1041] An "issue" refers to a problem to be discussed or resolved during a meeting.

[1042] "Categorization" refers to classifying the issues discussed into specific categories.

[1043] A "proposal" refers to a specific action plan or measure offered to solve a problem.

[1044] "Facial expression data" is information indicating the emotional state obtained from a person's face.

[1045] "Voice data" refers to acoustic information obtained from a person's voice, and is also used to analyze emotional states.

[1046] "Emotion data" is information indicating an emotional state based on facial expression data and voice data analyzed in real time.

[1047] An "industrial terminal" is a computer terminal used in factories and manufacturing sites, and is typically a rugged, durable piece of equipment.

[1048] "Adjusting the progress script" means changing the meeting progress script based on real-time emotional data and responding according to the participants' situations.

[1049] MODE FOR CARRYING OUT THE INVENTION

[1050] This invention is built around a conference support system installed on industrial terminals, and provides consistent support from receiving conference setting information to proposing solutions to problems after the conference. The system is composed of multiple components, including a server, terminals, and an emotion analysis engine.

[1051] The server operates as follows:

[1052] 1. Receives conference setting information and generates and saves a conference schedule. The schedule is created based on the conference date, time, location, participant information, etc. set by the user.

[1053] 2. Uses facial recognition technology to identify meeting participants and update their attendance status. The server manages this information and updates the meeting attendee list in real time.

[1054] 3. Generate a meeting script, prepare materials to be used during the meeting, save them in cloud storage, and then download these materials to the device.

[1055] 4. During the conference, participants can ask questions, search for answers in the database, and generate answers. Questions are converted into text using speech recognition technology.

[1056] 5. Record the meeting and categorize the issues discussed. The recorded data will be organized and categorized for later analysis.

[1057] 6. The emotion engine analyzes participants' facial expressions and voice data in real time and stores them as emotion data. This information is used to adjust the script as the meeting progresses.

[1058] 7. After the meeting, a proposal for solving the problem is generated based on the recorded meeting content and emotional data, and reported to the device. This proposal is provided in an accessible format for later use.

[1059] The terminal operates as follows:

[1060] 1. Prepare for the meeting based on the meeting schedule, progress script, and materials downloaded from the server.

[1061] 2. Use face recognition to identify participants and send their attendance status to the server. Use a camera and face recognition technology to identify and confirm participants' faces.

[1062] 3. During the meeting, the server displays the progress script and presents materials to attendees. It also recognizes participants' questions through a microphone and sends them to the server.

[1063] 4. The AI ​​announcer provides audible support for the meeting, adjusting the script as needed based on emotional data. If participants show confusion or dissatisfaction, the AI ​​announcer will provide additional explanations to help participants understand.

[1064] 5. The contents of the meeting discussion are recorded in real time and sent to the server along with emotion data.

[1065] The user uses it as follows:

[1066] 1. Enter and register the meeting setting information on the server. For example, you could set a new product kickoff meeting for May 15th at 2:00 PM, with the department head, project manager, and engineering team as participants.

[1067] 2. Ask questions or make comments during the meeting and send them to the system. For example, "Tell me about your plans to bring new products to market."

[1068] 3. After the meeting, the solution to the problem proposed by the server is confirmed and implemented.

[1069] The main hardware and software used are as follows:

[1070] Face recognition technology: OpenCV

[1071] Speech recognition technology: Google Speech-to-Text API

[1072] Emotion Recognition Engine: TensorFlow-based model

[1073] Meeting Script Generation: Custom Script Generation Tool

[1074] Announcer AI: Voice generation using TTS (Text-to-Speech) (e.g., pyttsx3)

[1075] The following prompt sentences are concrete examples of practical applications of this system:

[1076] "There are five participants in the meeting. Please show the script and provide additional explanations when participants show signs of dissatisfaction."

[1077] In this way, this invention supports the conduct of meetings within factories and helps achieve more effective and efficient communication by analyzing participants' emotions in real time.

[1078] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1079] Processing flow

[1080] Step 1: Receiving conference setting information and generating a conference schedule

[1081] The server receives meeting setting information (date and time, location, participant information, etc.) entered by the user. It then generates a meeting schedule based on this information and stores it in a database. The generated meeting schedule includes the date and time of the meeting, the order of agenda items, and the duration of each agenda item.

[1082] Input: Conference setup information from the user

[1083] Data processing / calculation: Analysis and scheduling of setting information

[1084] Output: Meeting schedule

[1085] Step 2: Face recognition and attendance updates for meeting participants

[1086] At the start of a meeting, the device uses a camera to capture the faces of participants and sends the data to the server, which then uses facial recognition technology to identify participants and update their attendance status, which is then saved in a database in real time.

[1087] Input: Facial image data from the camera

[1088] Data processing / calculation: Facial recognition analysis, attendance status update

[1089] Output: Attendees list

[1090] Step 3: Prepare the meeting script and materials

[1091] The server generates a progress script based on the meeting setting information and past meeting data, and saves the materials to be used during the meeting in cloud storage. The server then sends the progress script and materials to the device, which then downloads them and prepares for the meeting.

[1092] Input: Meeting setting information, past meeting data

[1093] Data processing / calculation: Script generation and document creation

[1094] Output: Progress script, meeting materials

[1095] Step 4: Real-time emotion recognition and script adjustment

[1096] During the meeting, the device uses a camera and microphone to analyze participants' facial expressions and voices and transmits the data to a server. The server then uses a generative AI model (such as TensorFlow) to analyze the emotional data in real time and adjust the meeting script as needed. For example, if a participant shows dissatisfaction or confusion, the server will change the meeting script to provide additional explanation.

[1097] Input: facial expression and voice data from camera and microphone

[1098] Data processing / calculation: Emotion analysis, script adjustment

[1099] Output: Adjusted progress script

[1100] Step 5: Speech recognition of questions and answer generation

[1101] When a participant asks a question during a meeting, the device captures the voice through the microphone and converts it into text data using the Google Speech-to-Text API. This text data is sent to a server, which then searches a database for relevant information, generates an appropriate answer, and sends it to the device. The device then uses announcer AI to provide the answer via voice.

[1102] Input: Audio data from microphone (question)

[1103] Data processing / calculation: speech recognition, database search, answer generation

[1104] Output: Answer to question (text and audio)

[1105] Step 6: Record meeting content and emotional data

[1106] The device records what is said and the key points of discussion during the meeting in real time and sends the data to a server. The server also stores participants' emotional data. The server then organizes this data and provides it in a format that can be accessed later.

[1107] Input: Contents of remarks made during meetings, emotional data

[1108] Data processing / calculation: Data recording and organization

[1109] Output: Recorded meeting content, organized emotion data

[1110] Step 7: Generate problem-solving proposals

[1111] After the meeting, the server analyzes the recorded content and emotional data, categorizes the issues discussed, and then proposes specific solutions for each issue and sends them to the device. These proposals are then provided in a format that can be accessed later.

[1112] Input: Recorded meeting content, emotion data

[1113] Data processing / calculation: Data analysis, categorization, and proposal generation

[1114] Output: Problem-solving proposal

[1115] The implementation of this system will enable meetings within factories to proceed efficiently and effectively, and will achieve better communication by reflecting the emotions of participants in real time.

[1116] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1117] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1118] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1119] [Third embodiment]

[1120] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1121] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1122] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1123] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1124] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1125] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1126] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1127] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1128] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1129] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1130] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1131] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1132] This invention relates to a system that supports efficient progress of meetings and problem-solving. This system receives and manages meeting setting information, identifies participants using face recognition technology, prepares meeting scripts and materials, provides prompt answers to questions from participants, and organizes issues and proposes solutions after the meeting.

[1133] (System configuration)

[1134] The system includes the following major components:

[1135] 1. Server

[1136] The server controls the entire system and is responsible for managing and processing data.

[1137] Receives meeting setting information and generates and saves the meeting schedule.

[1138] Processes facial recognition data of meeting participants and updates their attendance status.

[1139] Generate progress scripts and materials and send them to the terminal.

[1140] It searches for relevant information and generates answers to questions from participants.

[1141] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[1142] 2. Terminal

[1143] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[1144] Meeting participants are identified using a camera and facial recognition function.

[1145] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[1146] Participants' questions are recognized by voice and sent to the server as text data.

[1147] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[1148] Discussions during meetings are recorded in real time and sent to a server.

[1149] 3. Users

[1150] The user enters the conference setting information and registers it in the system.

[1151] Ask questions and give opinions during the meeting and use the system interface.

[1152] (Specific operation example)

[1153] As an example, consider holding a kickoff meeting for a new product.

[1154] 1. A user schedules a meeting

[1155] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[1156] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[1157] 2. Pre-meeting preparation

[1158] The server creates a meeting script and prepares and stores materials related to the agenda (e.g., market research data, new product specifications, etc.) in cloud storage.

[1159] The terminal downloads this data from the server and prepares to start the meeting.

[1160] 3. Participant facial recognition

[1161] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[1162] The server updates attendance status based on facial recognition data and also records information on absentees.

[1163] 4. Meeting Management

[1164] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[1165] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[1166] 5. Questions and Answers

[1167] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[1168] The device recognizes the question and sends it to the server as text data.

[1169] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[1170] The device displays the answer in document format, and an AI announcer explains it aloud.

[1171] 6. Recording and reviewing the meeting

[1172] The device records the content of discussions during the meeting in real time and sends it to the server.

[1173] After the meeting, the server organizes the recordings and categorizes each topic.

[1174] The server performs the analysis, proposes solutions to each problem, and reports them to the user via the terminal.

[1175] Using this system, you can consistently and efficiently prepare for meetings, conduct them, answer questions, and review and resolve issues after the meeting.

[1176] The processing flow will be explained below.

[1177] Step 1:

[1178] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[1179] Step 2:

[1180] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[1181] Step 3:

[1182] At least a few hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[1183] Step 4:

[1184] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[1185] Step 5:

[1186] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data is then sent to the server.

[1187] Step 6:

[1188] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[1189] Step 7:

[1190] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[1191] Step 8:

[1192] The terminal displays materials based on the progress script, and an AI announcer provides explanations.

[1193] Step 9:

[1194] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[1195] Step 10:

[1196] The terminal sends the text data of the question to the server.

[1197] Step 11:

[1198] The server searches the database for relevant information and generates answers to the questions, which are then sent to the device.

[1199] Step 12:

[1200] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[1201] Step 13:

[1202] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[1203] Step 14:

[1204] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[1205] Step 15:

[1206] The server performs a detailed analysis of each problem and generates a solution proposal, including an action plan and predicted outcomes.

[1207] Step 16:

[1208] The terminal reports the proposed solution to the user and provides it in an exportable format.

[1209] Step 17:

[1210] The user receives the report and develops an action plan as needed.

[1211] Example 1

[1212] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1213] Conventional meeting management has the problem of being difficult to manage consistently from setting up the meeting to proceeding, answering questions, and reviewing and resolving issues after the meeting. This has led to problems such as the inability to efficiently organize the information necessary for the meeting, confirm attendance, record the meeting content in real time, and process the post-meeting information.

[1214] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1215] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for recognizing characteristics of conference participants and updating attendance status, means for generating a conference progress script and preparing information to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and classifying the issues discussed, and means for generating and reporting proposals for solving the issues. This makes it possible to efficiently and consistently manage the entire conference process, and quickly organize information, confirm attendance, record in real time, and organize issues and propose solutions after the conference.

[1216] "Meeting setting information" refers to basic information required to hold a meeting, such as the date and time of the meeting, location, participants, and agenda.

[1217] "Means for generating a meeting schedule" refers to a function that automatically creates and saves specific meeting schedules based on received meeting setting information.

[1218] "Feature recognition of meeting participants" refers to technology that uses facial recognition technology to identify meeting participants and confirm their attendance status.

[1219] "Means for updating attendance status" refers to a function that updates the attendee list in real time based on the results of feature recognition of meeting participants and manages attendance and absence status.

[1220] "Means for generating a meeting progress script" refers to a function that automatically creates a series of steps that compile the instructions and information necessary to progress the meeting.

[1221] "Means for preparing information to be used during a meeting" refers to the function of collecting and organizing necessary materials and data based on the meeting progress script and making them available during the meeting.

[1222] "Means for accepting questions and generating answers to those questions" refers to a function for receiving questions submitted by participants during a meeting and generating answers to address those questions immediately.

[1223] "A means of recording meeting content and classifying the issues discussed" refers to a function that records what is said and discussed during a meeting in real time, and then organizes and classifies the content by issue.

[1224] "Means for generating and reporting proposals for resolving issues" refers to the function of analyzing the recorded meeting content, proposing specific solutions to identified issues, and reporting them to the relevant parties.

[1225] The system of the present invention is designed to support efficient progress of meetings and problem-solving. This system is composed of a server, terminals, and users.

[1226] server

[1227] The server controls the entire system and is responsible for managing and processing data. Specifically, the server receives meeting setting information from users, generates a meeting schedule, stores it in a database, and then automatically sends invitation emails to each participant.

[1228] The server uses facial recognition technology to identify meeting participants and update their attendance status.The server uses a built-in database system (e.g., MySQL) and cloud storage (e.g., Amazon S3) to generate and store the meeting script and related materials.

[1229] During the meeting, the system accepts questions from participants, searches for relevant information in its database, and generates answers. The answers are sent to the terminal in real time. It also records the contents of the meeting, categorizes the issues after the meeting, and generates and reports proposed solutions.

[1230] Terminal

[1231] The device functions as a user interface and receives instructions from the server to proceed with the conference. The device is equipped with a camera and facial recognition function, which identifies participants at the start of the conference and sends that data to the server.

[1232] The AI ​​announcer on the device will conduct the meeting based on the script and materials received from the server, outputting instructions such as "First, I'll give an overview of the new product."

[1233] During the meeting, the device recognizes participants' voice questions and sends them as text data to the server. Answers and materials received from the server are displayed on the device's screen, and an AI announcer provides audible explanations.

[1234] The device records the meeting contents in real time and sends the data to a server, allowing participants to review the meeting contents later and use them as reference material for problem-solving.

[1235] User

[1236] The user enters meeting setting information and registers it in the system. For example, the user enters information such as "The kickoff meeting for the new product is scheduled for May 15th at 2:00 PM."

[1237] During the meeting, users can ask questions and give opinions and use the system's interface to help the meeting proceed.

[1238] Specific examples

[1239] As an example, consider the case of holding a kickoff meeting for a new product. The user sets up the meeting, and the server generates a schedule and notifies participants. At the start of the meeting, the device performs facial recognition and reports attendance status to the server. The server sends a progress script to the device, and an announcer AI leads the meeting. Questions from participants are sent to the server via the device, and answers are instantly generated and displayed. After the meeting ends, the server organizes the recorded content, categorizes issues, and proposes solutions.

[1240] Example prompts for generative AI models

[1241] "Please explain in detail the process for generating the meeting script and related materials."

[1242] "Please explain the specific flow of questions and responses during a meeting."

[1243] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[1244] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1245] Step 1:

[1246] Enter and register meeting setting information

[1247] The user inputs the meeting setting information (e.g., the name of the meeting, the date and time, the location, the participant list, and the details of the agenda) and registers it in the system. The input data is sent to the server.

[1248] Input: Meeting settings information (meeting name, date, time, location, participant list, agenda details)

[1249] Data processing / calculation: The server receives the input information, generates a meeting schedule and participant list, stores them in a database, and generates and sends invitation emails to each participant.

[1250] Output: Meeting schedule, participant list, invitation email

[1251] Step 2:

[1252] Preparing the script and materials

[1253] The server generates a progress script and related materials (e.g., market research data, specifications for new products, etc.) required for the conference.

[1254] Input: Meeting setting information, past meeting data, reference materials

[1255] Data processing / calculation: Based on this information, the server automatically generates a progress script and uploads related materials to cloud storage.

[1256] Output: Progress script, related materials (uploaded to cloud storage)

[1257] Step 3:

[1258] Meeting start and participant recognition

[1259] At the start of a conference, the device uses a camera to identify participants and sends that data to the server.

[1260] Input: Participant's face data (obtained from camera)

[1261] Data processing / calculation: The device uses facial recognition technology to identify participants and sends data to the server to update their attendance status.

[1262] Output: Updated attendance data (sent to server)

[1263] Step 4:

[1264] Meeting proceedings

[1265] The server sends the generated script to the device, and the device's announcer AI runs the conference based on the script.

[1266] Input: Progression script

[1267] Data processing / calculation: Based on the progress script, the terminal uses an announcer AI to conduct the meeting by voice and display the necessary materials.

[1268] Output: Ongoing voice instructions, display of materials

[1269] Step 5:

[1270] Questions and Answers

[1271] During a meeting, a user asks a question. The device recognizes the question and sends it to the server as text data.

[1272] Input: Participant's question (audio data)

[1273] Data processing / calculation: The terminal converts the voice data into text, and the server receives the text data, searches for relevant information from a database, and generates an answer.

[1274] Output: Text data question (sent to server), answer (text format, sent to terminal)

[1275] Step 6:

[1276] Meeting minutes and review

[1277] The device records the content of discussions during the meeting in real time and sends the data to a server. After the meeting ends, the server organizes the recorded content, categorizes the issues, and proposes solutions.

[1278] Input: Real-time discussion content (audio data)

[1279] Data processing / calculation: The terminal converts the voice data into text, and the server organizes and classifies the content of the discussion based on the text data, generating solutions to the problems.

[1280] Output: Record contents, issue classification, solution method (report to user)

[1281] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[1282] (Application example 1)

[1283] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1284] There is a need for a system in factories that can efficiently issue work instructions and conduct meetings, quickly respond to worker questions, and allow users to review recorded meeting content and work content later. It is also necessary to utilize facial recognition and voice recognition technology to accurately grasp the attendance status of workers and quickly provide necessary information. The present invention aims to provide a system that solves these problems.

[1285] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1286] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and factory workers and updating attendance status, means for generating a conference progress script and preparing and displaying materials to be used during the conference and work instruction conference, means for acquiring questions from participants and workers as text data using voice recognition technology during the conference and during work and generating answers to those questions, means for recording the contents of the conference and work, categorizing the issues discussed, and means for generating and reporting proposals for solving the issues. This allows for smooth progress of conferences and work instruction discussions in the factory and for quick responses to worker questions. Furthermore, the recorded contents of the conference and work can be reviewed later, leading to efficient business improvement and optimization.

[1287] "Meeting setting information" refers to detailed information required to hold a meeting, such as the date, time, location, participants, and agenda of the meeting.

[1288] A "meeting schedule" is a plan of how a meeting will proceed, including the start time, end time, break times, and the order of each agenda item.

[1289] "Facial recognition" is a technology that uses a camera to identify a person's face and identify the person in question.

[1290] "Attendance status" indicates information on those who actually attended a meeting or task and those who were absent.

[1291] A "meeting progress script" is a script for smoothly running a meeting, and includes the procedure for proceeding with each agenda item and the order in which speakers will speak.

[1292] "Materials" refers to information media such as documents, charts, presentation materials, etc. used during meetings and work instruction meetings.

[1293] A "question" is a question or confirmation that a meeting participant or factory worker submits during a meeting or during work instructions.

[1294] An "answer" is an explanation or answer to a question posed.

[1295] "Speech recognition technology" is a technology that analyzes speech and converts it into text data.

[1296] "Recording" refers to the act of saving statements and discussions made during meetings or work instruction meetings as digital data.

[1297] "Issues" are problems or areas for improvement that were discussed during meetings or work instruction meetings.

[1298] "Categorization" is the process of classifying and organizing recorded tasks.

[1299] A "proposal" is a proposal that shows a solution or improvement to a specific problem.

[1300] "Factory workers" are employees or workers who actually perform work in a factory.

[1301] This invention relates to a system that supports efficient progress of meetings and work instructions in factories and problem-solving. This system receives and manages setting information for meetings and work instructions, identifies participants and workers using face recognition technology, prepares progress scripts and materials, provides quick answers to questions from participants and workers, and organizes issues and proposes solutions after meetings and work.

[1302] The system consists of the following main components:

[1303] 1. Server

[1304] The server controls the entire system and is responsible for managing and processing data.

[1305] The server receives the setting information of the meeting and the work instruction, and generates and saves the meeting schedule.

[1306] The server processes face recognition data of meeting and work order participants and updates their attendance status.

[1307] The server generates a progress script and materials and sends them to the terminal.

[1308] The server searches for relevant information in response to questions from participants and workers and generates answers.

[1309] The server records the meeting and work instructions and stores them in a format that can be accessed later.

[1310] 2. Terminal

[1311] The terminal functions as a user interface and receives instructions from the server to proceed with the conference and work instructions.

[1312] The device uses a camera and facial recognition function to identify participants and workers.

[1313] The terminal has the function of assisting the progress based on the progress script and materials received from the server.

[1314] The terminal recognizes the questions of participants and workers through voice recognition and transmits them to the server as text data.

[1315] The terminal displays the answers and materials received from the server to participants in the meeting and at the work instruction site, and provides audio explanations.

[1316] The terminal records discussions during meetings and work instructions in real time and transmits them to the server.

[1317] 3. Users

[1318] The user inputs the setting information for the meeting and work instructions and registers it in the system.

[1319] Users can ask questions and give opinions during meetings and work instructions, and use the system's interface.

[1320] The system uses the following hardware and software:

[1321] Camera: A device for performing facial recognition.

[1322] Microphone: A device for voice recognition.

[1323] Server: A computer that performs data management and processing.

[1324] AI assistant: Software that uses voice recognition technology and document generation, etc.

[1325] Uses the Python libraries: facial_recognition, audio_processing, and document_processing.

[1326] As a concrete example, consider the case of holding a meeting to explain new work procedures. In this case, the robot will start the meeting at a set date and time, recognize the faces of the participants, and take attendance. It will then present materials based on the procedure explanation script and answer safety-related questions in real time. After the meeting, it will automatically generate minutes and provide feedback to the relevant parties.

[1327] Example prompt sentence:

[1328] "Please schedule a meeting to explain the new procedures. The date and time is October 15th at 2 PM, and the participants are the engineering and quality control teams."

[1329] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1330] Step 1:

[1331] The user inputs the setting information for the meeting or work instruction and sends it to the server.

[1332] Input: Meeting or work order setup information such as date, time, location, participants, and agenda.

[1333] Specific operation: The user inputs the configuration information through the interface and clicks the "Submit" button, which sends the input information to the server.

[1334] Output: The server has received the configuration information and is ready to proceed to the next processing step.

[1335] Step 2:

[1336] The server generates a conference schedule based on the received setting information and transmits it to the terminal.

[1337] Input: Meeting and work order setup information received from users.

[1338] Specific operation: The server determines the timetable for each agenda item based on the setting information, generates a meeting schedule, and sends the generated schedule to the terminal.

[1339] Output: Meeting schedule.

[1340] Step 3:

[1341] The server generates a progress script and prepares the materials necessary for meetings and work instructions.

[1342] Input: Meeting schedule and related configuration information.

[1343] Specific operation: The server generates a progress script based on the schedule and stores appropriate materials (documents, diagrams, presentations, etc.) in cloud storage.

[1344] Output: Progress script and materials.

[1345] Step 4:

[1346] The device uses a camera to recognize the faces of participants and workers and sends the results to the server.

[1347] Input: Camera video data.

[1348] Specific operation: Images are captured in real time using the device's built-in camera, and participants and workers are identified using facial recognition technology. The identified facial recognition data is sent to the server.

[1349] Output: Recognized participant and worker data.

[1350] Step 5:

[1351] The server updates attendance status based on the facial recognition data and sends the attendee list to the terminal.

[1352] Input: facial recognition data.

[1353] Specific operation: The server verifies the sent facial recognition data and updates the attendee list, which is then sent to the terminal and displayed on the terminal during the meeting or work order.

[1354] Output: Attendees list.

[1355] Step 6:

[1356] The terminals conduct meetings and work based on the progress script and display materials.

[1357] Input: Progression script and materials.

[1358] Specific operation: The terminal reads out the progress script and displays materials on the screen while proceeding with the meeting and work instructions. It provides necessary information to participants and workers according to the progress instructions.

[1359] Output: Progress of meetings and work instructions.

[1360] Step 7:

[1361] Participants and workers ask questions by voice, and the device recognizes the voice and sends it to the server.

[1362] Input: Participant or worker voice question.

[1363] Specific operation: The device uses a microphone to capture the voice of the question, converts it into text data using voice recognition technology, and sends the converted data to the server.

[1364] Output: Question data converted from speech to text.

[1365] Step 8:

[1366] The server generates an answer to the question and sends it to the terminal.

[1367] Input: Speech-to-text question data.

[1368] Specific operation: The server accesses the database, searches for relevant information, generates an answer, and sends the answer to the terminal.

[1369] Output: Response data.

[1370] Step 9:

[1371] The device displays the answer received from the server and explains it aloud.

[1372] Input: Response data.

[1373] Specific operation: The terminal displays the received response data on the screen and uses voice synthesis technology to communicate the response to the participant or worker aloud.

[1374] Output: Providing answers to participants or workers.

[1375] Step 10:

[1376] The terminal records the contents of meetings and work instructions and sends them to the server.

[1377] Input: Discussions and progress during meetings and work instructions.

[1378] Specific operation: The device records the contents of meetings and work instructions in real time and periodically sends the data to the server.

[1379] Output: Recorded meetings and work.

[1380] Step 11:

[1381] After the meeting or work instruction is completed, the server organizes the recorded content and stores it in a format that can be accessed later.

[1382] Input: Recorded meeting and work data.

[1383] Specific operation: The server analyzes the recorded data, categorizes each issue, generates analysis results and proposals, and stores them in cloud storage or an internal database.

[1384] Output: Saved meetings and work, as well as analysis results and recommendations.

[1385] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1386] This invention combines an emotion engine with a system that supports efficient meeting progress and problem-solving. This system receives and manages meeting setting information, identifies participants using facial recognition technology, prepares meeting scripts and materials, provides quick answers to questions from participants, and organizes issues and proposes solutions after the meeting. It also improves the quality of meetings by recognizing and utilizing participants' emotions in real time.

[1387] (System configuration)

[1388] The system includes the following major components:

[1389] 1. Server

[1390] The server controls the entire system and is responsible for managing and processing data.

[1391] Receives meeting setting information and generates and saves the meeting schedule.

[1392] Processes facial recognition data of meeting participants and updates their attendance status.

[1393] Generate progress scripts and materials and send them to the terminal.

[1394] It searches for relevant information and generates answers to questions from participants.

[1395] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[1396] Uses an emotion engine to analyze and store emotional data from meeting participants' facial expressions and tone of voice.

[1397] 2. Terminal

[1398] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[1399] Meeting participants are identified using a camera and facial recognition function.

[1400] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[1401] Participants' questions are recognized by voice and sent to the server as text data.

[1402] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[1403] Discussions during meetings are recorded in real time and sent to a server.

[1404] Using a camera and microphone, participants' facial expressions and tone of voice are analyzed and emotional data is sent to a server.

[1405] 3. Users

[1406] The user enters the conference setting information and registers it in the system.

[1407] Ask questions and give opinions during the meeting and use the system interface.

[1408] (Specific operation example)

[1409] For example, consider a case where you are holding a kickoff meeting for a new product and want to recognize participants' emotions in real time and reflect them in the progress of the meeting.

[1410] 1. A user schedules a meeting

[1411] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[1412] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[1413] 2. Pre-meeting preparation

[1414] The server creates a meeting script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[1415] The terminal downloads this data from the server and prepares to start the meeting.

[1416] 3. Participant facial and emotion recognition

[1417] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[1418] The server updates attendance status based on facial recognition data and also records information on absentees.

[1419] At the same time, the device uses a camera and microphone to analyze the participants' facial expressions and tone of voice, and transmits the emotional data to the server.

[1420] 4. Meeting management and emotional feedback

[1421] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[1422] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[1423] The server uses an emotion engine to adjust the announcer AI's tone and method of proceeding according to the participant's emotional state. For example, if a participant shows dissatisfaction or confusion, the announcer AI will prompt questions or provide additional explanations.

[1424] 5. Questions and Answers

[1425] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[1426] The device recognizes the question and sends it to the server as text data.

[1427] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[1428] The device displays the answer in document form, and an AI announcer provides the answer aloud.

[1429] 6. Organizing meeting records and emotion data

[1430] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[1431] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[1432] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[1433] 7. Proposal for solving the problem

[1434] The proposed solution also includes feedback based on emotional data.

[1435] For example, you can provide particularly detailed solutions to issues that participants complained about during the meeting.

[1436] The terminal reports the proposed solution to the user and provides it in an exportable format.

[1437] By using this system, it is possible to consistently and efficiently carry out everything from preparing for meetings to conducting them, answering questions, and post-meeting reflection and problem-solving. It also makes it possible to further improve the quality of meetings by utilizing feedback from an emotion engine.

[1438] The processing flow will be explained below.

[1439] Step 1:

[1440] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[1441] Step 2:

[1442] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[1443] Step 3:

[1444] At least several hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, specifications for a new product, etc.) and stores them in cloud storage.

[1445] Step 4:

[1446] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[1447] Step 5:

[1448] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data and attendee information are then sent to the server.

[1449] Step 6:

[1450] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[1451] Step 7:

[1452] The device uses a camera and microphone to analyze the facial expressions and tone of voice of meeting participants, obtain emotional data, and send it to the server.

[1453] Step 8:

[1454] The server analyzes the emotional data using an emotion engine and records the emotional state of the participants.

[1455] Step 9:

[1456] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[1457] Step 10:

[1458] The terminal displays materials based on the progress script, and the announcer AI gives an explanation. The server uses emotional data to adjust the announcer AI's tone and progress according to the participants' reactions.

[1459] Step 11:

[1460] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[1461] Step 12:

[1462] The terminal sends the text data of the question to the server.

[1463] Step 13:

[1464] The server searches the database for relevant information, generates an answer to the question, and sends the generated information and answer to the terminal.

[1465] Step 14:

[1466] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[1467] Step 15:

[1468] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[1469] Step 16:

[1470] After the meeting, the server organizes the recorded data and categorizes the issues discussed.

[1471] Step 17:

[1472] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[1473] Step 18:

[1474] The terminal reports the proposed solution to the user and provides it in an exportable format.

[1475] Example 2

[1476] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1477] Conventional conference systems not only lack efficiency in meeting progress and problem-solving, but also tend to lower the quality of meetings because they cannot take into account the emotions of participants.In addition, there are issues with the overall process being complicated and time-consuming, as it requires a lot of manual work, such as setting up meetings, confirming participants, preparing materials, answering questions, and following up after meetings.

[1478] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1479] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for solving the issues, and means for recognizing the emotions of conference participants in real time and reflecting them in the conference progress. This enables consistent and efficient processing of the conference, from preparation to progress, question and answer session, and post-conference issue organization and solution proposals. Furthermore, recognizing participants' emotions in real time and reflecting them in the conference progress can improve the quality of the conference.

[1480] 1. "Meeting setting information" refers to the detailed information necessary to hold a meeting, such as the date and time of the meeting, participants, location, and agenda.

[1481] 2. "Means for generating a conference schedule" refers to a device or program that has the function of automatically creating and saving a schedule, such as the date and time, location, and participant list of a conference, based on the conference setting information.

[1482] 3. "Means for recognizing the faces of meeting participants and updating attendance status" refers to devices or programs that use cameras and facial recognition technology to identify the faces of meeting participants, confirm attendance, and update logs.

[1483] 4. A "meeting script" is a document or instruction manual that describes the procedures and steps for conducting a meeting, and serves as a guideline for ensuring that the meeting proceeds on time.

[1484] 5. "Materials used during the meeting" refers to information that supports the progress of the meeting, such as data, reports, and presentation materials related to the meeting agenda.

[1485] 6. "Means for accepting questions from participants during the conference and generating answers to those questions" refers to devices or programs that have the function of receiving questions from participants, analyzing their content, and providing appropriate answers.

[1486] 7. "Means for recording meeting content and categorizing the issues discussed" refers to devices or programs that have the function of recording the content of discussions during meetings and categorizing and organizing that content.

[1487] 8. "Means for generating and reporting proposals for solving problems" refers to devices or programs that have the function of analyzing the problems discussed in the meeting, coming up with solutions, and compiling them into documents or reports.

[1488] 9. "Means of recognizing the emotions of meeting participants in real time and reflecting them in the progress of the meeting" refers to devices or programs that use cameras and microphones to analyze the facial expressions and tone of voice of meeting participants and adjust the way the meeting is conducted based on that emotional data.

[1489] The present invention combines an emotion engine with a system that supports efficient meeting progress and problem solving, and includes the following main components:

[1490] System configuration

[1491] This system is composed of server, terminal, and user components, and by clearly defining the roles of each component, efficient conference management is achieved.

[1492] server

[1493] Managing conference settings information

[1494] The server receives the meeting setting information and generates a meeting schedule. For example, specific setting information such as "A kickoff meeting for a new product is scheduled for May 15th at 2:00 PM. Participants include the general manager, project manager, and engineering team" is entered. The meeting schedule is then saved in the database and invitation emails are sent to participants.

[1495] Facial recognition and attendance updates

[1496] The server processes the facial recognition data of the conference participants and updates the attendance status. Specifically, it analyzes the facial recognition data sent from the terminals and updates the attendee list.

[1497] Preparing the script and materials

[1498] The server generates a meeting progress script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage and prepared for transmission to the terminals.

[1499] Q&A and problem identification

[1500] The server accepts questions from participants and searches a database to generate answers to those questions. It also includes a means to record the meeting content and categorize the issues discussed.

[1501] Analysis by emotion engine

[1502] The server analyzes the emotional data sent from the devices and reflects the feedback in the progress script. The emotion engine analyzes the participants' facial expressions and tone of voice and makes adjustments according to the progress of the meeting.

[1503] Terminal

[1504] User Interface

[1505] The device functions as a user interface and receives instructions from the server to conduct the conference. It uses a camera and facial recognition function to recognize the faces of participants and transmits the data to the server.

[1506] Announcer AI

[1507] The AI ​​announcer on the device will conduct the meeting by voice, displaying the necessary materials based on the meeting script, and will recognize the user's questions and send them to the server as text data.

[1508] Real-time recording and analysis of emotion data

[1509] The device records the content of discussions in real time during the meeting and sends it to the server. It also uses a camera and microphone to analyze participants' facial expressions and tone of voice, and sends emotional data to the server.

[1510] User

[1511] Meeting setup and questions

[1512] Users enter meeting setting information and register it in the system. During the meeting, they can ask questions and give opinions and use the system's interface.

[1513] Specific examples

[1514] Consider a case where a kickoff meeting for a new product is being held and participants' emotions are recognized in real time and reflected in the progress of the meeting. The user sets up the meeting, and the server generates a meeting schedule and notifies participants. The device receives the prepared materials and performs facial recognition at the start of the meeting. The server sends instructions to the device based on the progress script, and the device's announcer AI follows these instructions to proceed with the meeting. During the meeting, the user asks a question, and the device recognizes the question through voice recognition and sends it to the server. The server generates an answer, which the device displays and explains in audio. After the meeting ends, the server organizes the recorded data and generates proposals for solving problems based on the emotional data. This improves the efficiency and quality of meetings.

[1515] The advantage of this system is that it can efficiently handle everything from preparing for meetings to conducting them, answering questions, and sorting out issues and proposing solutions after the meeting. Furthermore, it can improve the quality of meetings by utilizing feedback from an emotion engine.

[1516] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1517] Step 1: Receiving meeting settings

[1518] The user enters meeting setting information. For example, the user might enter "New product kickoff meeting scheduled for May 15th at 2:00 PM. Participants will be the general manager, project manager, and engineering team."

[1519] Input: Conference setup information from the user.

[1520] Data processing: The server receives this information and analyzes and organizes the conference setting information.

[1521] Output: Structured conference setup data.

[1522] Specific operation: The server saves the conference setting information in a database and sends invitation emails to participants as needed.

[1523] Step 2: Generate and save the meeting schedule

[1524] The server generates a conference schedule based on the received setting information.

[1525] Input: Conference setup information.

[1526] Data processing: Generate schedules based on meeting dates, times, locations, and participant lists.

[1527] Output: Meeting schedule information.

[1528] Specific operation: The server saves the generated schedule in the database and sends invitation emails to participants.

[1529] Step 3: Prepare the script and materials

[1530] The server creates a progress script for the conference and prepares the necessary materials.

[1531] Input: Meeting setup information and related materials (market research data, new product specifications, etc.).

[1532] Data processing: Organizing and generating procedures and materials.

[1533] Output: Progress script and preparation materials.

[1534] Specific operation: The material is stored in cloud storage and prepared to be sent to the device.

[1535] Step 4: Participant facial and emotion recognition

[1536] The device uses a camera to recognize the faces of participants at the start of the meeting.

[1537] Input: Face images of meeting participants.

[1538] Data processing: Data processing using facial recognition algorithms.

[1539] Output: Participant's facial recognition data.

[1540] Specific operation: The device sends facial recognition data to the server, which then updates the attendance status. At the same time, the device uses the camera and microphone to analyze emotion data and send it to the server.

[1541] Step 5: Conducting the meeting

[1542] The server sends instructions to the terminal based on the progress script.

[1543] Input: Progress script.

[1544] Data processing: Analyzes the instructions and sends them to the terminal.

[1545] Output: Progression instructions data.

[1546] How it works: The AI ​​announcer on the device conducts the meeting by voice while displaying the necessary materials. The emotion engine on the server analyzes the emotions of the participants and sends feedback to the device to adjust the way the meeting is conducted.

[1547] Step 6: Receiving questions and providing answers

[1548] The user (participant) asks a question by voice. For example, "Tell me about your new product launch plan."

[1549] Input: Participant's question audio data.

[1550] Data processing: Performs voice recognition and converts it into text data.

[1551] Output: Question text data.

[1552] How it works: The device recognizes the question and sends it as text data to the server. The server searches for relevant information in a database and generates an answer using a generative AI model. The device displays the answer in a document format, and the announcer AI provides the answer via voice.

[1553] Step 7: Record the meeting and organize the emotional data

[1554] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[1555] Input: Meeting discussion content and participants' emotional data.

[1556] Data processing: Converting discussion content into text and analyzing emotional data.

[1557] Output: Recorded data and emotion data.

[1558] What it does: The server organizes the recorded data and categorizes the issues discussed.

[1559] Step 8: Generate problem-solving proposals

[1560] The server generates problem-solving proposals based on the recorded data.

[1561] Input: Recorded data and emotion data.

[1562] Data processing: Data analysis and generation of proposals using generative AI models.

[1563] Output: A document proposing a solution to the problem.

[1564] How it works: The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[1565] Step 9: Report the proposed solution

[1566] The server documents the generated proposals and reports them to the user.

[1567] Input: A proposal document for solving the problem.

[1568] Data processing: Proposal documentation and format conversion.

[1569] Output: Proposal report.

[1570] Specific operation: The server exports the proposed solution in document format, and the device displays and reports it to the user.

[1571] (Application example 2)

[1572] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1573] Although systems exist to support efficient meeting progress, there is a problem in the industrial sector where communication and discussions between workers are not sufficiently efficient. Furthermore, because there is no technology to analyze emotional data in real time and reflect it in the progress of meetings, it is difficult to respond quickly when participants have complaints or questions. This can lead to a decline in the quality and productivity of meetings and delays in problem-solving.

[1574] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1575] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for resolving the issues, means for analyzing participants' facial expressions and voice data in real time and saving the data as emotion data, and means installed on industrial terminals for adjusting the progress script based on the emotion data during the conference. This makes it possible to improve the efficiency and quality of industrial conferences and to analyze and respond to participants' emotions in real time.

[1576] definition statement

[1577] "Meeting setting information" is information that includes details of the date, time, location, participants and agenda of the meeting.

[1578] The "conference schedule" is a schedule generated based on the setting information of the conference, and includes the date and time of the conference, the order of proceedings on the agenda, and the duration of each agenda item.

[1579] A "conference participant" is someone who is scheduled to attend a conference.

[1580] "Facial recognition" is a technology that identifies a person's face from image data and identifies that person.

[1581] "Attendance status" is information indicating the attendance of conference participants.

[1582] A "meeting progress script" is a script created to support the progress of a meeting, and describes in detail the explanation of each agenda item and the progress procedure.

[1583] "Materials" are files containing documents, presentations, graphs, and other information prepared for use during a meeting.

[1584] "Questions" refer to questions or comments submitted by meeting participants during the meeting.

[1585] "Response" refers to an answer or explanation to a submitted question.

[1586] "Meeting content" is information including the topics discussed, statements, and decisions made during a meeting.

[1587] An "issue" refers to a problem to be discussed or resolved during a meeting.

[1588] "Categorization" refers to classifying the issues discussed into specific categories.

[1589] A "proposal" refers to a specific action plan or measure offered to solve a problem.

[1590] "Facial expression data" is information indicating the emotional state obtained from a person's face.

[1591] "Voice data" refers to acoustic information obtained from a person's voice, and is also used to analyze emotional states.

[1592] "Emotion data" is information indicating an emotional state based on facial expression data and voice data analyzed in real time.

[1593] An "industrial terminal" is a computer terminal used in factories and manufacturing sites, and is typically a rugged, durable piece of equipment.

[1594] "Adjusting the progress script" means changing the meeting progress script based on real-time emotional data and responding according to the participants' situations.

[1595] MODE FOR CARRYING OUT THE INVENTION

[1596] This invention is built around a conference support system installed on industrial terminals, and provides consistent support from receiving conference setting information to proposing solutions to problems after the conference. The system is composed of multiple components, including a server, terminals, and an emotion analysis engine.

[1597] The server operates as follows:

[1598] 1. Receives conference setting information and generates and saves a conference schedule. The schedule is created based on the conference date, time, location, participant information, etc. set by the user.

[1599] 2. Uses facial recognition technology to identify meeting participants and update their attendance status. The server manages this information and updates the meeting attendee list in real time.

[1600] 3. Generate a meeting script, prepare materials to be used during the meeting, save them in cloud storage, and then download these materials to the device.

[1601] 4. During the conference, participants can ask questions, search for answers in the database, and generate answers. Questions are converted into text using speech recognition technology.

[1602] 5. Record the meeting and categorize the issues discussed. The recorded data will be organized and categorized for later analysis.

[1603] 6. The emotion engine analyzes participants' facial expressions and voice data in real time and stores them as emotion data. This information is used to adjust the script as the meeting progresses.

[1604] 7. After the meeting, a proposal for solving the problem is generated based on the recorded meeting content and emotional data, and reported to the device. This proposal is provided in an accessible format for later use.

[1605] The terminal operates as follows:

[1606] 1. Prepare for the meeting based on the meeting schedule, progress script, and materials downloaded from the server.

[1607] 2. Use face recognition to identify participants and send their attendance status to the server. Use a camera and face recognition technology to identify and confirm participants' faces.

[1608] 3. During the meeting, the server displays the progress script and presents materials to attendees. It also recognizes participants' questions through a microphone and sends them to the server.

[1609] 4. The AI ​​announcer provides audible support for the meeting, adjusting the script as needed based on emotional data. If participants show confusion or dissatisfaction, the AI ​​announcer will provide additional explanations to help participants understand.

[1610] 5. The contents of the meeting discussion are recorded in real time and sent to the server along with emotion data.

[1611] The user uses it as follows:

[1612] 1. Enter and register the meeting setting information on the server. For example, you could set a new product kickoff meeting for May 15th at 2:00 PM, with the department head, project manager, and engineering team as participants.

[1613] 2. Ask questions or make comments during the meeting and send them to the system. For example, "Tell me about your plans to bring new products to market."

[1614] 3. After the meeting, the solution to the problem proposed by the server is confirmed and implemented.

[1615] The main hardware and software used are as follows:

[1616] Face recognition technology: OpenCV

[1617] Speech recognition technology: Google Speech-to-Text API

[1618] Emotion Recognition Engine: TensorFlow-based model

[1619] Meeting Script Generation: Custom Script Generation Tool

[1620] Announcer AI: Voice generation using TTS (Text-to-Speech) (e.g., pyttsx3)

[1621] The following prompt sentences are concrete examples of practical applications of this system:

[1622] "There are five participants in the meeting. Please show the script and provide additional explanations when participants show signs of dissatisfaction."

[1623] In this way, this invention supports the conduct of meetings within factories and helps achieve more effective and efficient communication by analyzing participants' emotions in real time.

[1624] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1625] Processing flow

[1626] Step 1: Receiving conference setting information and generating a conference schedule

[1627] The server receives meeting setting information (date and time, location, participant information, etc.) entered by the user. It then generates a meeting schedule based on this information and stores it in a database. The generated meeting schedule includes the date and time of the meeting, the order of agenda items, and the duration of each agenda item.

[1628] Input: Conference setup information from the user

[1629] Data processing / calculation: Analysis and scheduling of setting information

[1630] Output: Meeting schedule

[1631] Step 2: Face recognition and attendance updates for meeting participants

[1632] At the start of a meeting, the device uses a camera to capture the faces of participants and sends the data to the server, which then uses facial recognition technology to identify participants and update their attendance status, which is then saved in a database in real time.

[1633] Input: Facial image data from the camera

[1634] Data processing / calculation: Facial recognition analysis, attendance status update

[1635] Output: Attendees list

[1636] Step 3: Prepare the meeting script and materials

[1637] The server generates a progress script based on the meeting setting information and past meeting data, and saves the materials to be used during the meeting in cloud storage. The server then sends the progress script and materials to the device, which then downloads them and prepares for the meeting.

[1638] Input: Meeting setting information, past meeting data

[1639] Data processing / calculation: Script generation and document creation

[1640] Output: Progress script, meeting materials

[1641] Step 4: Real-time emotion recognition and script adjustment

[1642] During the meeting, the device uses a camera and microphone to analyze participants' facial expressions and voices and transmits the data to a server. The server then uses a generative AI model (such as TensorFlow) to analyze the emotional data in real time and adjust the meeting script as needed. For example, if a participant shows dissatisfaction or confusion, the server will change the meeting script to provide additional explanation.

[1643] Input: facial expression and voice data from camera and microphone

[1644] Data processing / calculation: Emotion analysis, script adjustment

[1645] Output: Adjusted progress script

[1646] Step 5: Speech recognition of questions and answer generation

[1647] When a participant asks a question during a meeting, the device captures the voice through the microphone and converts it into text data using the Google Speech-to-Text API. This text data is sent to a server, which then searches a database for relevant information, generates an appropriate answer, and sends it to the device. The device then uses announcer AI to provide the answer via voice.

[1648] Input: Audio data from microphone (question)

[1649] Data processing / calculation: speech recognition, database search, answer generation

[1650] Output: Answer to question (text and audio)

[1651] Step 6: Record meeting content and emotional data

[1652] The device records what is said and the key points of discussion during the meeting in real time and sends the data to a server. The server also stores participants' emotional data. The server then organizes this data and provides it in a format that can be accessed later.

[1653] Input: Contents of remarks made during meetings, emotional data

[1654] Data processing / calculation: Data recording and organization

[1655] Output: Recorded meeting content, organized emotion data

[1656] Step 7: Generate problem-solving proposals

[1657] After the meeting, the server analyzes the recorded content and emotional data, categorizes the issues discussed, and then proposes specific solutions for each issue and sends them to the device. These proposals are then provided in a format that can be accessed later.

[1658] Input: Recorded meeting content, emotion data

[1659] Data processing / calculation: Data analysis, categorization, and proposal generation

[1660] Output: Problem-solving proposal

[1661] The implementation of this system will enable meetings within factories to proceed efficiently and effectively, and will achieve better communication by reflecting the emotions of participants in real time.

[1662] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1663] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1664] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1665] [Fourth embodiment]

[1666] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1667] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1668] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1669] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1670] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1671] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1672] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1673] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1674] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1675] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1676] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1677] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1678] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1679] This invention relates to a system that supports efficient progress of meetings and problem-solving. This system receives and manages meeting setting information, identifies participants using face recognition technology, prepares meeting scripts and materials, provides prompt answers to questions from participants, and organizes issues and proposes solutions after the meeting.

[1680] (System configuration)

[1681] The system includes the following major components:

[1682] 1. Server

[1683] The server controls the entire system and is responsible for managing and processing data.

[1684] Receives meeting setting information and generates and saves the meeting schedule.

[1685] Processes facial recognition data of meeting participants and updates their attendance status.

[1686] Generate progress scripts and materials and send them to the terminal.

[1687] It searches for relevant information and generates answers to questions from participants.

[1688] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[1689] 2. Terminal

[1690] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[1691] Meeting participants are identified using a camera and facial recognition function.

[1692] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[1693] Participants' questions are recognized by voice and sent to the server as text data.

[1694] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[1695] Discussions during meetings are recorded in real time and sent to a server.

[1696] 3. Users

[1697] The user enters the conference setting information and registers it in the system.

[1698] Ask questions and give opinions during the meeting and use the system interface.

[1699] (Specific operation example)

[1700] As an example, consider holding a kickoff meeting for a new product.

[1701] 1. A user schedules a meeting

[1702] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[1703] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[1704] 2. Pre-meeting preparation

[1705] The server creates a meeting script and prepares and stores materials related to the agenda (e.g., market research data, new product specifications, etc.) in cloud storage.

[1706] The terminal downloads this data from the server and prepares to start the meeting.

[1707] 3. Participant facial recognition

[1708] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[1709] The server updates attendance status based on facial recognition data and also records information on absentees.

[1710] 4. Meeting Management

[1711] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[1712] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[1713] 5. Questions and Answers

[1714] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[1715] The device recognizes the question and sends it to the server as text data.

[1716] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[1717] The device displays the answer in document format, and an AI announcer explains it aloud.

[1718] 6. Recording and reviewing the meeting

[1719] The device records the content of discussions during the meeting in real time and sends it to the server.

[1720] After the meeting, the server organizes the recordings and categorizes each topic.

[1721] The server performs the analysis, proposes solutions to each problem, and reports them to the user via the terminal.

[1722] Using this system, you can consistently and efficiently prepare for meetings, conduct them, answer questions, and review and resolve issues after the meeting.

[1723] The processing flow will be explained below.

[1724] Step 1:

[1725] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[1726] Step 2:

[1727] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[1728] Step 3:

[1729] At least a few hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[1730] Step 4:

[1731] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[1732] Step 5:

[1733] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data is then sent to the server.

[1734] Step 6:

[1735] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[1736] Step 7:

[1737] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[1738] Step 8:

[1739] The terminal displays materials based on the progress script, and an AI announcer provides explanations.

[1740] Step 9:

[1741] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[1742] Step 10:

[1743] The terminal sends the text data of the question to the server.

[1744] Step 11:

[1745] The server searches the database for relevant information and generates answers to the questions, which are then sent to the device.

[1746] Step 12:

[1747] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[1748] Step 13:

[1749] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[1750] Step 14:

[1751] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[1752] Step 15:

[1753] The server performs a detailed analysis of each problem and generates a solution proposal, including an action plan and predicted outcomes.

[1754] Step 16:

[1755] The terminal reports the proposed solution to the user and provides it in an exportable format.

[1756] Step 17:

[1757] The user receives the report and develops an action plan as needed.

[1758] Example 1

[1759] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1760] Conventional meeting management has the problem of being difficult to manage consistently from setting up the meeting to proceeding, answering questions, and reviewing and resolving issues after the meeting. This has led to problems such as the inability to efficiently organize the information necessary for the meeting, confirm attendance, record the meeting content in real time, and process the post-meeting information.

[1761] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1762] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for recognizing characteristics of conference participants and updating attendance status, means for generating a conference progress script and preparing information to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and classifying the issues discussed, and means for generating and reporting proposals for solving the issues. This makes it possible to efficiently and consistently manage the entire conference process, and quickly organize information, confirm attendance, record in real time, and organize issues and propose solutions after the conference.

[1763] "Meeting setting information" refers to basic information required to hold a meeting, such as the date and time of the meeting, location, participants, and agenda.

[1764] "Means for generating a meeting schedule" refers to a function that automatically creates and saves specific meeting schedules based on received meeting setting information.

[1765] "Feature recognition of meeting participants" refers to technology that uses facial recognition technology to identify meeting participants and confirm their attendance status.

[1766] "Means for updating attendance status" refers to a function that updates the attendee list in real time based on the results of feature recognition of meeting participants and manages attendance and absence status.

[1767] "Means for generating a meeting progress script" refers to a function that automatically creates a series of steps that compile the instructions and information necessary to progress the meeting.

[1768] "Means for preparing information to be used during a meeting" refers to the function of collecting and organizing necessary materials and data based on the meeting progress script and making them available during the meeting.

[1769] "Means for accepting questions and generating answers to those questions" refers to a function for receiving questions submitted by participants during a meeting and generating answers to address those questions immediately.

[1770] "A means of recording meeting content and classifying the issues discussed" refers to a function that records what is said and discussed during a meeting in real time, and then organizes and classifies the content by issue.

[1771] "Means for generating and reporting proposals for resolving issues" refers to the function of analyzing the recorded meeting content, proposing specific solutions to identified issues, and reporting them to the relevant parties.

[1772] The system of the present invention is designed to support efficient progress of meetings and problem-solving. This system is composed of a server, terminals, and users.

[1773] server

[1774] The server controls the entire system and is responsible for managing and processing data. Specifically, the server receives meeting setting information from users, generates a meeting schedule, stores it in a database, and then automatically sends invitation emails to each participant.

[1775] The server uses facial recognition technology to identify meeting participants and update their attendance status.The server uses a built-in database system (e.g., MySQL) and cloud storage (e.g., Amazon S3) to generate and store the meeting script and related materials.

[1776] During the meeting, the system accepts questions from participants, searches for relevant information in its database, and generates answers. The answers are sent to the terminal in real time. It also records the contents of the meeting, categorizes the issues after the meeting, and generates and reports proposed solutions.

[1777] Terminal

[1778] The device functions as a user interface and receives instructions from the server to proceed with the conference. The device is equipped with a camera and facial recognition function, which identifies participants at the start of the conference and sends that data to the server.

[1779] The AI ​​announcer on the device will conduct the meeting based on the script and materials received from the server, outputting instructions such as "First, I'll give an overview of the new product."

[1780] During the meeting, the device recognizes participants' voice questions and sends them as text data to the server. Answers and materials received from the server are displayed on the device's screen, and an AI announcer provides audible explanations.

[1781] The device records the meeting contents in real time and sends the data to a server, allowing participants to review the meeting contents later and use them as reference material for problem-solving.

[1782] User

[1783] The user enters meeting setting information and registers it in the system. For example, the user enters information such as "The kickoff meeting for the new product is scheduled for May 15th at 2:00 PM."

[1784] During the meeting, users can ask questions and give opinions and use the system's interface to help the meeting proceed.

[1785] Specific examples

[1786] As an example, consider the case of holding a kickoff meeting for a new product. The user sets up the meeting, and the server generates a schedule and notifies participants. At the start of the meeting, the device performs facial recognition and reports attendance status to the server. The server sends a progress script to the device, and an announcer AI leads the meeting. Questions from participants are sent to the server via the device, and answers are instantly generated and displayed. After the meeting ends, the server organizes the recorded content, categorizes issues, and proposes solutions.

[1787] Example prompts for generative AI models

[1788] "Please explain in detail the process for generating the meeting script and related materials."

[1789] "Please explain the specific flow of questions and responses during a meeting."

[1790] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[1791] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1792] Step 1:

[1793] Enter and register meeting setting information

[1794] The user inputs the meeting setting information (e.g., the name of the meeting, the date and time, the location, the participant list, and the details of the agenda) and registers it in the system. The input data is sent to the server.

[1795] Input: Meeting settings information (meeting name, date, time, location, participant list, agenda details)

[1796] Data processing / calculation: The server receives the input information, generates a meeting schedule and participant list, stores them in a database, and generates and sends invitation emails to each participant.

[1797] Output: Meeting schedule, participant list, invitation email

[1798] Step 2:

[1799] Preparing the script and materials

[1800] The server generates a progress script and related materials (e.g., market research data, specifications for new products, etc.) required for the conference.

[1801] Input: Meeting setting information, past meeting data, reference materials

[1802] Data processing / calculation: Based on this information, the server automatically generates a progress script and uploads related materials to cloud storage.

[1803] Output: Progress script, related materials (uploaded to cloud storage)

[1804] Step 3:

[1805] Meeting start and participant recognition

[1806] At the start of a conference, the device uses a camera to identify participants and sends that data to the server.

[1807] Input: Participant's face data (obtained from camera)

[1808] Data processing / calculation: The device uses facial recognition technology to identify participants and sends data to the server to update their attendance status.

[1809] Output: Updated attendance data (sent to server)

[1810] Step 4:

[1811] Meeting proceedings

[1812] The server sends the generated script to the device, and the device's announcer AI runs the conference based on the script.

[1813] Input: Progression script

[1814] Data processing / calculation: Based on the progress script, the terminal uses an announcer AI to conduct the meeting by voice and display the necessary materials.

[1815] Output: Ongoing voice instructions, display of materials

[1816] Step 5:

[1817] Questions and Answers

[1818] During a meeting, a user asks a question. The device recognizes the question and sends it to the server as text data.

[1819] Input: Participant's question (audio data)

[1820] Data processing / calculation: The terminal converts the voice data into text, and the server receives the text data, searches for relevant information from a database, and generates an answer.

[1821] Output: Text data question (sent to server), answer (text format, sent to terminal)

[1822] Step 6:

[1823] Meeting minutes and review

[1824] The device records the content of discussions during the meeting in real time and sends the data to a server. After the meeting ends, the server organizes the recorded content, categorizes the issues, and proposes solutions.

[1825] Input: Real-time discussion content (audio data)

[1826] Data processing / calculation: The terminal converts the voice data into text, and the server organizes and classifies the content of the discussion based on the text data, generating solutions to the problems.

[1827] Output: Record contents, issue classification, solution method (report to user)

[1828] This system allows for consistent and efficient preparation, progress, Q&A, post-meeting review, and problem-solving.

[1829] (Application example 1)

[1830] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1831] There is a need for a system in factories that can efficiently issue work instructions and conduct meetings, quickly respond to worker questions, and allow users to review recorded meeting content and work content later. It is also necessary to utilize facial recognition and voice recognition technology to accurately grasp the attendance status of workers and quickly provide necessary information. The present invention aims to provide a system that solves these problems.

[1832] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1833] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and factory workers and updating attendance status, means for generating a conference progress script and preparing and displaying materials to be used during the conference and work instruction conference, means for acquiring questions from participants and workers as text data using voice recognition technology during the conference and during work and generating answers to those questions, means for recording the contents of the conference and work, categorizing the issues discussed, and means for generating and reporting proposals for solving the issues. This allows for smooth progress of conferences and work instruction discussions in the factory and for quick responses to worker questions. Furthermore, the recorded contents of the conference and work can be reviewed later, leading to efficient business improvement and optimization.

[1834] "Meeting setting information" refers to detailed information required to hold a meeting, such as the date, time, location, participants, and agenda of the meeting.

[1835] A "meeting schedule" is a plan of how a meeting will proceed, including the start time, end time, break times, and the order of each agenda item.

[1836] "Facial recognition" is a technology that uses a camera to identify a person's face and identify the person in question.

[1837] "Attendance status" indicates information on those who actually attended a meeting or task and those who were absent.

[1838] A "meeting progress script" is a script for smoothly running a meeting, and includes the procedure for proceeding with each agenda item and the order in which speakers will speak.

[1839] "Materials" refers to information media such as documents, charts, presentation materials, etc. used during meetings and work instruction meetings.

[1840] A "question" is a question or confirmation that a meeting participant or factory worker submits during a meeting or during work instructions.

[1841] An "answer" is an explanation or answer to a question posed.

[1842] "Speech recognition technology" is a technology that analyzes speech and converts it into text data.

[1843] "Recording" refers to the act of saving statements and discussions made during meetings or work instruction meetings as digital data.

[1844] "Issues" are problems or areas for improvement that were discussed during meetings or work instruction meetings.

[1845] "Categorization" is the process of classifying and organizing recorded tasks.

[1846] A "proposal" is a proposal that shows a solution or improvement to a specific problem.

[1847] "Factory workers" are employees or workers who actually perform work in a factory.

[1848] This invention relates to a system that supports the efficient progress of meetings and work instructions in factories and problem-solving. This system receives and manages setting information for meetings and work instructions, identifies participants and workers using face recognition technology, prepares progress scripts and materials, provides quick answers to questions from participants and workers, and organizes issues and proposes solutions after meetings and work.

[1849] The system consists of the following main components:

[1850] 1. Server

[1851] The server controls the entire system and is responsible for managing and processing data.

[1852] The server receives the setting information of the meeting and the work instruction, and generates and saves the meeting schedule.

[1853] The server processes face recognition data of meeting and work order participants and updates their attendance status.

[1854] The server generates a progress script and materials and sends them to the terminal.

[1855] The server searches for relevant information in response to questions from participants and workers and generates answers.

[1856] The server records the meeting and work instructions and stores them in a format that can be accessed later.

[1857] 2. Terminal

[1858] The terminal functions as a user interface and receives instructions from the server to proceed with the conference and work instructions.

[1859] The device uses a camera and facial recognition function to identify participants and workers.

[1860] The terminal has the function of assisting the progress based on the progress script and materials received from the server.

[1861] The terminal recognizes the questions of participants and workers through voice recognition and transmits them to the server as text data.

[1862] The terminal displays the answers and materials received from the server to participants in the meeting and at the work instruction site, and provides audio explanations.

[1863] The terminal records discussions during meetings and work instructions in real time and transmits them to the server.

[1864] 3. Users

[1865] The user inputs the setting information for the meeting and work instructions and registers it in the system.

[1866] Users can ask questions and give opinions during meetings and work instructions, and use the system's interface.

[1867] The system uses the following hardware and software:

[1868] Camera: A device for performing facial recognition.

[1869] Microphone: A device for voice recognition.

[1870] Server: A computer that performs data management and processing.

[1871] AI assistant: Software that uses voice recognition technology and document generation, etc.

[1872] Uses the Python libraries: facial_recognition, audio_processing, and document_processing.

[1873] As a concrete example, consider the case of holding a meeting to explain new work procedures. In this case, the robot will start the meeting at a set date and time, recognize the faces of the participants, and take attendance. It will then present materials based on the procedure explanation script and answer safety-related questions in real time. After the meeting, it will automatically generate minutes and provide feedback to the relevant parties.

[1874] Example prompt sentence:

[1875] "Please schedule a meeting to explain the new procedures. The date and time is October 15th at 2 PM, and the participants are the engineering and quality control teams."

[1876] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1877] Step 1:

[1878] The user inputs the setting information for the meeting or work instruction and sends it to the server.

[1879] Input: Meeting or work order setup information such as date, time, location, participants, and agenda.

[1880] Specific operation: The user inputs the configuration information through the interface and clicks the "Submit" button, which sends the input information to the server.

[1881] Output: The server has received the configuration information and is ready to proceed to the next processing step.

[1882] Step 2:

[1883] The server generates a conference schedule based on the received setting information and transmits it to the terminal.

[1884] Input: Meeting and work order setup information received from users.

[1885] Specific operation: The server determines the timetable for each agenda item based on the setting information, generates a meeting schedule, and sends the generated schedule to the terminal.

[1886] Output: Meeting schedule.

[1887] Step 3:

[1888] The server generates a progress script and prepares the materials necessary for meetings and work instructions.

[1889] Input: Meeting schedule and related configuration information.

[1890] Specific operation: The server generates a progress script based on the schedule and stores appropriate materials (documents, diagrams, presentations, etc.) in cloud storage.

[1891] Output: Progress script and materials.

[1892] Step 4:

[1893] The device uses a camera to recognize the faces of participants and workers and sends the results to the server.

[1894] Input: Camera video data.

[1895] Specific operation: Images are captured in real time using the device's built-in camera, and participants and workers are identified using facial recognition technology. The identified facial recognition data is sent to the server.

[1896] Output: Recognized participant and worker data.

[1897] Step 5:

[1898] The server updates attendance status based on the facial recognition data and sends the attendee list to the terminal.

[1899] Input: facial recognition data.

[1900] Specific operation: The server verifies the sent facial recognition data and updates the attendee list, which is then sent to the terminal and displayed on the terminal during the meeting or work order.

[1901] Output: Attendees list.

[1902] Step 6:

[1903] The terminals conduct meetings and work based on the progress script and display materials.

[1904] Input: Progression script and materials.

[1905] Specific operation: The terminal reads out the progress script and displays materials on the screen while proceeding with the meeting and work instructions. It provides necessary information to participants and workers according to the progress instructions.

[1906] Output: Progress of meetings and work instructions.

[1907] Step 7:

[1908] Participants and workers ask questions by voice, and the device recognizes the voice and sends it to the server.

[1909] Input: Participant or worker voice question.

[1910] Specific operation: The device uses a microphone to capture the voice of the question, converts it into text data using voice recognition technology, and sends the converted data to the server.

[1911] Output: Question data converted from speech to text.

[1912] Step 8:

[1913] The server generates an answer to the question and sends it to the terminal.

[1914] Input: Speech-to-text question data.

[1915] Specific operation: The server accesses the database, searches for relevant information, generates an answer, and sends the answer to the terminal.

[1916] Output: Response data.

[1917] Step 9:

[1918] The device displays the answer received from the server and explains it aloud.

[1919] Input: Response data.

[1920] Specific operation: The terminal displays the received response data on the screen and uses voice synthesis technology to communicate the response to the participant or worker aloud.

[1921] Output: Providing answers to participants or workers.

[1922] Step 10:

[1923] The terminal records the contents of meetings and work instructions and sends them to the server.

[1924] Input: Discussions and progress during meetings and work instructions.

[1925] Specific operation: The device records the contents of meetings and work instructions in real time and periodically sends the data to the server.

[1926] Output: Recorded meetings and work.

[1927] Step 11:

[1928] After the meeting or work instruction is completed, the server organizes the recorded content and stores it in a format that can be accessed later.

[1929] Input: Recorded meeting and work data.

[1930] Specific operation: The server analyzes the recorded data, categorizes each issue, generates analysis results and proposals, and stores them in cloud storage or an internal database.

[1931] Output: Saved meetings and work, as well as analysis results and recommendations.

[1932] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1933] This invention combines an emotion engine with a system that supports efficient meeting progress and problem-solving. This system receives and manages meeting setting information, identifies participants using facial recognition technology, prepares meeting scripts and materials, provides quick answers to questions from participants, and organizes issues and proposes solutions after the meeting. It also improves the quality of meetings by recognizing and utilizing participants' emotions in real time.

[1934] (System configuration)

[1935] The system includes the following major components:

[1936] 1. Server

[1937] The server controls the entire system and is responsible for managing and processing data.

[1938] Receives meeting setting information and generates and saves the meeting schedule.

[1939] Processes facial recognition data of meeting participants and updates their attendance status.

[1940] Generate progress scripts and materials and send them to the terminal.

[1941] It searches for relevant information and generates answers to questions from participants.

[1942] Record the meeting contents, categorize the issues after the meeting, and generate solution proposals.

[1943] Uses an emotion engine to analyze and store emotional data from meeting participants' facial expressions and tone of voice.

[1944] 2. Terminal

[1945] The terminal functions as a user interface and receives instructions from the server to proceed with the conference.

[1946] Meeting participants are identified using a camera and facial recognition function.

[1947] The announcer AI will conduct the meeting based on the progress script and materials received from the server.

[1948] Participants' questions are recognized by voice and sent to the server as text data.

[1949] Answers and materials received from the server are displayed to the meeting participants and explained aloud.

[1950] Discussions during meetings are recorded in real time and sent to a server.

[1951] Using a camera and microphone, participants' facial expressions and tone of voice are analyzed and emotional data is sent to a server.

[1952] 3. Users

[1953] The user enters the conference setting information and registers it in the system.

[1954] Ask questions and give opinions during the meeting and use the system interface.

[1955] (Specific operation example)

[1956] For example, consider a case where you are holding a kickoff meeting for a new product and want to recognize participants' emotions in real time and reflect them in the progress of the meeting.

[1957] 1. A user schedules a meeting

[1958] A user inputs configuration information to the server, such as "A kickoff meeting for a new product is scheduled for 2:00 p.m. on May 15th. Participants will be the department manager, project manager, and engineering team."

[1959] The server receives this information, generates a meeting schedule, stores it in a database, and then sends invitation emails to participants.

[1960] 2. Pre-meeting preparation

[1961] The server creates a meeting script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage.

[1962] The terminal downloads this data from the server and prepares to start the meeting.

[1963] 3. Participant facial and emotion recognition

[1964] When the conference starts, the terminal uses the camera to recognize the faces of the participants and sends the data to the server.

[1965] The server updates attendance status based on facial recognition data and also records information on absentees.

[1966] At the same time, the device uses a camera and microphone to analyze the participants' facial expressions and tone of voice, and transmits the emotional data to the server.

[1967] 4. Meeting management and emotional feedback

[1968] Based on the progress script generated by the server, a message such as "First, we will give you an overview of our new product" is sent to the terminal.

[1969] The device's AI announcer will conduct the meeting via voice while displaying the necessary materials.

[1970] The server uses an emotion engine to adjust the announcer AI's tone and method of proceeding according to the participant's emotional state. For example, if a participant shows dissatisfaction or confusion, the announcer AI will prompt questions or provide additional explanations.

[1971] 5. Questions and Answers

[1972] During a meeting, a user (participant) asks, "Tell me about your plans to bring new products to market."

[1973] The device recognizes the question and sends it to the server as text data.

[1974] The server searches for relevant information from its accumulated database, generates a response, and sends it to the terminal.

[1975] The device displays the answer in document form, and an AI announcer provides the answer aloud.

[1976] 6. Organizing meeting records and emotion data

[1977] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[1978] After the meeting, the server organizes the recorded data and categorizes the issues discussed into categories such as finance or marketing.

[1979] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[1980] 7. Proposal for solving the problem

[1981] The proposed solution also includes feedback based on emotional data.

[1982] For example, you can provide particularly detailed solutions to issues that participants complained about during the meeting.

[1983] The terminal reports the proposed solution to the user and provides it in an exportable format.

[1984] By using this system, it is possible to consistently and efficiently carry out everything from preparing for meetings to conducting them, answering questions, and post-meeting reflection and problem-solving. It also makes it possible to further improve the quality of meetings by utilizing feedback from an emotion engine.

[1985] The processing flow will be explained below.

[1986] Step 1:

[1987] The user enters the meeting setting information (date and time, participant list, agenda, etc.) and sends it to the server.

[1988] Step 2:

[1989] The server stores the received conference setting information in a database, generates a conference schedule, and simultaneously sends invitation emails to participants.

[1990] Step 3:

[1991] At least several hours before the start of the meeting, the server generates a progress script and prepares materials related to the meeting agenda (e.g., market research data, specifications for a new product, etc.) and stores them in cloud storage.

[1992] Step 4:

[1993] The terminal downloads the progress script and related materials from the server and prepares to start the meeting.

[1994] Step 5:

[1995] When a meeting starts, the device activates the camera and uses the facial recognition function to recognize the faces of participants. The acquired facial data and attendee information are then sent to the server.

[1996] Step 6:

[1997] The server matches the facial data and updates the participant list, recording the information of attendees and absentees in a database.

[1998] Step 7:

[1999] The device uses a camera and microphone to analyze the facial expressions and tone of voice of meeting participants, obtain emotional data, and send it to the server.

[2000] Step 8:

[2001] The server analyzes the emotional data using an emotion engine and records the emotional state of the participants.

[2002] Step 9:

[2003] The server sends the progress script to the terminal, and the terminal starts the meeting through the announcer AI. For example, it announces, "Today's meeting will begin. The first topic on the agenda is an overview of the new product."

[2004] Step 10:

[2005] The terminal displays materials based on the progress script, and the announcer AI gives an explanation. The server uses emotional data to adjust the announcer AI's tone and progress according to the participants' reactions.

[2006] Step 11:

[2007] During a meeting, a user (participant) asks a question. The device recognizes the question through a microphone and converts it into text data.

[2008] Step 12:

[2009] The terminal sends the text data of the question to the server.

[2010] Step 13:

[2011] The server searches the database for relevant information, generates an answer to the question, and sends the generated information and answer to the terminal.

[2012] Step 14:

[2013] The device displays the received materials and answers to the meeting participants, and an announcer AI provides answers via voice.

[2014] Step 15:

[2015] The contents of discussions and progress steps during the meeting are recorded in real time and periodically sent to the server.

[2016] Step 16:

[2017] After the meeting, the server organizes the recorded data and categorizes the issues discussed.

[2018] Step 17:

[2019] The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[2020] Step 18:

[2021] The terminal reports the proposed solution to the user and provides it in an exportable format.

[2022] Example 2

[2023] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2024] Conventional conference systems not only lack efficiency in meeting progress and problem-solving, but also tend to lower the quality of meetings because they cannot take into account the emotions of participants.In addition, there are issues with the overall process being complicated and time-consuming, as it requires a lot of manual work, such as setting up meetings, confirming participants, preparing materials, answering questions, and following up after meetings.

[2025] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2026] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for solving the issues, and means for recognizing the emotions of conference participants in real time and reflecting them in the conference progress. This enables consistent and efficient processing of the conference, from preparation to progress, question and answer session, and post-conference issue organization and solution proposals. Furthermore, recognizing participants' emotions in real time and reflecting them in the conference progress can improve the quality of the conference.

[2027] 1. "Meeting setting information" refers to the detailed information necessary to hold a meeting, such as the date and time of the meeting, participants, location, and agenda.

[2028] 2. "Means for generating a conference schedule" refers to a device or program that has the function of automatically creating and saving a schedule, such as the date and time, location, and participant list of a conference, based on the conference setting information.

[2029] 3. "Means for recognizing the faces of meeting participants and updating attendance status" refers to devices or programs that use cameras and facial recognition technology to identify the faces of meeting participants, confirm attendance, and update logs.

[2030] 4. A "meeting script" is a document or instruction manual that describes the procedures and steps for conducting a meeting, and serves as a guideline for ensuring that the meeting proceeds on time.

[2031] 5. "Materials used during the meeting" refers to information that supports the progress of the meeting, such as data, reports, and presentation materials related to the meeting agenda.

[2032] 6. "Means for accepting questions from participants during the conference and generating answers to those questions" refers to devices or programs that have the function of receiving questions from participants, analyzing their content, and providing appropriate answers.

[2033] 7. "Means for recording meeting content and categorizing the issues discussed" refers to devices or programs that have the function of recording the content of discussions during meetings and categorizing and organizing that content.

[2034] 8. "Means for generating and reporting proposals for solving problems" refers to devices or programs that have the function of analyzing the problems discussed in the meeting, coming up with solutions, and compiling them into documents or reports.

[2035] 9. "Means of recognizing the emotions of meeting participants in real time and reflecting them in the progress of the meeting" refers to devices or programs that use cameras and microphones to analyze the facial expressions and tone of voice of meeting participants and adjust the way the meeting is conducted based on that emotional data.

[2036] The present invention combines an emotion engine with a system that supports efficient meeting progress and problem solving, and includes the following main components:

[2037] System configuration

[2038] This system is composed of server, terminal, and user components, and by clearly defining the roles of each component, efficient conference management is achieved.

[2039] server

[2040] Managing conference settings information

[2041] The server receives the meeting setting information and generates a meeting schedule. For example, specific setting information such as "A kickoff meeting for a new product is scheduled for May 15th at 2:00 PM. Participants include the general manager, project manager, and engineering team" is entered. The meeting schedule is then saved in the database and invitation emails are sent to participants.

[2042] Facial recognition and attendance updates

[2043] The server processes the facial recognition data of the conference participants and updates the attendance status. Specifically, it analyzes the facial recognition data sent from the terminals and updates the attendee list.

[2044] Preparing the script and materials

[2045] The server generates a meeting progress script and prepares materials related to the agenda (e.g., market research data, new product specifications, etc.). These materials are stored in cloud storage and prepared for transmission to the terminals.

[2046] Q&A and problem identification

[2047] The server accepts questions from participants and searches a database to generate answers to those questions. It also includes a means to record the meeting content and categorize the issues discussed.

[2048] Analysis by emotion engine

[2049] The server analyzes the emotional data sent from the devices and reflects the feedback in the progress script. The emotion engine analyzes the participants' facial expressions and tone of voice and makes adjustments according to the progress of the meeting.

[2050] Terminal

[2051] User Interface

[2052] The device functions as a user interface and receives instructions from the server to conduct the conference. It uses a camera and facial recognition function to recognize the faces of participants and transmits the data to the server.

[2053] Announcer AI

[2054] The AI ​​announcer on the device will conduct the meeting by voice, displaying the necessary materials based on the meeting script, and will recognize the user's questions and send them to the server as text data.

[2055] Real-time recording and analysis of emotion data

[2056] The device records the content of discussions in real time during the meeting and sends it to the server. It also uses a camera and microphone to analyze participants' facial expressions and tone of voice, and sends emotional data to the server.

[2057] User

[2058] Meeting setup and questions

[2059] Users enter meeting setting information and register it in the system. During the meeting, they can ask questions and give opinions and use the system's interface.

[2060] Specific examples

[2061] Consider a case where a kickoff meeting for a new product is being held and participants' emotions are recognized in real time and reflected in the progress of the meeting. The user sets up the meeting, and the server generates a meeting schedule and notifies participants. The device receives the prepared materials and performs facial recognition at the start of the meeting. The server sends instructions to the device based on the progress script, and the device's announcer AI follows these instructions to proceed with the meeting. During the meeting, the user asks a question, and the device recognizes the question through voice recognition and sends it to the server. The server generates an answer, which the device displays and explains in audio. After the meeting ends, the server organizes the recorded data and generates proposals for solving problems based on the emotional data. This improves the efficiency and quality of meetings.

[2062] The advantage of this system is that it can efficiently handle everything from preparing for meetings to conducting them, answering questions, and sorting out issues and proposing solutions after the meeting. Furthermore, it can improve the quality of meetings by utilizing feedback from an emotion engine.

[2063] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2064] Step 1: Receiving meeting settings

[2065] The user enters meeting setting information. For example, the user might enter "New product kickoff meeting scheduled for May 15th at 2:00 PM. Participants will be the general manager, project manager, and engineering team."

[2066] Input: Conference setup information from the user.

[2067] Data processing: The server receives this information and analyzes and organizes the conference setting information.

[2068] Output: Structured conference setup data.

[2069] Specific operation: The server saves the conference setting information in a database and sends invitation emails to participants as needed.

[2070] Step 2: Generate and save the meeting schedule

[2071] The server generates a conference schedule based on the received setting information.

[2072] Input: Conference setup information.

[2073] Data processing: Generate schedules based on meeting dates, times, locations, and participant lists.

[2074] Output: Meeting schedule information.

[2075] Specific operation: The server saves the generated schedule in the database and sends invitation emails to participants.

[2076] Step 3: Prepare the script and materials

[2077] The server creates a progress script for the conference and prepares the necessary materials.

[2078] Input: Meeting setup information and related materials (market research data, new product specifications, etc.).

[2079] Data processing: Organizing and generating procedures and materials.

[2080] Output: Progress script and preparation materials.

[2081] Specific operation: The material is stored in cloud storage and prepared to be sent to the device.

[2082] Step 4: Participant facial and emotion recognition

[2083] The device uses a camera to recognize the faces of participants at the start of the meeting.

[2084] Input: Face images of meeting participants.

[2085] Data processing: Data processing using facial recognition algorithms.

[2086] Output: Participant's facial recognition data.

[2087] Specific operation: The device sends facial recognition data to the server, which then updates the attendance status. At the same time, the device uses the camera and microphone to analyze emotion data and send it to the server.

[2088] Step 5: Conducting the meeting

[2089] The server sends instructions to the terminal based on the progress script.

[2090] Input: Progress script.

[2091] Data processing: Analyzes the instructions and sends them to the terminal.

[2092] Output: Progression instructions data.

[2093] How it works: The AI ​​announcer on the device conducts the meeting by voice while displaying the necessary materials. The emotion engine on the server analyzes the emotions of the participants and sends feedback to the device to adjust the way the meeting is conducted.

[2094] Step 6: Receiving questions and providing answers

[2095] The user (participant) asks a question by voice. For example, "Tell me about your new product launch plan."

[2096] Input: Participant's question audio data.

[2097] Data processing: Performs voice recognition and converts it into text data.

[2098] Output: Question text data.

[2099] How it works: The device recognizes the question and sends it as text data to the server. The server searches for relevant information in a database and generates an answer using a generative AI model. The device displays the answer in a document format, and the announcer AI provides the answer via voice.

[2100] Step 7: Record the meeting and organize the emotional data

[2101] The device records the content of discussions and emotional data during the meeting in real time and transmits it to the server.

[2102] Input: Meeting discussion content and participants' emotional data.

[2103] Data processing: Converting discussion content into text and analyzing emotional data.

[2104] Output: Recorded data and emotion data.

[2105] What it does: The server organizes the recorded data and categorizes the issues discussed.

[2106] Step 8: Generate problem-solving proposals

[2107] The server generates problem-solving proposals based on the recorded data.

[2108] Input: Recorded data and emotion data.

[2109] Data processing: Data analysis and generation of proposals using generative AI models.

[2110] Output: A document proposing a solution to the problem.

[2111] How it works: The server performs a detailed analysis of each problem, taking into account emotional data, and generates a solution proposal, including an action plan and predicted outcomes.

[2112] Step 9: Report the proposed solution

[2113] The server documents the generated proposals and reports them to the user.

[2114] Input: A proposal document for solving the problem.

[2115] Data processing: Proposal documentation and format conversion.

[2116] Output: Proposal report.

[2117] Specific operation: The server exports the proposed solution in document format, and the device displays and reports it to the user.

[2118] (Application example 2)

[2119] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2120] Although systems exist to support efficient meeting progress, there is a problem in the industrial sector where communication and discussions between workers are not sufficiently efficient. Furthermore, because there is no technology to analyze emotional data in real time and reflect it in the progress of meetings, it is difficult to respond quickly when participants have complaints or questions. This can lead to a decline in the quality and productivity of meetings and delays in problem-solving.

[2121] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2122] In this invention, the server includes means for receiving conference setting information and generating a conference schedule, means for performing facial recognition of conference participants and updating their attendance status, means for generating a conference progress script and preparing materials to be used during the conference, means for accepting questions from participants during the conference and generating answers to those questions, means for recording the conference content and categorizing the issues discussed, means for generating and reporting proposals for resolving the issues, means for analyzing participants' facial expressions and voice data in real time and saving the data as emotion data, and means installed on industrial terminals for adjusting the progress script based on the emotion data during the conference. This makes it possible to improve the efficiency and quality of industrial conferences and to analyze and respond to participants' emotions in real time.

[2123] definition statement

[2124] "Meeting setting information" is information that includes details of the date, time, location, participants and agenda of the meeting.

[2125] The "conference schedule" is a schedule generated based on the setting information of the conference, and includes the date and time of the conference, the order of proceedings on the agenda, and the duration of each agenda item.

[2126] A "conference participant" is someone who is scheduled to attend a conference.

[2127] "Facial recognition" is a technology that identifies a person's face from image data and identifies that person.

[2128] "Attendance status" is information indicating the attendance of conference participants.

[2129] A "meeting progress script" is a script created to support the progress of a meeting, and describes in detail the explanation of each agenda item and the progress procedure.

[2130] "Materials" are files containing documents, presentations, graphs, and other information prepared for use during a meeting.

[2131] "Questions" refer to questions or comments submitted by meeting participants during the meeting.

[2132] "Response" refers to an answer or explanation to a submitted question.

[2133] "Meeting content" is information including the topics discussed, statements, and decisions made during a meeting.

[2134] An "issue" refers to a problem to be discussed or resolved during a meeting.

[2135] "Categorization" refers to classifying the issues discussed into specific categories.

[2136] A "proposal" refers to a specific action plan or measure offered to solve a problem.

[2137] "Facial expression data" is information indicating the emotional state obtained from a person's face.

[2138] "Voice data" refers to acoustic information obtained from a person's voice, and is also used to analyze emotional states.

[2139] "Emotion data" is information indicating an emotional state based on facial expression data and voice data analyzed in real time.

[2140] An "industrial terminal" is a computer terminal used in factories and manufacturing sites, and is typically a rugged, durable piece of equipment.

[2141] "Adjusting the progress script" means changing the meeting progress script based on real-time emotional data and respond...

Claims

1. means for receiving conference setting information and generating a conference schedule; a means for performing facial recognition of meeting participants and updating their attendance status; A means for generating a meeting progress script and preparing materials to be used during the meeting; a means for accepting questions from participants during the conference and generating answers to those questions; A means of recording the meeting and categorizing the issues discussed; A means for generating and reporting proposals for resolving the problem; A system including:

2. 2. The system according to claim 1, further comprising a function for recognizing voice of the conference participants, and acquiring questions of the participants as text data.

3. 2. The system of claim 1, wherein after the conference, the recorded conference contents are stored and provided in a format that can be accessed later.

Images