System

Abstract

A system is provided.SOLUTION: A system comprising: means for inputting a purpose, agenda, and participant list of a conference; means for setting and storing details of the conference based on the input information using a generative AI model; means for receiving text input from a user by real-time typing and broadcasting the text input to other participants; means for monitoring progress of the conference and promoting conversation using the generative AI model; means for automatically analyzing conference contents and generating a summary after the conference; and means for storing the generated summary and transmitting the summary to the participants.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional meetings suffer from inefficient progress, a lot of unnecessary conversation, and the time it takes to make important decisions. Furthermore, what is said during a meeting is often not recorded, making it impossible to review the details later. Furthermore, in many cases, a meeting facilitator is required, and the efficiency of the meeting depends on the facilitator's skills. To solve these issues and realize efficient and productive meetings, the introduction of a new conferencing system is required. [Means for solving the problem]

[0005] The present invention includes a means for a meeting organizer to input the purpose, agenda, and participant list of a meeting, and a means for using a generative AI model to set and save the details of the meeting based on the input information. It also includes a means for receiving text input from a user via real-time typing and broadcasting it to other participants. Additionally, the generative AI model monitors the progress of the meeting and facilitates conversation, and after the meeting ends, it automatically analyzes the meeting content and generates a summary. It also includes a means for saving the generated summary and sending it to participants.

[0006] By configuring the system in this way, the entire process from start to finish of the meeting will proceed efficiently, eliminating unnecessary conversations and inefficiencies. Furthermore, the contents of the meeting will be recorded and summarized, making it easy to review later. As a result, we can expect to see an improvement in meeting productivity and an increase in the speed of decision-making.

[0007] A "meeting organizer" is a person who sets the purpose, agenda, and participant list of a meeting and is responsible for overseeing the entire event.

[0008] A "generative AI model" is an artificial intelligence algorithm that sets meeting details based on information entered by the user and assists in the progress of the meeting.

[0009] "Real-time typing" is a technique in which a user types text in real time, which is instantly displayed to other users.

[0010] "Text input" refers to the act of a user inputting text using a keyboard or the like during a conference.

[0011] "Broadcast" means sending specific data to multiple recipients simultaneously.

[0012] "Conversation promotion" refers to the act of the AI ​​model suggesting appropriate questions or comments to stimulate conversation when the progress of a meeting becomes stalled.

[0013] A "summary" is a document that briefly summarizes what was said during the meeting.

[0014] "Storage" refers to the act of storing input data or generated summaries in a storage device.

[0015] "Sending" refers to the act of delivering the generated summary to the conference participants via email or the like.

[0016] "Progress monitoring" means that the generative AI model checks the progress of the meeting in real time and intervenes if necessary. [Brief explanation of the drawings]

[0017] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11]FIG. 3 is a sequence diagram showing a processing flow of the data processing system according to the first embodiment. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1. [Figure 13] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system according to the second embodiment when an emotion engine is combined. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0018] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0019] First, the terms used in the following description will be explained.

[0020] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), and an APU (Accelerated Processing Unit).

[0021] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0022] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0023] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0024] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0025] [First embodiment]

[0026] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0027] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0028] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0029] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0030] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0031] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0032] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0033] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0034] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0035] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0036] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0037] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0038] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system is composed of a server, terminals, and users who play the following roles.

[0039] Meeting Settings

[0040] 1. User (Organizer):

[0041] Enter the purpose of the meeting, agenda, and participant list into the input form.

[0042] 2. Terminal:

[0043] The entered conference information is sent to the server.

[0044] 3. Server:

[0045] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[0046] Starting a meeting

[0047] 1. User:

[0048] Participants click the provided meeting link to join the meeting.

[0049] 2. Terminal:

[0050] Connect to the real-time communication server and send the user ID and conference ID.

[0051] Display meeting information on the UI.

[0052] 3. Server:

[0053] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[0054] Real-time typing for meeting management

[0055] 1. User:

[0056] Each participant will enter their opinions and suggestions in text in line with the agenda.

[0057] 2. Terminal:

[0058] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[0059] 3. Server:

[0060] A generative AI model monitors meeting progress and facilitates the conversation as needed.

[0061] Ending the meeting and generating a summary

[0062] 1. User:

[0063] After all agenda items have been decided, the organizer clicks the end button.

[0064] 2. Terminal:

[0065] The conference end information is sent to the server.

[0066] 3. Server:

[0067] A generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[0068] The generated summaries are stored in a database and sent to the participants.

[0069] Specific examples

[0070] Example configuration:

[0071] User (Organizer): "New product marketing strategy meeting"

[0072] Objective: "Determine a marketing strategy for a new product"

[0073] Agenda:

[0074] 1. Selecting your target market

[0075] 2. Advertising campaign planning

[0076] 3. Budget allocation

[0077] Progress example:

[0078] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[0079] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[0080] End example:

[0081] User (Organizer): Complete all agenda items and click the End Meeting button.

[0082] Server: The generative AI model summarizes the meeting content and sends the summary to all participants.

[0083] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[0084] The processing flow will be explained below.

[0085] Step 1:

[0086] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[0087] Step 2:

[0088] Terminal: Sends the entered meeting information to the server in JSON format.

[0089] Step 3:

[0090] Server: Stores the received conference information in a database.

[0091] Step 4:

[0092] Server: The generative AI model sets up the meeting details based on the received information.

[0093] Step 5:

[0094] User: Participants join the meeting by clicking the provided meeting link.

[0095] Step 6:

[0096] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[0097] Step 7:

[0098] Server: Retrieves conference information from the database based on the conference ID.

[0099] Step 8:

[0100] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[0101] Step 9:

[0102] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[0103] Step 10:

[0104] Terminal: The meeting assistant displays the meeting opening message on the screen.

[0105] Step 11:

[0106] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[0107] Step 12:

[0108] Terminal: Sends text data entered by the user to the real-time communication server.

[0109] Step 13:

[0110] Server: Broadcasts the received text data from each participant to all user terminals.

[0111] Step 14:

[0112] Terminal: Text received from other users is displayed on the UI in real time.

[0113] Step 15:

[0114] Server: A generative AI model monitors the progress of the meeting and facilitates the conversation as needed.

[0115] Step 16:

[0116] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[0117] Step 17:

[0118] Terminal: Sends conference end information to the real-time communication server.

[0119] Step 18:

[0120] Server: The generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[0121] Step 19:

[0122] Server: Stores the generated summaries in a database.

[0123] Step 20:

[0124] Server: Sends the generated summary to the participants.

[0125] Example 1

[0126] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0127] Meetings often involve unnecessary conversations, making it difficult to hold efficient discussions and make decisions. It also requires a lot of effort to quickly and accurately record the contents of meetings and provide a summary to participants. Furthermore, when multilingual support is required, communication becomes difficult when participants speak different languages.

[0128] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0129] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This allows for efficient conference progress, eliminates unnecessary conversation, and enables quick and accurate recording of conference content and provision of summaries, as well as multilingual support.

[0130] A "meeting organizer" is the person responsible for setting the purpose, agenda, and participant list for a meeting.

[0131] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate text and manage meeting progress.

[0132] An "input form" is an interface that allows a conference organizer to input information about the conference.

[0133] A "real-time communication server" is a server for transmitting and receiving data in real time.

[0134] A "user ID" is an identifier for uniquely identifying a user participating in a conference.

[0135] A "conference ID" is an identifier for uniquely identifying an individual conference.

[0136] An "agenda" is a list of items or topics to be discussed at a meeting.

[0137] "Text data" is character information input by the user.

[0138] "Display in real time" means that the input information is instantly displayed on the terminals of the other participants.

[0139] The "end conference button" is a button used by the conference organizer to instruct the end of the conference.

[0140] The "summary" is a shortened version of the meeting content generated based on the text data recorded during the meeting.

[0141] A "database" is a storage system for storing meeting details and generated summaries.

[0142] "Multilingual support" refers to translating the generated abstracts and meeting information into multiple languages.

[0143] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system operates in cooperation with the server, terminals, and users.

[0144] Meeting Settings

[0145] First, the user, who is the meeting organizer, enters the purpose, agenda, and participant list of the meeting into the system's input form. The device sends this information to the server. The server stores the received meeting information in a database, and a generative AI model (e.g., GPT-3) sets the meeting details.

[0146] Examples:

[0147] The organizer types in "New Product Marketing Strategy Meeting" and adds agenda items such as "Select Target Market," "Plan Advertising Campaign," and "Allocate Budget."

[0148] Starting a meeting

[0149] Users click on the designated meeting link to join the meeting. The device then connects to the real-time communication server and sends the user ID and meeting ID. The server retrieves the meeting information from the database, and the generative AI model displays the purpose and agenda of the meeting.

[0150] Examples:

[0151] Users click the link to access the meeting page, where the purpose and agenda of the meeting will be displayed on the screen.

[0152] Real-time typing for meeting management

[0153] Each participant, a user, enters their opinions and suggestions in text based on the agenda. The device sends this input text data to a real-time communication server, which also receives and displays the text data of other participants in real time. The server uses a generative AI model to monitor the progress of the conversation and facilitate it as needed.

[0154] Examples:

[0155] When a user types "advertising campaigns should be primarily conducted on social media," the content is instantly shared with other participants.

[0156] The server uses a generative AI model to display "May we move on to the next topic?"

[0157] Ending the meeting and generating a summary

[0158] When all agenda items have been completed, the organizer clicks the end button. The device sends the end-of-meeting information to the server. The server then uses a generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is stored in a database and sent to participants.

[0159] Examples:

[0160] After the meeting, the generative AI model generates a document summarizing the key points of the meeting and emails it to each participant.

[0161] Prompt Sentence Examples

[0162] "Please proceed to the next agenda item."

[0163] "Does anyone else have any opinions on this topic?"

[0164] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[0165] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0166] Step 1:

[0167] The user enters the meeting information.

[0168] The user enters the purpose of the meeting, the agenda, and a list of participants into an input form. The input includes the meeting name "New product marketing strategy meeting," and agenda items such as "Selection of target market," "Advertising campaign planning," and "Budget allocation." This information is generated as output in JSON format. The user enters this information into the input form on a web browser.

[0169] Step 2:

[0170] The terminal transmits the conference information.

[0171] The terminal sends the conference information entered in step 1 to the server. As input, it receives the JSON-formatted conference information entered by the user. As data processing, it converts this JSON data into an HTTP request and sends it as output to the server. Specifically, the terminal generates a request and sends a POST request to the server's API endpoint.

[0172] Step 3:

[0173] The server stores the meeting information and configures the generative AI model.

[0174] The server stores the meeting information received from the device in a database and configures the meeting details in the generative AI model. As input, it receives the JSON-formatted meeting information from the device. As data processing, it inserts this information into the database and configures the required settings for the generative AI model. As output, it notifies the generative AI model that the meeting details have been configured. The server executes an SQL query to store the information in the database and configures parameters through the AI ​​model's configuration interface.

[0175] Step 4:

[0176] A user joins a conference.

[0177] Users join a meeting by clicking a specified meeting link. The input is the act of clicking the meeting link. The output is the display of the meeting UI. The specific behavior is that the user clicks a link embedded in an email or calendar event they received, and the meeting page opens in a browser.

[0178] Step 5:

[0179] The terminal acquires and displays the conference information.

[0180] The terminal connects to the real-time communication server and sends the user ID and conference ID. It also displays the conference information on the UI. As input, it obtains the user ID and conference ID. As data processing, it connects to the real-time communication server using WebSocket and sends these IDs as authentication information. As output, the display of the conference information is completed. Specifically, the terminal displays the conference details information it received from the server in the UI component.

[0181] Step 6:

[0182] The server sends the meeting details.

[0183] The server retrieves meeting information from the database, and the generative AI model sends the meeting purpose and agenda to the device. As input, it receives the user ID and meeting ID. As data processing, it executes an SQL query to retrieve information from the database, and the generative AI model formats the meeting details. As output, it sends the generated meeting details to the device. Specifically, the server sends the output of the generative AI model to the device in real time.

[0184] Step 7:

[0185] The user inputs their opinion.

[0186] Each participant, a user, enters their opinions and suggestions in text along the agenda. As input, they enter the text of their opinions and suggestions related to the agenda. As output, that text data is generated. Specifically, they enter their opinions in the chat window and click the send button.

[0187] Step 8:

[0188] The device sends and receives text data.

[0189] The device sends the entered text data to the real-time communication server, receives text data from other participants, and displays it in real time. As input, it receives text data entered by the user. As data processing, it sends and receives this data via WebSocket, and as output, it displays it in the chat window in real time. In concrete terms, the device sends text data and immediately displays the received data.

[0190] Step 9:

[0191] The server facilitates the conversation.

[0192] The server uses a generative AI model to monitor the progress of the meeting and facilitate the conversation as needed. As input, it receives text data from each participant. As data processing, the generative AI model analyzes the content of the conversation and generates appropriate prompts. As output, a prompt for promotion is generated and sent to the terminal. Specifically, the generative AI model generates and displays a message such as "Shall we move on to the next agenda item?"

[0193] Step 10:

[0194] The user ends the conference.

[0195] When all agenda items have been completed, the user (organizer) clicks the end conference button. The input is the operation of clicking the end conference button. The output is the generation of conference end information. The specific operation is the user clicking the end conference button.

[0196] Step 11:

[0197] The terminal transmits the conference end information.

[0198] The terminal sends the conference end information to the server. As input, it receives information that the conference end button has been pressed. As data processing, it sends this information as an HTTP request. As output, it sends the conference end information to the server. Specifically, the terminal POSTs the conference end information to the server.

[0199] Step 12:

[0200] The server generates and transmits the meeting summary.

[0201] The server uses the generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is saved in a database and sent to participants. All text data is received as input. As data processing, the generative AI model analyzes the text data and automatically generates a summary. As output, the generated summary is saved and sent to participants by email. Specifically, the server uses the generative AI model to generate a summary and sends it to participants using the email sending function.

[0202] (Application example 1)

[0203] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0204] With conventional conferencing systems, meeting progress management was often done manually, resulting in long meetings and frequent unnecessary conversations. Furthermore, there were limited ways for participants to share their opinions in real time, making meetings less efficient. In brick-and-mortar stores in particular, poor communication between store operations and staff can lead to a decline in work efficiency. Furthermore, organizing the content of meetings after they have ended was time-consuming, resulting in delays in summarizing and sharing results.

[0205] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0206] In this invention, the server includes: a means for a meeting organizer to input the purpose, agenda, and participant list of the meeting; a means for using a generative AI model to set and save the details of the meeting based on the input information; a means for receiving text input from the user via real-time typing and broadcasting it to other participants; a means for the generative AI model to monitor the progress of the meeting and promote conversation; a means for automatically analyzing the meeting content and generating a summary after the meeting ends; a means for saving the generated summary and sending it to participants; a means for staff in a physical store to join the meeting from a smart device and input their opinions in real time; a means for the generative AI model to generate a prompt comment during the meeting to encourage the next agenda item; a means for displaying the generated prompt comment on the staff member's smart device; and a means for sending a summary generated based on the meeting content to the staff member's smart device, thereby enabling efficient progress of meetings and smooth communication in a physical store.

[0207] A "meeting organizer" is a user whose role is to enter the purpose, agenda, and participant list of a meeting.

[0208] A "generative AI model" is an artificial intelligence that sets meeting details based on input information, monitors progress, and facilitates conversations.

[0209] "Real-time typing" is the process by which a user types text during a meeting, which is then shared with other participants in real time.

[0210] "Broadcast" is a communication method that distributes text input from a user to all participants simultaneously.

[0211] A "promotion comment" is a text message sent by the generative AI model during a meeting to prompt the participant to move on to the next agenda item.

[0212] "Smart devices" is a general term for portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[0213] A "brick and mortar store" is a sales establishment that exists in a physical location and conducts business face-to-face with customers.

[0214] A "meeting summary" is a concise summary of what was discussed during the meeting.

[0215] "Means for inputting opinions in real time" refers to a method in which meeting participants use smart devices to instantly input their opinions in text.

[0216] "Means for automatic translation into multiple languages" refers to a function that automatically converts the generated summary into different languages.

[0217] This invention is a system that efficiently conducts meetings between staff in a physical store and eliminates unnecessary conversations.

[0218] Meeting Settings

[0219] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[0220] The terminal transmits the input conference information to the server.

[0221] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[0222] Starting a meeting

[0223] Users click on the provided meeting link to join the meeting.

[0224] The terminal connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI.

[0225] The server retrieves meeting information from a database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[0226] Real-time typing for meeting management

[0227] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[0228] The terminal transmits text data entered by the user to the real-time communication server, and receives text data from other participants and displays it in real time.

[0229] The server uses a generative AI model to monitor the progress of the meeting, generate prompts to move on to the next agenda item, and display them on participants' devices.

[0230] As a specific example, the following prompt sentence is generated: "Meeting agenda: Marketing strategy meeting for a new product. Participant's message: What should be the target market for the new product? We are considering a plan for an advertising campaign. Please generate a message to encourage whether we should proceed next."

[0231] Ending the meeting and generating a summary

[0232] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[0233] The terminal transmits the conference end information to the server.

[0234] The server uses a generative AI model to analyze all text data recorded during the meeting and generate a summary.

[0235] The generated summaries are stored in a database and sent to all participants.

[0236] It also provides a means for the generated summary to be automatically translated into multiple languages.

[0237] The system allows store staff to quickly and efficiently share opinions using smart devices (smartphones, tablets, etc.), eliminating unnecessary conversations. It also uses a generative AI model to monitor meeting progress in real time and generate appropriate prompts to guide the meeting.

[0238] A specific example is when staff at a brick-and-mortar store exchange opinions in real time based on the agenda for a "New Product Marketing Strategy Meeting." The generative AI model provides prompt comments such as "Shall we move on to the next agenda item?", allowing the meeting to proceed smoothly.

[0239] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0240] Processing Steps

[0241] Step 1:

[0242] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[0243] Input: Meeting purpose, agenda, participant list

[0244] Output: Entered meeting information

[0245] Specific action: A user enters text into a web or application form.

[0246] Step 2:

[0247] The terminal transmits the input conference information to the server.

[0248] Input: Meeting Information

[0249] Output: Meeting information transferred to the server

[0250] Specific operation: Obtain data from the input form and send it to the server via an HTTP request.

[0251] Step 3:

[0252] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[0253] Input: Meeting Information

[0254] Output: Meeting information stored in the database, configured meeting details

[0255] Specific operation: The received data is stored in a database via an SQL query, and the AI ​​model uses that data to complete meeting details (purpose and agenda).

[0256] Step 4:

[0257] User clicks on the provided meeting link to join the meeting.

[0258] Input: Meeting link

[0259] Output: Meeting participation confirmation

[0260] What happens: A user clicks on a link they received in an email or message to access the meeting join page.

[0261] Step 5:

[0262] The device connects to the real-time communication server and sends the user ID and conference ID. The conference information is displayed on the UI.

[0263] Input: User ID, Meeting ID

[0264] Output: Connection status, displayed meeting information

[0265] Specific operation: User information is sent via WebSocket or HTTP request, and conference information is obtained and displayed on the screen.

[0266] Step 6:

[0267] The server retrieves meeting information from the database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[0268] Input: Meeting ID

[0269] Output: Meeting information sent to the device and display content generated by the AI ​​model

[0270] How it works: Meeting information is retrieved through a database query and sent to the device. Based on that information, the AI ​​model displays the purpose and agenda in the UI.

[0271] Step 7:

[0272] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[0273] Input: User text input

[0274] Output: Real-time shared text data

[0275] Specific operation: The user enters opinions and suggestions in the text box, which are then sent to the real-time communication server.

[0276] Step 8:

[0277] The terminal transmits text data entered by the user to a real-time communication server, receives text data from other participants, and displays it in real time.

[0278] Input: User's text data

[0279] Output: Text data broadcast to other participants, text data displayed

[0280] Specific operation: Send data to other participants via a real-time communication protocol (e.g., WebSocket) and display the received data on the screen.

[0281] Step 9:

[0282] The server monitors the progress of the meeting using a generative AI model, generates prompts to move on to the next agenda item, and displays them on participants' devices.

[0283] Input: Meeting progress data

[0284] Output: Generated promotion comment, Displayed promotion comment

[0285] Specific operation: The AI ​​model analyzes the progress of the meeting, generates prompts such as "Shall we move on to the next agenda item?", and sends them to the device for display.

[0286] Step 10:

[0287] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[0288] Input: End meeting instruction

[0289] Output:Confirmation of end of meeting

[0290] Specific operation: The terminal captures the click event of the end conference button and sends an end instruction to the server.

[0291] Step 11:

[0292] The terminal transmits conference end information to the server.

[0293] Input: Meeting end information

[0294] Output: Finished information sent to the server

[0295] Specific operation: Sends conference end information to the server via HTTP request or WebSocket.

[0296] Step 12:

[0297] The server uses the generative AI model to analyze all the text data recorded during the meeting and generate a summary.

[0298] Input: All text data during the meeting

[0299] Output: Generated meeting summary

[0300] Specific operation: Analyzes text data and generates summaries using natural language processing techniques.

[0301] Step 13:

[0302] The server stores the generated summaries in a database and sends them to all participants, and also provides a means for automatic translation into multiple languages.

[0303] Input: Generated meeting summary

[0304] Output: Summary stored in database, multilingual summary sent

[0305] Specific actions: The generated summary is saved and sent to each participant via email or notification. If necessary, the summary is translated into multiple languages ​​using an automatic translation API.

[0306] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0307] This invention is a system that uses a generative AI model and emotion engine to conduct meetings efficiently based on real-time typing and emotion information, eliminating unnecessary conversations. This system is composed of a server, terminals, and users who play the following roles.

[0308] Meeting Settings

[0309] 1. User (Organizer):

[0310] Enter the purpose of the meeting, agenda, and participant list into the input form.

[0311] 2. Terminal:

[0312] The entered conference information is sent to the server in JSON format.

[0313] 3. Server:

[0314] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[0315] Starting a meeting

[0316] 1. User:

[0317] Participants click the provided meeting link to join the meeting.

[0318] 2. Terminal:

[0319] Connect to the real-time communication server and send the user ID and conference ID.

[0320] Display the meeting information on the UI and activate the emotion engine.

[0321] 3. Server:

[0322] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[0323] The emotion engine analyzes each user's emotional data in real time and feeds the results back into the generative AI model.

[0324] Real-time typing for meeting progress and emotion analysis

[0325] 1. User:

[0326] Each participant will enter their opinions and suggestions in text in line with the agenda.

[0327] 2. Terminal:

[0328] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[0329] The emotional engine analyzes each participant's emotional state and displays it to other participants as appropriate.

[0330] 3. Server:

[0331] A generative AI model monitors the progress of the meeting and drives the conversation based on the user's emotional data.

[0332] For example, if the user is nervous, the generative AI model will insert questions or comments to help them relax.

[0333] Ending the meeting and generating a summary

[0334] 1. User:

[0335] After all agenda items have been decided, the organizer clicks the end button.

[0336] 2. Terminal:

[0337] The conference end information is transmitted to the real-time communication server.

[0338] Data including the analysis results of the emotion engine is sent to the server.

[0339] 3. Server:

[0340] A generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[0341] The summary includes not only the conclusions and key statements for each topic, but also the emotional state of the meeting.

[0342] The generated summaries are stored in a database and sent to the participants.

[0343] Specific examples

[0344] Example configuration:

[0345] User (Organizer): "New product marketing strategy meeting"

[0346] Objective: "Determine a marketing strategy for a new product"

[0347] Agenda:

[0348] 1. Selecting your target market

[0349] 2. Advertising campaign planning

[0350] 3. Budget allocation

[0351] Progress example:

[0352] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[0353] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[0354] Terminal: The emotion engine analyzes the emotions of each participant and displays relaxing comments to users who are nervous.

[0355] End example:

[0356] User (Organizer): Complete all agenda items and click the End Meeting button.

[0357] Server: The generative AI model summarizes the meeting content and sends the summary and sentiment analysis results to all participants.

[0358] In this way, the system of the present invention not only supports efficient and productive meeting progress and eliminates unnecessary conversations, but also improves the quality of meetings by utilizing emotional data.

[0359] The processing flow will be explained below.

[0360] Step 1:

[0361] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[0362] Step 2:

[0363] Terminal: Sends the entered meeting information to the server in JSON format.

[0364] Step 3:

[0365] Server: Stores the received conference information in a database.

[0366] Step 4:

[0367] Server: The generative AI model sets up the meeting details based on the received information.

[0368] Step 5:

[0369] User: Participants join the meeting by clicking the provided meeting link.

[0370] Step 6:

[0371] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[0372] Step 7:

[0373] Server: Retrieves conference information from the database based on the conference ID.

[0374] Step 8:

[0375] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[0376] Step 9:

[0377] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[0378] Step 10:

[0379] Terminal: The meeting assistant displays the meeting opening message on the screen.

[0380] Step 11:

[0381] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[0382] Step 12:

[0383] Terminal: Sends text data entered by the user to the real-time communication server.

[0384] Step 13:

[0385] Server: Broadcasts the received text data from each participant to all user terminals.

[0386] Step 14:

[0387] Terminal: Text received from other users is displayed on the UI in real time.

[0388] Step 15:

[0389] Terminal: The emotion engine analyzes the emotions of each participant and sends the results to the server in real time.

[0390] Step 16:

[0391] Server: Feedbacks emotional data from the emotion engine to the generative AI model and monitors the progress of the meeting.

[0392] Step 17:

[0393] Server: The generative AI model uses emotional data to insert comments and questions to encourage conversation.

[0394] Step 18:

[0395] Device: Analyzed emotion results are displayed in real time on the UI, providing feedback according to the situation.

[0396] Step 19:

[0397] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[0398] Step 20:

[0399] Terminal: Sends conference end information to the real-time communication server.

[0400] Step 21:

[0401] Server: The generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[0402] Step 22:

[0403] Server: Stores the generated summaries in a database.

[0404] Step 23:

[0405] Server: Sends summary results to participants.

[0406] Step 24:

[0407] Server: May automatically translate abstracts into multiple languages ​​and send each language version of the abstract to participants.

[0408] Example 2

[0409] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0410] The purpose of this invention is to improve the quality of meetings by efficiently conducting meetings, sharing information, and analyzing participants' emotional data. Conventional meeting systems have not adequately considered the efficiency of meeting progress or the emotional state of participants, resulting in problems such as participants losing concentration and discussion stalling. Furthermore, generating summaries of comments and discussions during meetings is often done manually, requiring time and effort.

[0411] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0412] In this invention, the server

[0413] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[0414] a means for configuring and saving meeting details based on input information using a generative AI model;

[0415] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[0416] means for analyzing the emotional state of each user in real time using an emotion engine;

[0417] A generative AI model monitors the progress of the meeting and uses user emotional data to facilitate the conversation.

[0418] means for automatically analyzing the contents of a meeting after the meeting and generating a summary from the text data and emotion data;

[0419] and means for storing and transmitting the generated summary to the participants.

[0420] This will enable meetings to proceed efficiently, improve the quality of meetings by utilizing participants' emotional data, and automatically generate summaries.

[0421] A "meeting organizer" is a person who is responsible for setting the purpose, agenda, and participant list of a meeting and managing the overall progress of the meeting.

[0422] A "generative AI model" is an artificial intelligence model that uses technologies such as natural language processing to assist in the progress of meetings, providing appropriate feedback and support for progress based on input information.

[0423] An "emotion engine" is a system that analyzes participants' emotional state in real time from voice, facial expressions, text, etc., and provides feedback on the analysis results.

[0424] "Real-time typing" refers to the process by which a user types text in real time, which is instantly broadcast to other participants.

[0425] "Broadcast" refers to the simultaneous transmission of data (e.g., a text message) from one user to multiple recipients in real time.

[0426] "Meeting progress" refers to the state of the meeting, showing how well it is progressing according to the agenda, the flow of comments, progress on the agenda, etc.

[0427] "User emotional data" refers to data that indicates the emotional state of a participant, and is obtained from voice tone, text content, facial expressions, etc.

[0428] A "summary" refers to a concise report containing the overall conclusions and important statements of the meeting, generated based on all text and emotion data recorded during the meeting.

[0429] A "real-time communication server" refers to a server that manages real-time data communication between users using WebSocket or similar.

[0430] This invention is a system that uses a generative AI model and emotion engine to conduct meetings in real time based on typing and emotion information, eliminating unnecessary conversations, in order to ensure effective and productive meeting proceedings. This system is composed of three entities: a server, a terminal, and a user.

[0431] Meeting Settings

[0432] User (organizer):

[0433] The user enters the purpose, agenda, and participant list of the meeting into a dedicated input form. For example, the user can set the purpose, such as "New product marketing strategy meeting," and each agenda item.

[0434] Device:

[0435] The meeting information entered by the user is converted into JSON format and sent to the server using an HTTP POST request.

[0436] server:

[0437] The server stores the received JSON data in a database such as MySQL. The server then uses a generative AI model to set up the meeting details based on the received information. The generative AI model automatically calculates the optimal order and time allocation for the meeting.

[0438] Starting a meeting

[0439] User:

[0440] Participants click on the designated meeting link and log in to the system using a web browser or dedicated application.

[0441] Device:

[0442] The device connects to a real-time communication server such as a WebSocket server and transmits the user ID and conference ID. After the connection is complete, the device displays the conference information on the UI and starts the emotion engine, which starts analyzing the user's emotions.

[0443] server:

[0444] The server retrieves meeting information from the database and sends it to the device. The generative AI model displays the meeting's purpose and agenda on the screen and supports the meeting's progress. The emotion engine analyzes each user's emotional data in real time and feeds the results back to the generative AI model.

[0445] Real-time typing for meeting progress and emotion analysis

[0446] User:

[0447] Each participant enters their opinions and suggestions in text form along the agenda, using UI components such as a chat box.

[0448] Device:

[0449] The device sends the text data entered by the user to a real-time communication server using WebSocket or similar, and also receives and displays the text data of other participants in real time. The device also analyzes the emotional state of each participant using an emotion engine and displays the results on the UI. For example, if a user is judged to be "tense," the device will display "Please relax."

[0450] server:

[0451] The server monitors the progress of the meeting using a generative AI model. Based on participants' comments and emotional data, the generative AI model suggests the next agenda item and necessary questions. For example, if the discussion stalls, the generative AI model will insert a question such as, "Do you have any more specific suggestions for the current issue?"

[0452] Ending the meeting and generating a summary

[0453] User:

[0454] After all agenda items have been decided, the organizer clicks the end button.

[0455] Device:

[0456] The terminal transmits the conference end information to the real-time communication server, and also transmits the final analysis result of the emotion engine to the server.

[0457] server:

[0458] The server uses a generative AI model to analyze all text and emotion data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting. The summaries are stored in a database and sent to participants in real time via email or app notifications.

[0459] Prompt Sentence Examples

[0460] "Please provide your opinion on selecting the target market for the new product marketing strategy meeting."

[0461] "May we move on to the next topic?"

[0462] "Do you have any more specific suggestions for the current issue?"

[0463] As a result, the system of the present invention supports efficient and productive conference progress, eliminates unnecessary conversations, and improves the quality of conferences by utilizing emotion data.

[0464] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0465] Step 1: Enter and submit meeting information

[0466] Input: The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[0467] How it works: The terminal converts the input data into JSON format.

[0468] Output: The terminal sends the converted JSON data to the server using an HTTP POST request.

[0469] Step 2: Save meeting information and configure it with a generative AI model

[0470] Input: JSON data sent from the terminal.

[0471] How it works: The server stores the received JSON data in a database such as MySQL. It then launches a generative AI model to set up the details of the meeting based on the received information. The generative AI model automatically calculates the optimal order of the meeting and time allocation.

[0472] Output: The configured meeting details are saved in the database.

[0473] Step 3: Share the meeting link and connect

[0474] Input: User clicks on the provided meeting link.

[0475] Operation: The device connects to a real-time communication server (such as a WebSocket server) and sends the user ID and conference ID.

[0476] Output: The connection is authenticated and the meeting information is displayed in the UI.

[0477] Step 4: Displaying meeting information and launching the emotion engine

[0478] Input: Meeting information retrieved from the server.

[0479] Operation: The device displays the received conference information on the UI and starts the emotion engine, which starts emotion analysis.

[0480] Output: The meeting information is displayed on the screen and the emotion engine starts working.

[0481] Step 5: Real-time typing and sending and receiving data

[0482] Input: The user types their opinion or suggestion into the text box.

[0483] Operation: The device sends the user's input data to the real-time communication server, and simultaneously receives text data from other participants and displays it in real time. The emotion engine analyzes the emotional state of each participant, and the results are reflected in the UI.

[0484] Output: The input text data is broadcast to other participants in real time, and the analyzed emotional information is displayed.

[0485] Step 6: Monitor and facilitate meeting progress

[0486] Input: User utterances and emotional state data.

[0487] How it works: The server monitors the progress of the meeting using a generative AI model. The generative AI model uses participants' comments and emotional data to suggest the next agenda item and necessary questions. For example, if the discussion stagnates, the generative AI model inserts appropriate questions to stimulate the conversation.

[0488] Output: A prompt message based on feedback from the generative AI model is displayed.

[0489] Step 7: Ending the meeting and sending data

[0490] Input: Organizer clicks end meeting button.

[0491] Operation: The terminal sends the conference end information to the real-time communication server, and also sends the final analysis results from the emotion engine to the server.

[0492] Output: Information on the end of the conference and the final analysis results are sent to the server.

[0493] Step 8: Generate and send the summary

[0494] Input: All text and emotion data recorded during the meeting.

[0495] How it works: The server uses a generative AI model to automatically generate a summary of the meeting content, including conclusions for each topic, important comments, and the emotional state of the meeting.

[0496] Output: The generated summaries are stored in a database and sent to participants via email or a dedicated app notification.

[0497] By following the above steps, this system is able to conduct a conference effectively and efficiently.

[0498] (Application example 2)

[0499] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0500] In modern factory operations, improving meeting efficiency and speeding up decision-making are important challenges. However, conventional methods make meetings slow and prone to unnecessary conversation. Furthermore, there is no way to grasp the emotional state of meeting participants and provide appropriate feedback, leading to tension and stress that reduces the quality of meetings. This invention aims to use a generative AI model and emotion engine to efficiently conduct factory meetings and improve the quality of meetings by taking into account the emotional state of participants.

[0501] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0502] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for receiving text input from the user via real-time typing and broadcasting it to other participants, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for analyzing the user's emotional state and providing feedback, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This enables the conference to proceed more efficiently and the emotional states of the participants to be managed appropriately.

[0503] A "meeting organizer" is a person in charge of setting up a meeting by entering the purpose, agenda, participant list, etc. of the meeting.

[0504] A "generative AI model" is a system that uses artificial intelligence to support detailed meeting setup and progress based on input information.

[0505] An "agenda" refers to the items or topics to be discussed at a meeting.

[0506] "Real-time typing" is a system in which participants enter their opinions and comments in text during a meeting, and the content is instantly transmitted to other participants.

[0507] "Broadcast" means sending specific information to multiple recipients simultaneously.

[0508] The "Emotion Engine" is a system that analyzes the emotional state of participants in real time and reflects the results in the progress of the meeting.

[0509] "Emotional state" refers to the emotional state, such as tension, joy, or fatigue, that participants feel during the meeting.

[0510] "Feedback" refers to providing appropriate responses or comments to participants based on the emotional data analyzed by the emotion engine.

[0511] A "summary" is a short summary of the entire meeting and its important points, generated after the meeting is over.

[0512] "Storage" means recording the generated data in a database or the like so that it can be referenced later.

[0513] "Transmit" means transmitting information via email or other means to provide the generated summary and sentiment analysis results to participants.

[0514] This invention is a system that efficiently facilitates factory meetings and improves their quality by taking into account the emotional state of participants. The system utilizes a generative AI model and an emotion engine to analyze real-time text input and emotional data to facilitate meetings.

[0515] Program processing and hardware / software used

[0516] The system includes the following components:

[0517] 1. Server:

[0518] Generative AI models, such as OpenAI GPT-3, are used to set meeting details and facilitate the conversation. They use input to set meeting details, monitor progress, and facilitate the conversation.

[0519] Database: MongoDB or similar is used to store conference setting information and generated summaries.

[0520] Real-time communication server: Supports real-time data exchange between users using WebSockets, etc.

[0521] 2. Terminal:

[0522] User interface: HTML, CSS, and JavaScript are used to set up meetings, input real-time information, display emotional states, and display meeting summaries.

[0523] Emotion engine: Analyzes the user's emotional state in real time and generates feedback using a custom analysis module or an existing emotion analysis API (e.g., Microsoft Azure Emotion API).

[0524] 3. User:

[0525] Organizer: Set up the meeting by entering the purpose, agenda, and participant list.

[0526] Participants: Enter thoughts and comments in real time during the meeting, which are broadcast to other participants and displayed in real time.

[0527] Specific examples

[0528] Conference Settings:

[0529] The user (organizer) inputs the purpose, agenda, and participant list of the meeting. For example, if the purpose is a "marketing strategy meeting for a new product," the agenda may include "selecting the target market," "planning the advertising campaign," and "allocating the budget."

[0530] Meeting proceedings:

[0531] The device connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI and the emotion engine is activated.

[0532] Each participant enters their opinions and suggestions in text based on the agenda, and the text data is displayed to other participants in real time. The emotion engine analyzes each participant's emotional state, and if the emotion analyzed is "tension," for example, the system displays a message such as "Please relax."

[0533] Conference Summary:

[0534] After the meeting, the generative AI model analyzes all the text and emotion data recorded during the meeting and generates a summary of the entire meeting, including the conclusions and key comments made on each topic, as well as the emotional state of the meeting.

[0535] The summaries will be stored in a database and communicated to participants.

[0536] Prompt Sentence Examples

[0537] Here are some examples of prompts:

[0538] Meeting Information:

[0539] Title: Improve work efficiency

[0540] Agenda:

[0541] 1. Review of work processes

[0542] 2. Reassignment of workers

[0543] 3. Introducing new tools

[0544] Real-time input:

[0545] User1: "I think there's a lot of overlap in our current workflow."

[0546] User 2: "I think introducing new tools will improve efficiency."

[0547] Prompt for GPT-3:

[0548] Summarize the conversation below and consider the sentiment data:

[0549] "I think there is a lot of overlap in the current work process." (Average)

[0550] "I think that introducing new tools will improve efficiency." (Optimistic)

[0551] As described above, the present invention can improve the efficiency of meetings and manage emotions, thereby improving the quality of meetings within a factory.

[0552] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0553] Step 1:

[0554] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list. The entered data is sent from the terminal to the server. The input data is sent to the server in JSON format and saved in the database as configuration information.

[0555] Step 2:

[0556] The server uses a generative AI model to set up detailed meetings based on the input meeting information. Specifically, it checks the purpose of the meeting and the order of the agenda, and creates an appropriate timetable. This information is also stored in a database.

[0557] Step 3:

[0558] When a conference starts, users (participants) click on the designated conference link from their devices to join the conference. The devices connect to the real-time communication server and send the user ID and conference ID. The server then sends the conference information to the devices and activates the emotion engine.

[0559] Step 4:

[0560] Each user inputs their opinions and suggestions in text format into their device, following the agenda. The input text data is sent to the real-time communication server and broadcast to the devices of other participants in real time. The input data is displayed immediately.

[0561] Step 5:

[0562] The emotion engine analyzes each user's input text and determines their emotional state. The device receives the analysis results and displays the user's emotional state (e.g., "tension" or "joy") on the screens of other participants. If necessary, it also displays a feedback message to encourage relaxation.

[0563] Step 6:

[0564] The server's generative AI model monitors the progress of the meeting and generates and displays text prompting the user to move on to the next agenda item as needed, such as "Shall we move on to the next item?"

[0565] Step 7:

[0566] After the meeting is over, the organizer clicks the end button. The device sends the end-of-meeting information to the real-time communication server, and all data, including the analysis results of the emotion engine, is sent to the server.

[0567] Step 8:

[0568] The server uses a generative AI model to analyze all text and emotional data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting.

[0569] Step 9:

[0570] The generated summary is stored in a database and sent from the server to all participants via email, etc. The summary is used as a review of the meeting and is also useful for preparing for the next meeting.

[0571] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0572] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0573] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0574] [Second embodiment]

[0575] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0576] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0577] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0578] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0579] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0580] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0581] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0582] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0583] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0584] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0585] In the smart glasses 214, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0586] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal."

[0587] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system is composed of a server, terminals, and users who play the following roles.

[0588] Meeting Settings

[0589] 1. User (Organizer):

[0590] Enter the purpose of the meeting, agenda, and participant list into the input form.

[0591] 2. Terminal:

[0592] The entered conference information is sent to the server.

[0593] 3. Server:

[0594] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[0595] Starting a meeting

[0596] 1. User:

[0597] Participants click the provided meeting link to join the meeting.

[0598] 2. Terminal:

[0599] Connect to the real-time communication server and send the user ID and conference ID.

[0600] Display meeting information on the UI.

[0601] 3. Server:

[0602] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[0603] Real-time typing for meeting management

[0604] 1. User:

[0605] Each participant will enter their opinions and suggestions in text in line with the agenda.

[0606] 2. Terminal:

[0607] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[0608] 3. Server:

[0609] A generative AI model monitors meeting progress and facilitates the conversation as needed.

[0610] Ending the meeting and generating a summary

[0611] 1. User:

[0612] After all agenda items have been decided, the organizer clicks the end button.

[0613] 2. Terminal:

[0614] The conference end information is sent to the server.

[0615] 3. Server:

[0616] A generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[0617] The generated summaries are stored in a database and sent to the participants.

[0618] Specific examples

[0619] Example configuration:

[0620] User (Organizer): "New product marketing strategy meeting"

[0621] Objective: "Determine a marketing strategy for a new product"

[0622] Agenda:

[0623] 1. Selecting your target market

[0624] 2. Advertising campaign planning

[0625] 3. Budget allocation

[0626] Progress example:

[0627] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[0628] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[0629] End example:

[0630] User (Organizer): Complete all agenda items and click the End Meeting button.

[0631] Server: The generative AI model summarizes the meeting content and sends the summary to all participants.

[0632] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[0633] The processing flow will be explained below.

[0634] Step 1:

[0635] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[0636] Step 2:

[0637] Terminal: Sends the entered meeting information to the server in JSON format.

[0638] Step 3:

[0639] Server: Stores the received conference information in a database.

[0640] Step 4:

[0641] Server: The generative AI model sets up the meeting details based on the received information.

[0642] Step 5:

[0643] User: Participants join the meeting by clicking the provided meeting link.

[0644] Step 6:

[0645] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[0646] Step 7:

[0647] Server: Retrieves conference information from the database based on the conference ID.

[0648] Step 8:

[0649] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[0650] Step 9:

[0651] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[0652] Step 10:

[0653] Terminal: The meeting assistant displays the meeting opening message on the screen.

[0654] Step 11:

[0655] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[0656] Step 12:

[0657] Terminal: Sends text data entered by the user to the real-time communication server.

[0658] Step 13:

[0659] Server: Broadcasts the received text data from each participant to all user terminals.

[0660] Step 14:

[0661] Terminal: Text received from other users is displayed on the UI in real time.

[0662] Step 15:

[0663] Server: A generative AI model monitors the progress of the meeting and facilitates the conversation as needed.

[0664] Step 16:

[0665] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[0666] Step 17:

[0667] Terminal: Sends conference end information to the real-time communication server.

[0668] Step 18:

[0669] Server: The generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[0670] Step 19:

[0671] Server: Stores the generated summaries in a database.

[0672] Step 20:

[0673] Server: Sends the generated summary to the participants.

[0674] Example 1

[0675] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0676] Meetings often involve unnecessary conversations, making it difficult to hold efficient discussions and make decisions. It also requires a lot of effort to quickly and accurately record the contents of meetings and provide a summary to participants. Furthermore, when multilingual support is required, communication becomes difficult when participants speak different languages.

[0677] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0678] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This allows for efficient conference progress, eliminates unnecessary conversation, and enables quick and accurate recording of conference content and provision of summaries, as well as multilingual support.

[0679] A "meeting organizer" is the person responsible for setting the purpose, agenda, and participant list for a meeting.

[0680] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate text and manage meeting progress.

[0681] An "input form" is an interface that allows a conference organizer to input information about the conference.

[0682] A "real-time communication server" is a server for transmitting and receiving data in real time.

[0683] A "user ID" is an identifier for uniquely identifying a user participating in a conference.

[0684] A "conference ID" is an identifier for uniquely identifying an individual conference.

[0685] An "agenda" is a list of items or topics to be discussed at a meeting.

[0686] "Text data" is character information input by the user.

[0687] "Display in real time" means that the input information is instantly displayed on the terminals of the other participants.

[0688] The "end conference button" is a button used by the conference organizer to instruct the end of the conference.

[0689] The "summary" is a shortened version of the meeting content generated based on the text data recorded during the meeting.

[0690] A "database" is a storage system for storing meeting details and generated summaries.

[0691] "Multilingual support" refers to translating the generated abstracts and meeting information into multiple languages.

[0692] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system operates in cooperation with the server, terminals, and users.

[0693] Meeting Settings

[0694] First, the user, who is the meeting organizer, enters the purpose, agenda, and participant list of the meeting into the system's input form. The device sends this information to the server. The server stores the received meeting information in a database, and a generative AI model (e.g., GPT-3) sets the meeting details.

[0695] Examples:

[0696] The organizer types in "New Product Marketing Strategy Meeting" and adds agenda items such as "Select Target Market," "Plan Advertising Campaign," and "Allocate Budget."

[0697] Starting a meeting

[0698] Users click on the designated meeting link to join the meeting. The device then connects to the real-time communication server and sends the user ID and meeting ID. The server retrieves the meeting information from the database, and the generative AI model displays the purpose and agenda of the meeting.

[0699] Examples:

[0700] Users click the link to access the meeting page, where the purpose and agenda of the meeting will be displayed on the screen.

[0701] Real-time typing for meeting management

[0702] Each participant, a user, enters their opinions and suggestions in text based on the agenda. The device sends this input text data to a real-time communication server, which also receives and displays the text data of other participants in real time. The server uses a generative AI model to monitor the progress of the conversation and facilitate it as needed.

[0703] Examples:

[0704] When a user types "advertising campaigns should be primarily conducted on social media," the content is instantly shared with other participants.

[0705] The server uses a generative AI model to display "May we move on to the next topic?"

[0706] Ending the meeting and generating a summary

[0707] When all agenda items have been completed, the organizer clicks the end button. The device sends the end-of-meeting information to the server. The server then uses a generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is stored in a database and sent to participants.

[0708] Examples:

[0709] After the meeting, the generative AI model generates a document summarizing the key points of the meeting and emails it to each participant.

[0710] Prompt Sentence Examples

[0711] "Please proceed to the next agenda item."

[0712] "Does anyone else have any opinions on this topic?"

[0713] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[0714] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0715] Step 1:

[0716] The user enters the meeting information.

[0717] The user enters the purpose of the meeting, the agenda, and a list of participants into an input form. The input includes the meeting name "New product marketing strategy meeting," and agenda items such as "Selection of target market," "Advertising campaign planning," and "Budget allocation." This information is generated as output in JSON format. The user enters this information into the input form on a web browser.

[0718] Step 2:

[0719] The terminal transmits the conference information.

[0720] The terminal sends the conference information entered in step 1 to the server. As input, it receives the JSON-formatted conference information entered by the user. As data processing, it converts this JSON data into an HTTP request and sends it as output to the server. Specifically, the terminal generates a request and sends a POST request to the server's API endpoint.

[0721] Step 3:

[0722] The server stores the meeting information and configures the generative AI model.

[0723] The server stores the meeting information received from the device in a database and configures the meeting details in the generative AI model. As input, it receives the JSON-formatted meeting information from the device. As data processing, it inserts this information into the database and configures the required settings for the generative AI model. As output, it notifies the generative AI model that the meeting details have been configured. The server executes an SQL query to store the information in the database and configures parameters through the AI ​​model's configuration interface.

[0724] Step 4:

[0725] A user joins a conference.

[0726] Users join a meeting by clicking a specified meeting link. The input is the act of clicking the meeting link. The output is the display of the meeting UI. The specific behavior is that the user clicks a link embedded in an email or calendar event they received, and the meeting page opens in a browser.

[0727] Step 5:

[0728] The terminal acquires and displays the conference information.

[0729] The terminal connects to the real-time communication server and sends the user ID and conference ID. It also displays the conference information on the UI. As input, it obtains the user ID and conference ID. As data processing, it connects to the real-time communication server using WebSocket and sends these IDs as authentication information. As output, the display of the conference information is completed. Specifically, the terminal displays the conference details information it received from the server in the UI component.

[0730] Step 6:

[0731] The server sends the meeting details.

[0732] The server retrieves meeting information from the database, and the generative AI model sends the meeting purpose and agenda to the device. As input, it receives the user ID and meeting ID. As data processing, it executes an SQL query to retrieve information from the database, and the generative AI model formats the meeting details. As output, it sends the generated meeting details to the device. Specifically, the server sends the output of the generative AI model to the device in real time.

[0733] Step 7:

[0734] The user inputs their opinion.

[0735] Each participant, a user, enters their opinions and suggestions in text along the agenda. As input, they enter the text of their opinions and suggestions related to the agenda. As output, that text data is generated. Specifically, they enter their opinions in the chat window and click the send button.

[0736] Step 8:

[0737] The device sends and receives text data.

[0738] The device sends the entered text data to the real-time communication server, receives text data from other participants, and displays it in real time. As input, it receives text data entered by the user. As data processing, it sends and receives this data via WebSocket, and as output, it displays it in the chat window in real time. In concrete terms, the device sends text data and immediately displays the received data.

[0739] Step 9:

[0740] The server facilitates the conversation.

[0741] The server uses a generative AI model to monitor the progress of the meeting and facilitate the conversation as needed. As input, it receives text data from each participant. As data processing, the generative AI model analyzes the content of the conversation and generates appropriate prompts. As output, a prompt for promotion is generated and sent to the terminal. Specifically, the generative AI model generates and displays a message such as "Shall we move on to the next agenda item?"

[0742] Step 10:

[0743] The user ends the conference.

[0744] When all agenda items have been completed, the user (organizer) clicks the end conference button. The input is the operation of clicking the end conference button. The output is the generation of conference end information. The specific operation is the user clicking the end conference button.

[0745] Step 11:

[0746] The terminal transmits the conference end information.

[0747] The terminal sends the conference end information to the server. As input, it receives information that the conference end button has been pressed. As data processing, it sends this information as an HTTP request. As output, it sends the conference end information to the server. Specifically, the terminal POSTs the conference end information to the server.

[0748] Step 12:

[0749] The server generates and transmits the meeting summary.

[0750] The server uses the generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is saved in a database and sent to participants. All text data is received as input. As data processing, the generative AI model analyzes the text data and automatically generates a summary. As output, the generated summary is saved and sent to participants by email. Specifically, the server uses the generative AI model to generate a summary and sends it to participants using the email sending function.

[0751] (Application example 1)

[0752] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0753] With conventional conferencing systems, meeting progress management was often done manually, resulting in long meetings and frequent unnecessary conversations. Furthermore, there were limited ways for participants to share their opinions in real time, making meetings less efficient. In brick-and-mortar stores in particular, poor communication between store operations and staff can lead to a decline in work efficiency. Furthermore, organizing the content of meetings after they have ended was time-consuming, resulting in delays in summarizing and sharing results.

[0754] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0755] In this invention, the server includes: a means for a meeting organizer to input the purpose, agenda, and participant list of the meeting; a means for using a generative AI model to set and save the details of the meeting based on the input information; a means for receiving text input from the user via real-time typing and broadcasting it to other participants; a means for the generative AI model to monitor the progress of the meeting and promote conversation; a means for automatically analyzing the meeting content and generating a summary after the meeting ends; a means for saving the generated summary and sending it to participants; a means for staff in a physical store to join the meeting from a smart device and input their opinions in real time; a means for the generative AI model to generate a prompt comment during the meeting to encourage the next agenda item; a means for displaying the generated prompt comment on the staff member's smart device; and a means for sending a summary generated based on the meeting content to the staff member's smart device, thereby enabling efficient progress of meetings and smooth communication in a physical store.

[0756] A "meeting organizer" is a user whose role is to enter the purpose, agenda, and participant list of a meeting.

[0757] A "generative AI model" is an artificial intelligence that sets meeting details based on input information, monitors progress, and facilitates conversations.

[0758] "Real-time typing" is the process by which a user types text during a meeting, which is then shared with other participants in real time.

[0759] "Broadcast" is a communication method that distributes text input from a user to all participants simultaneously.

[0760] A "promotion comment" is a text message sent by the generative AI model during a meeting to prompt the participant to move on to the next agenda item.

[0761] "Smart devices" is a general term for portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[0762] A "brick and mortar store" is a sales establishment that exists in a physical location and conducts business face-to-face with customers.

[0763] A "meeting summary" is a concise summary of what was discussed during the meeting.

[0764] "Means for inputting opinions in real time" refers to a method in which meeting participants use smart devices to instantly input their opinions in text.

[0765] "Means for automatic translation into multiple languages" refers to a function that automatically converts the generated summary into different languages.

[0766] This invention is a system that efficiently conducts meetings between staff in a physical store and eliminates unnecessary conversations.

[0767] Meeting Settings

[0768] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[0769] The terminal transmits the input conference information to the server.

[0770] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[0771] Starting a meeting

[0772] Users click on the provided meeting link to join the meeting.

[0773] The terminal connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI.

[0774] The server retrieves meeting information from a database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[0775] Real-time typing for meeting management

[0776] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[0777] The terminal transmits text data entered by the user to the real-time communication server, and receives text data from other participants and displays it in real time.

[0778] The server uses a generative AI model to monitor the progress of the meeting, generate prompts to move on to the next agenda item, and display them on participants' devices.

[0779] As a specific example, the following prompt sentence is generated: "Meeting agenda: Marketing strategy meeting for a new product. Participant's message: What should be the target market for the new product? We are considering a plan for an advertising campaign. Please generate a message to encourage whether we should proceed next."

[0780] Ending the meeting and generating a summary

[0781] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[0782] The terminal transmits the conference end information to the server.

[0783] The server uses a generative AI model to analyze all text data recorded during the meeting and generate a summary.

[0784] The generated summaries are stored in a database and sent to all participants.

[0785] It also provides a means for the generated summary to be automatically translated into multiple languages.

[0786] The system allows store staff to quickly and efficiently share opinions using smart devices (smartphones, tablets, etc.), eliminating unnecessary conversations. It also uses a generative AI model to monitor meeting progress in real time and generate appropriate prompts to guide the meeting.

[0787] A specific example is when staff at a brick-and-mortar store exchange opinions in real time based on the agenda for a "New Product Marketing Strategy Meeting." The generative AI model provides prompt comments such as "Shall we move on to the next agenda item?", allowing the meeting to proceed smoothly.

[0788] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0789] Processing Steps

[0790] Step 1:

[0791] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[0792] Input: Meeting purpose, agenda, participant list

[0793] Output: Entered meeting information

[0794] Specific action: A user enters text into a web or application form.

[0795] Step 2:

[0796] The terminal transmits the input conference information to the server.

[0797] Input: Meeting Information

[0798] Output: Meeting information transferred to the server

[0799] Specific operation: Obtain data from the input form and send it to the server via an HTTP request.

[0800] Step 3:

[0801] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[0802] Input: Meeting Information

[0803] Output: Meeting information stored in the database, configured meeting details

[0804] Specific operation: The received data is stored in a database via an SQL query, and the AI ​​model uses that data to complete meeting details (purpose and agenda).

[0805] Step 4:

[0806] User clicks on the provided meeting link to join the meeting.

[0807] Input: Meeting link

[0808] Output: Meeting participation confirmation

[0809] What happens: A user clicks on a link they received in an email or message to access the meeting join page.

[0810] Step 5:

[0811] The device connects to the real-time communication server and sends the user ID and conference ID. The conference information is displayed on the UI.

[0812] Input: User ID, Meeting ID

[0813] Output: Connection status, displayed meeting information

[0814] Specific operation: User information is sent via WebSocket or HTTP request, and conference information is obtained and displayed on the screen.

[0815] Step 6:

[0816] The server retrieves meeting information from the database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[0817] Input: Meeting ID

[0818] Output: Meeting information sent to the device and display content generated by the AI ​​model

[0819] How it works: Meeting information is retrieved through a database query and sent to the device. Based on that information, the AI ​​model displays the purpose and agenda in the UI.

[0820] Step 7:

[0821] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[0822] Input: User text input

[0823] Output: Real-time shared text data

[0824] Specific operation: The user enters opinions and suggestions in the text box, which are then sent to the real-time communication server.

[0825] Step 8:

[0826] The terminal transmits text data entered by the user to a real-time communication server, receives text data from other participants, and displays it in real time.

[0827] Input: User's text data

[0828] Output: Text data broadcast to other participants, text data displayed

[0829] Specific operation: Send data to other participants via a real-time communication protocol (e.g., WebSocket) and display the received data on the screen.

[0830] Step 9:

[0831] The server monitors the progress of the meeting using a generative AI model, generates prompts to move on to the next agenda item, and displays them on participants' devices.

[0832] Input: Meeting progress data

[0833] Output: Generated promotion comment, Displayed promotion comment

[0834] Specific operation: The AI ​​model analyzes the progress of the meeting, generates prompts such as "Shall we move on to the next agenda item?", and sends them to the device for display.

[0835] Step 10:

[0836] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[0837] Input: End meeting instruction

[0838] Output:Confirmation of end of meeting

[0839] Specific operation: The terminal captures the click event of the end conference button and sends an end instruction to the server.

[0840] Step 11:

[0841] The terminal transmits conference end information to the server.

[0842] Input: Meeting end information

[0843] Output: Finished information sent to the server

[0844] Specific operation: Sends conference end information to the server via HTTP request or WebSocket.

[0845] Step 12:

[0846] The server uses the generative AI model to analyze all the text data recorded during the meeting and generate a summary.

[0847] Input: All text data during the meeting

[0848] Output: Generated meeting summary

[0849] Specific operation: Analyzes text data and generates summaries using natural language processing techniques.

[0850] Step 13:

[0851] The server stores the generated summaries in a database and sends them to all participants, and also provides a means for automatic translation into multiple languages.

[0852] Input: Generated meeting summary

[0853] Output: Summary stored in database, multilingual summary sent

[0854] Specific actions: The generated summary is saved and sent to each participant via email or notification. If necessary, the summary is translated into multiple languages ​​using an automatic translation API.

[0855] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[0856] This invention is a system that uses a generative AI model and emotion engine to conduct meetings efficiently based on real-time typing and emotion information, eliminating unnecessary conversations. This system is composed of a server, terminals, and users who play the following roles.

[0857] Meeting Settings

[0858] 1. User (Organizer):

[0859] Enter the purpose of the meeting, agenda, and participant list into the input form.

[0860] 2. Terminal:

[0861] The entered conference information is sent to the server in JSON format.

[0862] 3. Server:

[0863] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[0864] Starting a meeting

[0865] 1. User:

[0866] Participants click the provided meeting link to join the meeting.

[0867] 2. Terminal:

[0868] Connect to the real-time communication server and send the user ID and conference ID.

[0869] Display the meeting information on the UI and activate the emotion engine.

[0870] 3. Server:

[0871] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[0872] The emotion engine analyzes each user's emotional data in real time and feeds the results back into the generative AI model.

[0873] Real-time typing for meeting progress and emotion analysis

[0874] 1. User:

[0875] Each participant will enter their opinions and suggestions in text in line with the agenda.

[0876] 2. Terminal:

[0877] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[0878] The emotional engine analyzes each participant's emotional state and displays it to other participants as appropriate.

[0879] 3. Server:

[0880] A generative AI model monitors the progress of the meeting and drives the conversation based on the user's emotional data.

[0881] For example, if the user is nervous, the generative AI model will insert questions or comments to help them relax.

[0882] Ending the meeting and generating a summary

[0883] 1. User:

[0884] After all agenda items have been decided, the organizer clicks the end button.

[0885] 2. Terminal:

[0886] The conference end information is transmitted to the real-time communication server.

[0887] Data including the analysis results of the emotion engine is sent to the server.

[0888] 3. Server:

[0889] A generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[0890] The summary includes not only the conclusions and key statements for each topic, but also the emotional state of the meeting.

[0891] The generated summaries are stored in a database and sent to the participants.

[0892] Specific examples

[0893] Example configuration:

[0894] User (Organizer): "New product marketing strategy meeting"

[0895] Objective: "Determine a marketing strategy for a new product"

[0896] Agenda:

[0897] 1. Selecting your target market

[0898] 2. Advertising campaign planning

[0899] 3. Budget allocation

[0900] Progress example:

[0901] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[0902] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[0903] Terminal: The emotion engine analyzes the emotions of each participant and displays relaxing comments to users who are nervous.

[0904] End example:

[0905] User (Organizer): Complete all agenda items and click the End Meeting button.

[0906] Server: The generative AI model summarizes the meeting content and sends the summary and sentiment analysis results to all participants.

[0907] In this way, the system of the present invention not only supports efficient and productive meeting progress and eliminates unnecessary conversations, but also improves the quality of meetings by utilizing emotional data.

[0908] The processing flow will be explained below.

[0909] Step 1:

[0910] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[0911] Step 2:

[0912] Terminal: Sends the entered meeting information to the server in JSON format.

[0913] Step 3:

[0914] Server: Stores the received conference information in a database.

[0915] Step 4:

[0916] Server: The generative AI model sets up the meeting details based on the received information.

[0917] Step 5:

[0918] User: Participants join the meeting by clicking the provided meeting link.

[0919] Step 6:

[0920] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[0921] Step 7:

[0922] Server: Retrieves conference information from the database based on the conference ID.

[0923] Step 8:

[0924] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[0925] Step 9:

[0926] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[0927] Step 10:

[0928] Terminal: The meeting assistant displays the meeting opening message on the screen.

[0929] Step 11:

[0930] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[0931] Step 12:

[0932] Terminal: Sends text data entered by the user to the real-time communication server.

[0933] Step 13:

[0934] Server: Broadcasts the received text data from each participant to all user terminals.

[0935] Step 14:

[0936] Terminal: Text received from other users is displayed on the UI in real time.

[0937] Step 15:

[0938] Terminal: The emotion engine analyzes the emotions of each participant and sends the results to the server in real time.

[0939] Step 16:

[0940] Server: Feedbacks emotional data from the emotion engine to the generative AI model and monitors the progress of the meeting.

[0941] Step 17:

[0942] Server: The generative AI model uses emotional data to insert comments and questions to encourage conversation.

[0943] Step 18:

[0944] Device: Analyzed emotion results are displayed in real time on the UI, providing feedback according to the situation.

[0945] Step 19:

[0946] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[0947] Step 20:

[0948] Terminal: Sends conference end information to the real-time communication server.

[0949] Step 21:

[0950] Server: The generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[0951] Step 22:

[0952] Server: Stores the generated summaries in a database.

[0953] Step 23:

[0954] Server: Sends summary results to participants.

[0955] Step 24:

[0956] Server: May automatically translate abstracts into multiple languages ​​and send each language version of the abstract to participants.

[0957] Example 2

[0958] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0959] The purpose of this invention is to improve the quality of meetings by efficiently conducting meetings, sharing information, and analyzing participants' emotional data. Conventional meeting systems have not adequately considered the efficiency of meeting progress or the emotional state of participants, resulting in problems such as participants losing concentration and discussion stalling. Furthermore, generating summaries of comments and discussions during meetings is often done manually, requiring time and effort.

[0960] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0961] In this invention, the server

[0962] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[0963] a means for configuring and saving meeting details based on input information using a generative AI model;

[0964] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[0965] means for analyzing the emotional state of each user in real time using an emotion engine;

[0966] A generative AI model monitors the progress of the meeting and uses user emotional data to facilitate the conversation.

[0967] means for automatically analyzing the contents of a meeting after the meeting and generating a summary from the text data and emotion data;

[0968] and means for storing and transmitting the generated summary to the participants.

[0969] This will enable meetings to proceed efficiently, improve the quality of meetings by utilizing participants' emotional data, and automatically generate summaries.

[0970] A "meeting organizer" is a person who is responsible for setting the purpose, agenda, and participant list of a meeting and managing the overall progress of the meeting.

[0971] A "generative AI model" is an artificial intelligence model that uses technologies such as natural language processing to assist in the progress of meetings, providing appropriate feedback and support for progress based on input information.

[0972] An "emotion engine" is a system that analyzes participants' emotional state in real time from voice, facial expressions, text, etc., and provides feedback on the analysis results.

[0973] "Real-time typing" refers to the process by which a user types text in real time, which is instantly broadcast to other participants.

[0974] "Broadcast" refers to the simultaneous transmission of data (e.g., a text message) from one user to multiple recipients in real time.

[0975] "Meeting progress" refers to the state of the meeting, showing how well it is progressing according to the agenda, the flow of comments, progress on the agenda, etc.

[0976] "User emotional data" refers to data that indicates the emotional state of a participant, and is obtained from voice tone, text content, facial expressions, etc.

[0977] A "summary" refers to a concise report containing the overall conclusions and important statements of the meeting, generated based on all text and emotion data recorded during the meeting.

[0978] A "real-time communication server" refers to a server that manages real-time data communication between users using WebSocket or similar.

[0979] This invention is a system that uses a generative AI model and emotion engine to conduct meetings in real time based on typing and emotion information, eliminating unnecessary conversations, in order to ensure effective and productive meeting proceedings. This system is composed of three entities: a server, a terminal, and a user.

[0980] Meeting Settings

[0981] User (organizer):

[0982] The user enters the purpose, agenda, and participant list of the meeting into a dedicated input form. For example, the user can set the purpose, such as "New product marketing strategy meeting," and each agenda item.

[0983] Device:

[0984] The meeting information entered by the user is converted into JSON format and sent to the server using an HTTP POST request.

[0985] server:

[0986] The server stores the received JSON data in a database such as MySQL. The server then uses a generative AI model to set up the meeting details based on the received information. The generative AI model automatically calculates the optimal order and time allocation for the meeting.

[0987] Starting a meeting

[0988] User:

[0989] Participants click on the designated meeting link and log in to the system using a web browser or dedicated application.

[0990] Device:

[0991] The device connects to a real-time communication server such as a WebSocket server and transmits the user ID and conference ID. After the connection is complete, the device displays the conference information on the UI and starts the emotion engine, which starts analyzing the user's emotions.

[0992] server:

[0993] The server retrieves meeting information from the database and sends it to the device. The generative AI model displays the meeting's purpose and agenda on the screen and supports the meeting's progress. The emotion engine analyzes each user's emotional data in real time and feeds the results back to the generative AI model.

[0994] Real-time typing for meeting progress and emotion analysis

[0995] User:

[0996] Each participant enters their opinions and suggestions in text form along the agenda, using UI components such as a chat box.

[0997] Device:

[0998] The device sends the text data entered by the user to a real-time communication server using WebSocket or similar, and also receives and displays the text data of other participants in real time. The device also analyzes the emotional state of each participant using an emotion engine and displays the results on the UI. For example, if a user is judged to be "tense," the device will display "Please relax."

[0999] server:

[1000] The server monitors the progress of the meeting using a generative AI model. Based on participants' comments and emotional data, the generative AI model suggests the next agenda item and necessary questions. For example, if the discussion stalls, the generative AI model will insert a question such as, "Do you have any more specific suggestions for the current issue?"

[1001] Ending the meeting and generating a summary

[1002] User:

[1003] After all agenda items have been decided, the organizer clicks the end button.

[1004] Device:

[1005] The terminal transmits the conference end information to the real-time communication server, and also transmits the final analysis result of the emotion engine to the server.

[1006] server:

[1007] The server uses a generative AI model to analyze all text and emotion data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting. The summaries are stored in a database and sent to participants in real time via email or app notifications.

[1008] Prompt Sentence Examples

[1009] "Please provide your opinion on selecting the target market for the new product marketing strategy meeting."

[1010] "May we move on to the next topic?"

[1011] "Do you have any more specific suggestions for the current issue?"

[1012] As a result, the system of the present invention supports efficient and productive conference progress, eliminates unnecessary conversations, and improves the quality of conferences by utilizing emotion data.

[1013] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1014] Step 1: Enter and submit meeting information

[1015] Input: The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[1016] How it works: The terminal converts the input data into JSON format.

[1017] Output: The terminal sends the converted JSON data to the server using an HTTP POST request.

[1018] Step 2: Save meeting information and configure it with a generative AI model

[1019] Input: JSON data sent from the terminal.

[1020] How it works: The server stores the received JSON data in a database such as MySQL. It then launches a generative AI model to set up the details of the meeting based on the received information. The generative AI model automatically calculates the optimal order of the meeting and time allocation.

[1021] Output: The configured meeting details are saved in the database.

[1022] Step 3: Share the meeting link and connect

[1023] Input: User clicks on the provided meeting link.

[1024] Operation: The device connects to a real-time communication server (such as a WebSocket server) and sends the user ID and conference ID.

[1025] Output: The connection is authenticated and the meeting information is displayed in the UI.

[1026] Step 4: Displaying meeting information and launching the emotion engine

[1027] Input: Meeting information retrieved from the server.

[1028] Operation: The device displays the received conference information on the UI and starts the emotion engine, which starts emotion analysis.

[1029] Output: The meeting information is displayed on the screen and the emotion engine starts working.

[1030] Step 5: Real-time typing and sending and receiving data

[1031] Input: The user types their opinion or suggestion into the text box.

[1032] Operation: The device sends the user's input data to the real-time communication server, and simultaneously receives text data from other participants and displays it in real time. The emotion engine analyzes the emotional state of each participant, and the results are reflected in the UI.

[1033] Output: The input text data is broadcast to other participants in real time, and the analyzed emotional information is displayed.

[1034] Step 6: Monitor and facilitate meeting progress

[1035] Input: User utterances and emotional state data.

[1036] How it works: The server monitors the progress of the meeting using a generative AI model. The generative AI model uses participants' comments and emotional data to suggest the next agenda item and necessary questions. For example, if the discussion stagnates, the generative AI model inserts appropriate questions to stimulate the conversation.

[1037] Output: A prompt message based on feedback from the generative AI model is displayed.

[1038] Step 7: Ending the meeting and sending data

[1039] Input: Organizer clicks end meeting button.

[1040] Operation: The terminal sends the conference end information to the real-time communication server, and also sends the final analysis results from the emotion engine to the server.

[1041] Output: Information on the end of the conference and the final analysis results are sent to the server.

[1042] Step 8: Generate and send the summary

[1043] Input: All text and emotion data recorded during the meeting.

[1044] How it works: The server uses a generative AI model to automatically generate a summary of the meeting content, including conclusions for each topic, important comments, and the emotional state of the meeting.

[1045] Output: The generated summaries are stored in a database and sent to participants via email or a dedicated app notification.

[1046] By following the above steps, this system is able to conduct a conference effectively and efficiently.

[1047] (Application example 2)

[1048] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1049] In modern factory operations, improving meeting efficiency and speeding up decision-making are important challenges. However, conventional methods make meetings slow and prone to unnecessary conversation. Furthermore, there is no way to grasp the emotional state of meeting participants and provide appropriate feedback, leading to tension and stress that reduces the quality of meetings. This invention aims to use a generative AI model and emotion engine to efficiently conduct factory meetings and improve the quality of meetings by taking into account the emotional state of participants.

[1050] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1051] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for receiving text input from the user via real-time typing and broadcasting it to other participants, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for analyzing the user's emotional state and providing feedback, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This enables the conference to proceed more efficiently and the emotional states of the participants to be managed appropriately.

[1052] A "meeting organizer" is a person in charge of setting up a meeting by entering the purpose, agenda, participant list, etc. of the meeting.

[1053] A "generative AI model" is a system that uses artificial intelligence to support detailed meeting setup and progress based on input information.

[1054] An "agenda" refers to the items or topics to be discussed at a meeting.

[1055] "Real-time typing" is a system in which participants enter their opinions and comments in text during a meeting, and the content is instantly transmitted to other participants.

[1056] "Broadcast" means sending specific information to multiple recipients simultaneously.

[1057] The "Emotion Engine" is a system that analyzes the emotional state of participants in real time and reflects the results in the progress of the meeting.

[1058] "Emotional state" refers to the emotional state, such as tension, joy, or fatigue, that participants feel during the meeting.

[1059] "Feedback" refers to providing appropriate responses or comments to participants based on the emotional data analyzed by the emotion engine.

[1060] A "summary" is a short summary of the entire meeting and its important points, generated after the meeting is over.

[1061] "Storage" means recording the generated data in a database or the like so that it can be referenced later.

[1062] "Transmit" means transmitting information via email or other means to provide the generated summary and sentiment analysis results to participants.

[1063] This invention is a system that efficiently facilitates factory meetings and improves their quality by taking into account the emotional state of participants. The system utilizes a generative AI model and an emotion engine to analyze real-time text input and emotional data to facilitate meetings.

[1064] Program processing and hardware / software used

[1065] The system includes the following components:

[1066] 1. Server:

[1067] Generative AI models, such as OpenAI GPT-3, are used to set meeting details and facilitate the conversation. They use input to set meeting details, monitor progress, and facilitate the conversation.

[1068] Database: MongoDB or similar is used to store conference setting information and generated summaries.

[1069] Real-time communication server: Supports real-time data exchange between users using WebSockets, etc.

[1070] 2. Terminal:

[1071] User interface: HTML, CSS, and JavaScript are used to set up meetings, input real-time information, display emotional states, and display meeting summaries.

[1072] Emotion engine: Analyzes the user's emotional state in real time and generates feedback using a custom analysis module or an existing emotion analysis API (e.g., Microsoft Azure Emotion API).

[1073] 3. User:

[1074] Organizer: Set up the meeting by entering the purpose, agenda, and participant list.

[1075] Participants: Enter thoughts and comments in real time during the meeting, which are broadcast to other participants and displayed in real time.

[1076] Specific examples

[1077] Conference Settings:

[1078] The user (organizer) inputs the purpose, agenda, and participant list of the meeting. For example, if the purpose is a "marketing strategy meeting for a new product," the agenda may include "selecting the target market," "planning the advertising campaign," and "allocating the budget."

[1079] Meeting proceedings:

[1080] The device connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI and the emotion engine is activated.

[1081] Each participant enters their opinions and suggestions in text based on the agenda, and the text data is displayed to other participants in real time. The emotion engine analyzes each participant's emotional state, and if the emotion analyzed is "tension," for example, the system displays a message such as "Please relax."

[1082] Conference Summary:

[1083] After the meeting, the generative AI model analyzes all the text and emotion data recorded during the meeting and generates a summary of the entire meeting, including the conclusions and key comments made on each topic, as well as the emotional state of the meeting.

[1084] The summaries will be stored in a database and communicated to participants.

[1085] Prompt Sentence Examples

[1086] Here are some examples of prompts:

[1087] Meeting Information:

[1088] Title: Improve work efficiency

[1089] Agenda:

[1090] 1. Review of work processes

[1091] 2. Reassignment of workers

[1092] 3. Introducing new tools

[1093] Real-time input:

[1094] User1: "I think there's a lot of overlap in our current workflow."

[1095] User 2: "I think introducing new tools will improve efficiency."

[1096] Prompt for GPT-3:

[1097] Summarize the conversation below and consider the sentiment data:

[1098] "I think there is a lot of overlap in the current work process." (Average)

[1099] "I think that introducing new tools will improve efficiency." (Optimistic)

[1100] As described above, the present invention can improve the efficiency of meetings and manage emotions, thereby improving the quality of meetings within a factory.

[1101] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1102] Step 1:

[1103] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list. The entered data is sent from the terminal to the server. The input data is sent to the server in JSON format and saved in the database as configuration information.

[1104] Step 2:

[1105] The server uses a generative AI model to set up detailed meetings based on the input meeting information. Specifically, it checks the purpose of the meeting and the order of the agenda, and creates an appropriate timetable. This information is also stored in a database.

[1106] Step 3:

[1107] When a conference starts, users (participants) click on the designated conference link from their devices to join the conference. The devices connect to the real-time communication server and send the user ID and conference ID. The server then sends the conference information to the devices and activates the emotion engine.

[1108] Step 4:

[1109] Each user inputs their opinions and suggestions in text format into their device, following the agenda. The input text data is sent to the real-time communication server and broadcast to the devices of other participants in real time. The input data is displayed immediately.

[1110] Step 5:

[1111] The emotion engine analyzes each user's input text and determines their emotional state. The device receives the analysis results and displays the user's emotional state (e.g., "tension" or "joy") on the screens of other participants. If necessary, it also displays a feedback message to encourage relaxation.

[1112] Step 6:

[1113] The server's generative AI model monitors the progress of the meeting and generates and displays text prompting the user to move on to the next agenda item as needed, such as "Shall we move on to the next item?"

[1114] Step 7:

[1115] After the meeting is over, the organizer clicks the end button. The device sends the end-of-meeting information to the real-time communication server, and all data, including the analysis results of the emotion engine, is sent to the server.

[1116] Step 8:

[1117] The server uses a generative AI model to analyze all text and emotional data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting.

[1118] Step 9:

[1119] The generated summary is stored in a database and sent from the server to all participants via email, etc. The summary is used as a review of the meeting and is also useful for preparing for the next meeting.

[1120] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1121] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1122] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1123] [Third embodiment]

[1124] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1125] 5, the data processing system 310 includes the data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[1126] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1127] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1128] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1129] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1130] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1131] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1132] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1133] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1134] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1135] Next, a description will be given of the identification process performed by the identification processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as the "server" and the headset type terminal 314 will be referred to as the "terminal."

[1136] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system is composed of a server, terminals, and users who play the following roles.

[1137] Meeting Settings

[1138] 1. User (Organizer):

[1139] Enter the purpose of the meeting, agenda, and participant list into the input form.

[1140] 2. Terminal:

[1141] The entered conference information is sent to the server.

[1142] 3. Server:

[1143] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[1144] Starting a meeting

[1145] 1. User:

[1146] Participants click the provided meeting link to join the meeting.

[1147] 2. Terminal:

[1148] Connect to the real-time communication server and send the user ID and conference ID.

[1149] Display meeting information on the UI.

[1150] 3. Server:

[1151] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[1152] Real-time typing for meeting management

[1153] 1. User:

[1154] Each participant will enter their opinions and suggestions in text in line with the agenda.

[1155] 2. Terminal:

[1156] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[1157] 3. Server:

[1158] A generative AI model monitors meeting progress and facilitates the conversation as needed.

[1159] Ending the meeting and generating a summary

[1160] 1. User:

[1161] After all agenda items have been decided, the organizer clicks the end button.

[1162] 2. Terminal:

[1163] The conference end information is sent to the server.

[1164] 3. Server:

[1165] A generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[1166] The generated summaries are stored in a database and sent to the participants.

[1167] Specific examples

[1168] Example configuration:

[1169] User (Organizer): "New product marketing strategy meeting"

[1170] Objective: "Determine a marketing strategy for a new product"

[1171] Agenda:

[1172] 1. Selecting your target market

[1173] 2. Advertising campaign planning

[1174] 3. Budget allocation

[1175] Progress example:

[1176] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[1177] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[1178] End example:

[1179] User (Organizer): Complete all agenda items and click the End Meeting button.

[1180] Server: The generative AI model summarizes the meeting content and sends the summary to all participants.

[1181] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[1182] The processing flow will be explained below.

[1183] Step 1:

[1184] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[1185] Step 2:

[1186] Terminal: Sends the entered meeting information to the server in JSON format.

[1187] Step 3:

[1188] Server: Stores the received conference information in a database.

[1189] Step 4:

[1190] Server: The generative AI model sets up the meeting details based on the received information.

[1191] Step 5:

[1192] User: Participants join the meeting by clicking the provided meeting link.

[1193] Step 6:

[1194] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[1195] Step 7:

[1196] Server: Retrieves conference information from the database based on the conference ID.

[1197] Step 8:

[1198] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[1199] Step 9:

[1200] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[1201] Step 10:

[1202] Terminal: The meeting assistant displays the meeting opening message on the screen.

[1203] Step 11:

[1204] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[1205] Step 12:

[1206] Terminal: Sends text data entered by the user to the real-time communication server.

[1207] Step 13:

[1208] Server: Broadcasts the received text data from each participant to all user terminals.

[1209] Step 14:

[1210] Terminal: Text received from other users is displayed on the UI in real time.

[1211] Step 15:

[1212] Server: A generative AI model monitors the progress of the meeting and facilitates the conversation as needed.

[1213] Step 16:

[1214] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[1215] Step 17:

[1216] Terminal: Sends conference end information to the real-time communication server.

[1217] Step 18:

[1218] Server: The generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[1219] Step 19:

[1220] Server: Stores the generated summaries in a database.

[1221] Step 20:

[1222] Server: Sends the generated summary to the participants.

[1223] Example 1

[1224] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1225] Meetings often involve unnecessary conversations, making it difficult to hold efficient discussions and make decisions. It also requires a lot of effort to quickly and accurately record the contents of meetings and provide a summary to participants. Furthermore, when multilingual support is required, communication becomes difficult when participants speak different languages.

[1226] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1227] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This allows for efficient conference progress, eliminates unnecessary conversation, and enables quick and accurate recording of conference content and provision of summaries, as well as multilingual support.

[1228] A "meeting organizer" is the person responsible for setting the purpose, agenda, and participant list for a meeting.

[1229] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate text and manage meeting progress.

[1230] An "input form" is an interface that allows a conference organizer to input information about the conference.

[1231] A "real-time communication server" is a server for transmitting and receiving data in real time.

[1232] A "user ID" is an identifier for uniquely identifying a user participating in a conference.

[1233] A "conference ID" is an identifier for uniquely identifying an individual conference.

[1234] An "agenda" is a list of items or topics to be discussed at a meeting.

[1235] "Text data" is character information input by the user.

[1236] "Display in real time" means that the input information is instantly displayed on the terminals of the other participants.

[1237] The "end conference button" is a button used by the conference organizer to instruct the end of the conference.

[1238] The "summary" is a shortened version of the meeting content generated based on the text data recorded during the meeting.

[1239] A "database" is a storage system for storing meeting details and generated summaries.

[1240] "Multilingual support" refers to translating the generated abstracts and meeting information into multiple languages.

[1241] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system operates in cooperation with the server, terminals, and users.

[1242] Meeting Settings

[1243] First, the user, who is the meeting organizer, enters the purpose, agenda, and participant list of the meeting into the system's input form. The device sends this information to the server. The server stores the received meeting information in a database, and a generative AI model (e.g., GPT-3) sets the meeting details.

[1244] Examples:

[1245] The organizer types in "New Product Marketing Strategy Meeting" and adds agenda items such as "Select Target Market," "Plan Advertising Campaign," and "Allocate Budget."

[1246] Starting a meeting

[1247] Users click on the designated meeting link to join the meeting. The device then connects to the real-time communication server and sends the user ID and meeting ID. The server retrieves the meeting information from the database, and the generative AI model displays the purpose and agenda of the meeting.

[1248] Examples:

[1249] Users click the link to access the meeting page, where the purpose and agenda of the meeting will be displayed on the screen.

[1250] Real-time typing for meeting management

[1251] Each participant, a user, enters their opinions and suggestions in text based on the agenda. The device sends this input text data to a real-time communication server, which also receives and displays the text data of other participants in real time. The server uses a generative AI model to monitor the progress of the conversation and facilitate it as needed.

[1252] Examples:

[1253] When a user types "advertising campaigns should be primarily conducted on social media," the content is instantly shared with other participants.

[1254] The server uses a generative AI model to display "May we move on to the next topic?"

[1255] Ending the meeting and generating a summary

[1256] When all agenda items have been completed, the organizer clicks the end button. The device sends the end-of-meeting information to the server. The server then uses a generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is stored in a database and sent to participants.

[1257] Examples:

[1258] After the meeting, the generative AI model generates a document summarizing the key points of the meeting and emails it to each participant.

[1259] Prompt Sentence Examples

[1260] "Please proceed to the next agenda item."

[1261] "Does anyone else have any opinions on this topic?"

[1262] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[1263] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1264] Step 1:

[1265] The user enters the meeting information.

[1266] The user enters the purpose of the meeting, the agenda, and a list of participants into an input form. The input includes the meeting name "New product marketing strategy meeting," and agenda items such as "Selection of target market," "Advertising campaign planning," and "Budget allocation." This information is generated as output in JSON format. The user enters this information into the input form on a web browser.

[1267] Step 2:

[1268] The terminal transmits the conference information.

[1269] The terminal sends the conference information entered in step 1 to the server. As input, it receives the JSON-formatted conference information entered by the user. As data processing, it converts this JSON data into an HTTP request and sends it as output to the server. Specifically, the terminal generates a request and sends a POST request to the server's API endpoint.

[1270] Step 3:

[1271] The server stores the meeting information and configures the generative AI model.

[1272] The server stores the meeting information received from the device in a database and configures the meeting details in the generative AI model. As input, it receives the JSON-formatted meeting information from the device. As data processing, it inserts this information into the database and configures the required settings for the generative AI model. As output, it notifies the generative AI model that the meeting details have been configured. The server executes an SQL query to store the information in the database and configures parameters through the AI ​​model's configuration interface.

[1273] Step 4:

[1274] A user joins a conference.

[1275] Users join a meeting by clicking a specified meeting link. The input is the act of clicking the meeting link. The output is the display of the meeting UI. The specific behavior is that the user clicks a link embedded in an email or calendar event they received, and the meeting page opens in a browser.

[1276] Step 5:

[1277] The terminal acquires and displays the conference information.

[1278] The terminal connects to the real-time communication server and sends the user ID and conference ID. It also displays the conference information on the UI. As input, it obtains the user ID and conference ID. As data processing, it connects to the real-time communication server using WebSocket and sends these IDs as authentication information. As output, the display of the conference information is completed. Specifically, the terminal displays the conference details information it received from the server in the UI component.

[1279] Step 6:

[1280] The server sends the meeting details.

[1281] The server retrieves meeting information from the database, and the generative AI model sends the meeting purpose and agenda to the device. As input, it receives the user ID and meeting ID. As data processing, it executes an SQL query to retrieve information from the database, and the generative AI model formats the meeting details. As output, it sends the generated meeting details to the device. Specifically, the server sends the output of the generative AI model to the device in real time.

[1282] Step 7:

[1283] The user inputs their opinion.

[1284] Each participant, a user, enters their opinions and suggestions in text along the agenda. As input, they enter the text of their opinions and suggestions related to the agenda. As output, that text data is generated. Specifically, they enter their opinions in the chat window and click the send button.

[1285] Step 8:

[1286] The device sends and receives text data.

[1287] The device sends the entered text data to the real-time communication server, receives text data from other participants, and displays it in real time. As input, it receives text data entered by the user. As data processing, it sends and receives this data via WebSocket, and as output, it displays it in the chat window in real time. In concrete terms, the device sends text data and immediately displays the received data.

[1288] Step 9:

[1289] The server facilitates the conversation.

[1290] The server uses a generative AI model to monitor the progress of the meeting and facilitate the conversation as needed. As input, it receives text data from each participant. As data processing, the generative AI model analyzes the content of the conversation and generates appropriate prompts. As output, a prompt for promotion is generated and sent to the terminal. Specifically, the generative AI model generates and displays a message such as "Shall we move on to the next agenda item?"

[1291] Step 10:

[1292] The user ends the conference.

[1293] When all agenda items have been completed, the user (organizer) clicks the end conference button. The input is the operation of clicking the end conference button. The output is the generation of conference end information. The specific operation is the user clicking the end conference button.

[1294] Step 11:

[1295] The terminal transmits the conference end information.

[1296] The terminal sends the conference end information to the server. As input, it receives information that the conference end button has been pressed. As data processing, it sends this information as an HTTP request. As output, it sends the conference end information to the server. Specifically, the terminal POSTs the conference end information to the server.

[1297] Step 12:

[1298] The server generates and transmits the meeting summary.

[1299] The server uses the generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is saved in a database and sent to participants. All text data is received as input. As data processing, the generative AI model analyzes the text data and automatically generates a summary. As output, the generated summary is saved and sent to participants by email. Specifically, the server uses the generative AI model to generate a summary and sends it to participants using the email sending function.

[1300] (Application example 1)

[1301] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1302] With conventional conferencing systems, meeting progress management was often done manually, resulting in long meetings and frequent unnecessary conversations. Furthermore, there were limited ways for participants to share their opinions in real time, making meetings less efficient. In brick-and-mortar stores in particular, poor communication between store operations and staff can lead to a decline in work efficiency. Furthermore, organizing the content of meetings after they have ended was time-consuming, resulting in delays in summarizing and sharing results.

[1303] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1304] In this invention, the server includes: a means for a meeting organizer to input the purpose, agenda, and participant list of the meeting; a means for using a generative AI model to set and save the details of the meeting based on the input information; a means for receiving text input from the user via real-time typing and broadcasting it to other participants; a means for the generative AI model to monitor the progress of the meeting and promote conversation; a means for automatically analyzing the meeting content and generating a summary after the meeting ends; a means for saving the generated summary and sending it to participants; a means for staff in a physical store to join the meeting from a smart device and input their opinions in real time; a means for the generative AI model to generate a prompt comment during the meeting to encourage the next agenda item; a means for displaying the generated prompt comment on the staff member's smart device; and a means for sending a summary generated based on the meeting content to the staff member's smart device, thereby enabling efficient progress of meetings and smooth communication in a physical store.

[1305] A "meeting organizer" is a user whose role is to enter the purpose, agenda, and participant list of a meeting.

[1306] A "generative AI model" is an artificial intelligence that sets meeting details based on input information, monitors progress, and facilitates conversations.

[1307] "Real-time typing" is the process by which a user types text during a meeting, which is then shared with other participants in real time.

[1308] "Broadcast" is a communication method that distributes text input from a user to all participants simultaneously.

[1309] A "promotion comment" is a text message sent by the generative AI model during a meeting to prompt the participant to move on to the next agenda item.

[1310] "Smart devices" is a general term for portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[1311] A "brick and mortar store" is a sales establishment that exists in a physical location and conducts business face-to-face with customers.

[1312] A "meeting summary" is a concise summary of what was discussed during the meeting.

[1313] "Means for inputting opinions in real time" refers to a method in which meeting participants use smart devices to instantly input their opinions in text.

[1314] "Means for automatic translation into multiple languages" refers to a function that automatically converts the generated summary into different languages.

[1315] This invention is a system that efficiently conducts meetings between staff in a physical store and eliminates unnecessary conversations.

[1316] Meeting Settings

[1317] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[1318] The terminal transmits the input conference information to the server.

[1319] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[1320] Starting a meeting

[1321] Users click on the provided meeting link to join the meeting.

[1322] The terminal connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI.

[1323] The server retrieves meeting information from a database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[1324] Real-time typing for meeting management

[1325] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[1326] The terminal transmits text data entered by the user to the real-time communication server, and receives text data from other participants and displays it in real time.

[1327] The server uses a generative AI model to monitor the progress of the meeting, generate prompts to move on to the next agenda item, and display them on participants' devices.

[1328] As a specific example, the following prompt sentence is generated: "Meeting agenda: Marketing strategy meeting for a new product. Participant's message: What should be the target market for the new product? We are considering a plan for an advertising campaign. Please generate a message to encourage whether we should proceed next."

[1329] Ending the meeting and generating a summary

[1330] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[1331] The terminal transmits the conference end information to the server.

[1332] The server uses a generative AI model to analyze all text data recorded during the meeting and generate a summary.

[1333] The generated summaries are stored in a database and sent to all participants.

[1334] It also provides a means for the generated summary to be automatically translated into multiple languages.

[1335] The system allows store staff to quickly and efficiently share opinions using smart devices (smartphones, tablets, etc.), eliminating unnecessary conversations. It also uses a generative AI model to monitor meeting progress in real time and generate appropriate prompts to guide the meeting.

[1336] A specific example is when staff at a brick-and-mortar store exchange opinions in real time based on the agenda for a "New Product Marketing Strategy Meeting." The generative AI model provides prompt comments such as "Shall we move on to the next agenda item?", allowing the meeting to proceed smoothly.

[1337] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1338] Processing Steps

[1339] Step 1:

[1340] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[1341] Input: Meeting purpose, agenda, participant list

[1342] Output: Entered meeting information

[1343] Specific action: A user enters text into a web or application form.

[1344] Step 2:

[1345] The terminal transmits the input conference information to the server.

[1346] Input: Meeting Information

[1347] Output: Meeting information transferred to the server

[1348] Specific operation: Obtain data from the input form and send it to the server via an HTTP request.

[1349] Step 3:

[1350] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[1351] Input: Meeting Information

[1352] Output: Meeting information stored in the database, configured meeting details

[1353] Specific operation: The received data is stored in a database via an SQL query, and the AI ​​model uses that data to complete meeting details (purpose and agenda).

[1354] Step 4:

[1355] User clicks on the provided meeting link to join the meeting.

[1356] Input: Meeting link

[1357] Output: Meeting participation confirmation

[1358] What happens: A user clicks on a link they received in an email or message to access the meeting join page.

[1359] Step 5:

[1360] The device connects to the real-time communication server and sends the user ID and conference ID. The conference information is displayed on the UI.

[1361] Input: User ID, Meeting ID

[1362] Output: Connection status, displayed meeting information

[1363] Specific operation: User information is sent via WebSocket or HTTP request, and conference information is obtained and displayed on the screen.

[1364] Step 6:

[1365] The server retrieves meeting information from the database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[1366] Input: Meeting ID

[1367] Output: Meeting information sent to the device and display content generated by the AI ​​model

[1368] How it works: Meeting information is retrieved through a database query and sent to the device. Based on that information, the AI ​​model displays the purpose and agenda in the UI.

[1369] Step 7:

[1370] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[1371] Input: User text input

[1372] Output: Real-time shared text data

[1373] Specific operation: The user enters opinions and suggestions in the text box, which are then sent to the real-time communication server.

[1374] Step 8:

[1375] The terminal transmits text data entered by the user to a real-time communication server, receives text data from other participants, and displays it in real time.

[1376] Input: User's text data

[1377] Output: Text data broadcast to other participants, text data displayed

[1378] Specific operation: Send data to other participants via a real-time communication protocol (e.g., WebSocket) and display the received data on the screen.

[1379] Step 9:

[1380] The server monitors the progress of the meeting using a generative AI model, generates prompts to move on to the next agenda item, and displays them on participants' devices.

[1381] Input: Meeting progress data

[1382] Output: Generated promotion comment, Displayed promotion comment

[1383] Specific operation: The AI ​​model analyzes the progress of the meeting, generates prompts such as "Shall we move on to the next agenda item?", and sends them to the device for display.

[1384] Step 10:

[1385] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[1386] Input: End meeting instruction

[1387] Output:Confirmation of end of meeting

[1388] Specific operation: The terminal captures the click event of the end conference button and sends an end instruction to the server.

[1389] Step 11:

[1390] The terminal transmits conference end information to the server.

[1391] Input: Meeting end information

[1392] Output: Finished information sent to the server

[1393] Specific operation: Sends conference end information to the server via HTTP request or WebSocket.

[1394] Step 12:

[1395] The server uses the generative AI model to analyze all the text data recorded during the meeting and generate a summary.

[1396] Input: All text data during the meeting

[1397] Output: Generated meeting summary

[1398] Specific operation: Analyzes text data and generates summaries using natural language processing techniques.

[1399] Step 13:

[1400] The server stores the generated summaries in a database and sends them to all participants, and also provides a means for automatic translation into multiple languages.

[1401] Input: Generated meeting summary

[1402] Output: Summary stored in database, multilingual summary sent

[1403] Specific actions: The generated summary is saved and sent to each participant via email or notification. If necessary, the summary is translated into multiple languages ​​using an automatic translation API.

[1404] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1405] This invention is a system that uses a generative AI model and emotion engine to conduct meetings efficiently based on real-time typing and emotion information, eliminating unnecessary conversations. This system is composed of a server, terminals, and users who play the following roles.

[1406] Meeting Settings

[1407] 1. User (Organizer):

[1408] Enter the purpose of the meeting, agenda, and participant list into the input form.

[1409] 2. Terminal:

[1410] The entered conference information is sent to the server in JSON format.

[1411] 3. Server:

[1412] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[1413] Starting a meeting

[1414] 1. User:

[1415] Participants click the provided meeting link to join the meeting.

[1416] 2. Terminal:

[1417] Connect to the real-time communication server and send the user ID and conference ID.

[1418] Display the meeting information on the UI and activate the emotion engine.

[1419] 3. Server:

[1420] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[1421] The emotion engine analyzes each user's emotional data in real time and feeds the results back into the generative AI model.

[1422] Real-time typing for meeting progress and emotion analysis

[1423] 1. User:

[1424] Each participant will enter their opinions and suggestions in text in line with the agenda.

[1425] 2. Terminal:

[1426] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[1427] The emotional engine analyzes each participant's emotional state and displays it to other participants as appropriate.

[1428] 3. Server:

[1429] A generative AI model monitors the progress of the meeting and drives the conversation based on the user's emotional data.

[1430] For example, if the user is nervous, the generative AI model will insert questions or comments to help them relax.

[1431] Ending the meeting and generating a summary

[1432] 1. User:

[1433] After all agenda items have been decided, the organizer clicks the end button.

[1434] 2. Terminal:

[1435] The conference end information is transmitted to the real-time communication server.

[1436] Data including the analysis results of the emotion engine is sent to the server.

[1437] 3. Server:

[1438] A generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[1439] The summary includes not only the conclusions and key statements for each topic, but also the emotional state of the meeting.

[1440] The generated summaries are stored in a database and sent to the participants.

[1441] Specific examples

[1442] Example configuration:

[1443] User (Organizer): "New product marketing strategy meeting"

[1444] Objective: "Determine a marketing strategy for a new product"

[1445] Agenda:

[1446] 1. Selecting your target market

[1447] 2. Advertising campaign planning

[1448] 3. Budget allocation

[1449] Progress example:

[1450] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[1451] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[1452] Terminal: The emotion engine analyzes the emotions of each participant and displays relaxing comments to users who are nervous.

[1453] End example:

[1454] User (Organizer): Complete all agenda items and click the End Meeting button.

[1455] Server: The generative AI model summarizes the meeting content and sends the summary and sentiment analysis results to all participants.

[1456] In this way, the system of the present invention not only supports efficient and productive meeting progress and eliminates unnecessary conversations, but also improves the quality of meetings by utilizing emotional data.

[1457] The processing flow will be explained below.

[1458] Step 1:

[1459] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[1460] Step 2:

[1461] Terminal: Sends the entered meeting information to the server in JSON format.

[1462] Step 3:

[1463] Server: Stores the received conference information in a database.

[1464] Step 4:

[1465] Server: The generative AI model sets up the meeting details based on the received information.

[1466] Step 5:

[1467] User: Participants join the meeting by clicking the provided meeting link.

[1468] Step 6:

[1469] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[1470] Step 7:

[1471] Server: Retrieves conference information from the database based on the conference ID.

[1472] Step 8:

[1473] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[1474] Step 9:

[1475] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[1476] Step 10:

[1477] Terminal: The meeting assistant displays the meeting opening message on the screen.

[1478] Step 11:

[1479] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[1480] Step 12:

[1481] Terminal: Sends text data entered by the user to the real-time communication server.

[1482] Step 13:

[1483] Server: Broadcasts the received text data from each participant to all user terminals.

[1484] Step 14:

[1485] Terminal: Text received from other users is displayed on the UI in real time.

[1486] Step 15:

[1487] Terminal: The emotion engine analyzes the emotions of each participant and sends the results to the server in real time.

[1488] Step 16:

[1489] Server: Feedbacks emotional data from the emotion engine to the generative AI model and monitors the progress of the meeting.

[1490] Step 17:

[1491] Server: The generative AI model uses emotional data to insert comments and questions to encourage conversation.

[1492] Step 18:

[1493] Device: Analyzed emotion results are displayed in real time on the UI, providing feedback according to the situation.

[1494] Step 19:

[1495] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[1496] Step 20:

[1497] Terminal: Sends conference end information to the real-time communication server.

[1498] Step 21:

[1499] Server: The generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[1500] Step 22:

[1501] Server: Stores the generated summaries in a database.

[1502] Step 23:

[1503] Server: Sends summary results to participants.

[1504] Step 24:

[1505] Server: May automatically translate abstracts into multiple languages ​​and send each language version of the abstract to participants.

[1506] Example 2

[1507] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1508] The purpose of this invention is to improve the quality of meetings by efficiently conducting meetings, sharing information, and analyzing participants' emotional data. Conventional meeting systems have not adequately considered the efficiency of meeting progress or the emotional state of participants, resulting in problems such as participants losing concentration and discussion stalling. Furthermore, generating summaries of comments and discussions during meetings is often done manually, requiring time and effort.

[1509] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1510] In this invention, the server

[1511] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[1512] a means for configuring and saving meeting details based on input information using a generative AI model;

[1513] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[1514] means for analyzing the emotional state of each user in real time using an emotion engine;

[1515] A generative AI model monitors the progress of the meeting and uses user emotional data to facilitate the conversation.

[1516] means for automatically analyzing the contents of a meeting after the meeting and generating a summary from the text data and emotion data;

[1517] and means for storing and transmitting the generated summary to the participants.

[1518] This will enable meetings to proceed efficiently, improve the quality of meetings by utilizing participants' emotional data, and automatically generate summaries.

[1519] A "meeting organizer" is a person who is responsible for setting the purpose, agenda, and participant list of a meeting and managing the overall progress of the meeting.

[1520] A "generative AI model" is an artificial intelligence model that uses technologies such as natural language processing to assist in the progress of meetings, providing appropriate feedback and support for progress based on input information.

[1521] An "emotion engine" is a system that analyzes participants' emotional state in real time from voice, facial expressions, text, etc., and provides feedback on the analysis results.

[1522] "Real-time typing" refers to the process by which a user types text in real time, which is instantly broadcast to other participants.

[1523] "Broadcast" refers to the simultaneous transmission of data (e.g., a text message) from one user to multiple recipients in real time.

[1524] "Meeting progress" refers to the state of the meeting, showing how well it is progressing according to the agenda, the flow of comments, progress on the agenda, etc.

[1525] "User emotional data" refers to data that indicates the emotional state of a participant, and is obtained from voice tone, text content, facial expressions, etc.

[1526] A "summary" refers to a concise report containing the overall conclusions and important statements of the meeting, generated based on all text and emotion data recorded during the meeting.

[1527] A "real-time communication server" refers to a server that manages real-time data communication between users using WebSocket or similar.

[1528] This invention is a system that uses a generative AI model and emotion engine to conduct meetings in real time based on typing and emotion information, eliminating unnecessary conversations, in order to ensure effective and productive meeting proceedings. This system is composed of three entities: a server, a terminal, and a user.

[1529] Meeting Settings

[1530] User (organizer):

[1531] The user enters the purpose, agenda, and participant list of the meeting into a dedicated input form. For example, the user can set the purpose, such as "New product marketing strategy meeting," and each agenda item.

[1532] Device:

[1533] The meeting information entered by the user is converted into JSON format and sent to the server using an HTTP POST request.

[1534] server:

[1535] The server stores the received JSON data in a database such as MySQL. The server then uses a generative AI model to set up the meeting details based on the received information. The generative AI model automatically calculates the optimal order and time allocation for the meeting.

[1536] Starting a meeting

[1537] User:

[1538] Participants click on the designated meeting link and log in to the system using a web browser or dedicated application.

[1539] Device:

[1540] The device connects to a real-time communication server such as a WebSocket server and transmits the user ID and conference ID. After the connection is complete, the device displays the conference information on the UI and starts the emotion engine, which starts analyzing the user's emotions.

[1541] server:

[1542] The server retrieves meeting information from the database and sends it to the device. The generative AI model displays the meeting's purpose and agenda on the screen and supports the meeting's progress. The emotion engine analyzes each user's emotional data in real time and feeds the results back to the generative AI model.

[1543] Real-time typing for meeting progress and emotion analysis

[1544] User:

[1545] Each participant enters their opinions and suggestions in text form along the agenda, using UI components such as a chat box.

[1546] Device:

[1547] The device sends the text data entered by the user to a real-time communication server using WebSocket or similar, and also receives and displays the text data of other participants in real time. The device also analyzes the emotional state of each participant using an emotion engine and displays the results on the UI. For example, if a user is judged to be "tense," the device will display "Please relax."

[1548] server:

[1549] The server monitors the progress of the meeting using a generative AI model. Based on participants' comments and emotional data, the generative AI model suggests the next agenda item and necessary questions. For example, if the discussion stalls, the generative AI model will insert a question such as, "Do you have any more specific suggestions for the current issue?"

[1550] Ending the meeting and generating a summary

[1551] User:

[1552] After all agenda items have been decided, the organizer clicks the end button.

[1553] Device:

[1554] The terminal transmits the conference end information to the real-time communication server, and also transmits the final analysis result of the emotion engine to the server.

[1555] server:

[1556] The server uses a generative AI model to analyze all text and emotion data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting. The summaries are stored in a database and sent to participants in real time via email or app notifications.

[1557] Prompt Sentence Examples

[1558] "Please provide your opinion on selecting the target market for the new product marketing strategy meeting."

[1559] "May we move on to the next topic?"

[1560] "Do you have any more specific suggestions for the current issue?"

[1561] As a result, the system of the present invention supports efficient and productive conference progress, eliminates unnecessary conversations, and improves the quality of conferences by utilizing emotion data.

[1562] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1563] Step 1: Enter and submit meeting information

[1564] Input: The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[1565] How it works: The terminal converts the input data into JSON format.

[1566] Output: The terminal sends the converted JSON data to the server using an HTTP POST request.

[1567] Step 2: Save meeting information and configure it with a generative AI model

[1568] Input: JSON data sent from the terminal.

[1569] How it works: The server stores the received JSON data in a database such as MySQL. It then launches a generative AI model to set up the details of the meeting based on the received information. The generative AI model automatically calculates the optimal order of the meeting and time allocation.

[1570] Output: The configured meeting details are saved in the database.

[1571] Step 3: Share the meeting link and connect

[1572] Input: User clicks on the provided meeting link.

[1573] Operation: The device connects to a real-time communication server (such as a WebSocket server) and sends the user ID and conference ID.

[1574] Output: The connection is authenticated and the meeting information is displayed in the UI.

[1575] Step 4: Displaying meeting information and launching the emotion engine

[1576] Input: Meeting information retrieved from the server.

[1577] Operation: The device displays the received conference information on the UI and starts the emotion engine, which starts emotion analysis.

[1578] Output: The meeting information is displayed on the screen and the emotion engine starts working.

[1579] Step 5: Real-time typing and sending and receiving data

[1580] Input: The user types their opinion or suggestion into the text box.

[1581] Operation: The device sends the user's input data to the real-time communication server, and simultaneously receives text data from other participants and displays it in real time. The emotion engine analyzes the emotional state of each participant, and the results are reflected in the UI.

[1582] Output: The input text data is broadcast to other participants in real time, and the analyzed emotional information is displayed.

[1583] Step 6: Monitor and facilitate meeting progress

[1584] Input: User utterances and emotional state data.

[1585] How it works: The server monitors the progress of the meeting using a generative AI model. The generative AI model uses participants' comments and emotional data to suggest the next agenda item and necessary questions. For example, if the discussion stagnates, the generative AI model inserts appropriate questions to stimulate the conversation.

[1586] Output: A prompt message based on feedback from the generative AI model is displayed.

[1587] Step 7: Ending the meeting and sending data

[1588] Input: Organizer clicks end meeting button.

[1589] Operation: The terminal sends the conference end information to the real-time communication server, and also sends the final analysis results from the emotion engine to the server.

[1590] Output: Information on the end of the conference and the final analysis results are sent to the server.

[1591] Step 8: Generate and send the summary

[1592] Input: All text and emotion data recorded during the meeting.

[1593] How it works: The server uses a generative AI model to automatically generate a summary of the meeting content, including conclusions for each topic, important comments, and the emotional state of the meeting.

[1594] Output: The generated summaries are stored in a database and sent to participants via email or a dedicated app notification.

[1595] By following the above steps, this system is able to conduct a conference effectively and efficiently.

[1596] (Application example 2)

[1597] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1598] In modern factory operations, improving meeting efficiency and speeding up decision-making are important challenges. However, conventional methods make meetings slow and prone to unnecessary conversation. Furthermore, there is no way to grasp the emotional state of meeting participants and provide appropriate feedback, leading to tension and stress that reduces the quality of meetings. This invention aims to use a generative AI model and emotion engine to efficiently conduct factory meetings and improve the quality of meetings by taking into account the emotional state of participants.

[1599] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1600] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for receiving text input from the user via real-time typing and broadcasting it to other participants, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for analyzing the user's emotional state and providing feedback, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This enables the conference to proceed more efficiently and the emotional states of the participants to be managed appropriately.

[1601] A "meeting organizer" is a person in charge of setting up a meeting by entering the purpose, agenda, participant list, etc. of the meeting.

[1602] A "generative AI model" is a system that uses artificial intelligence to support detailed meeting setup and progress based on input information.

[1603] An "agenda" refers to the items or topics to be discussed at a meeting.

[1604] "Real-time typing" is a system in which participants enter their opinions and comments in text during a meeting, and the content is instantly transmitted to other participants.

[1605] "Broadcast" means sending specific information to multiple recipients simultaneously.

[1606] The "Emotion Engine" is a system that analyzes the emotional state of participants in real time and reflects the results in the progress of the meeting.

[1607] "Emotional state" refers to the emotional state, such as tension, joy, or fatigue, that participants feel during the meeting.

[1608] "Feedback" refers to providing appropriate responses or comments to participants based on the emotional data analyzed by the emotion engine.

[1609] A "summary" is a short summary of the entire meeting and its important points, generated after the meeting is over.

[1610] "Storage" means recording the generated data in a database or the like so that it can be referenced later.

[1611] "Transmit" means transmitting information via email or other means to provide the generated summary and sentiment analysis results to participants.

[1612] This invention is a system that efficiently facilitates factory meetings and improves their quality by taking into account the emotional state of participants. The system utilizes a generative AI model and an emotion engine to analyze real-time text input and emotional data to facilitate meetings.

[1613] Program processing and hardware / software used

[1614] The system includes the following components:

[1615] 1. Server:

[1616] Generative AI models, such as OpenAI GPT-3, are used to set meeting details and facilitate the conversation. They use input to set meeting details, monitor progress, and facilitate the conversation.

[1617] Database: MongoDB or similar is used to store conference setting information and generated summaries.

[1618] Real-time communication server: Supports real-time data exchange between users using WebSockets, etc.

[1619] 2. Terminal:

[1620] User interface: HTML, CSS, and JavaScript are used to set up meetings, input real-time information, display emotional states, and display meeting summaries.

[1621] Emotion engine: Analyzes the user's emotional state in real time and generates feedback using a custom analysis module or an existing emotion analysis API (e.g., Microsoft Azure Emotion API).

[1622] 3. User:

[1623] Organizer: Set up the meeting by entering the purpose, agenda, and participant list.

[1624] Participants: Enter thoughts and comments in real time during the meeting, which are broadcast to other participants and displayed in real time.

[1625] Specific examples

[1626] Conference Settings:

[1627] The user (organizer) inputs the purpose, agenda, and participant list of the meeting. For example, if the purpose is a "marketing strategy meeting for a new product," the agenda may include "selecting the target market," "planning the advertising campaign," and "allocating the budget."

[1628] Meeting proceedings:

[1629] The device connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI and the emotion engine is activated.

[1630] Each participant enters their opinions and suggestions in text based on the agenda, and the text data is displayed to other participants in real time. The emotion engine analyzes each participant's emotional state, and if the emotion analyzed is "tension," for example, the system displays a message such as "Please relax."

[1631] Conference Summary:

[1632] After the meeting, the generative AI model analyzes all the text and emotion data recorded during the meeting and generates a summary of the entire meeting, including the conclusions and key comments made on each topic, as well as the emotional state of the meeting.

[1633] The summaries will be stored in a database and communicated to participants.

[1634] Prompt Sentence Examples

[1635] Here are some examples of prompts:

[1636] Meeting Information:

[1637] Title: Improve work efficiency

[1638] Agenda:

[1639] 1. Review of work processes

[1640] 2. Reassignment of workers

[1641] 3. Introducing new tools

[1642] Real-time input:

[1643] User1: "I think there's a lot of overlap in our current workflow."

[1644] User 2: "I think introducing new tools will improve efficiency."

[1645] Prompt for GPT-3:

[1646] Summarize the conversation below and consider the sentiment data:

[1647] "I think there is a lot of overlap in the current work process." (Average)

[1648] "I think that introducing new tools will improve efficiency." (Optimistic)

[1649] As described above, the present invention can improve the efficiency of meetings and manage emotions, thereby improving the quality of meetings within a factory.

[1650] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1651] Step 1:

[1652] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list. The entered data is sent from the terminal to the server. The input data is sent to the server in JSON format and saved in the database as configuration information.

[1653] Step 2:

[1654] The server uses a generative AI model to set up detailed meetings based on the input meeting information. Specifically, it checks the purpose of the meeting and the order of the agenda, and creates an appropriate timetable. This information is also stored in a database.

[1655] Step 3:

[1656] When a conference starts, users (participants) click on the designated conference link from their devices to join the conference. The devices connect to the real-time communication server and send the user ID and conference ID. The server then sends the conference information to the devices and activates the emotion engine.

[1657] Step 4:

[1658] Each user inputs their opinions and suggestions in text format into their device, following the agenda. The input text data is sent to the real-time communication server and broadcast to the devices of other participants in real time. The input data is displayed immediately.

[1659] Step 5:

[1660] The emotion engine analyzes each user's input text and determines their emotional state. The device receives the analysis results and displays the user's emotional state (e.g., "tension" or "joy") on the screens of other participants. If necessary, it also displays a feedback message to encourage relaxation.

[1661] Step 6:

[1662] The server's generative AI model monitors the progress of the meeting and generates and displays text prompting the user to move on to the next agenda item as needed, such as "Shall we move on to the next item?"

[1663] Step 7:

[1664] After the meeting is over, the organizer clicks the end button. The device sends the end-of-meeting information to the real-time communication server, and all data, including the analysis results of the emotion engine, is sent to the server.

[1665] Step 8:

[1666] The server uses a generative AI model to analyze all text and emotional data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting.

[1667] Step 9:

[1668] The generated summary is stored in a database and sent from the server to all participants via email, etc. The summary is used as a review of the meeting and is also useful for preparing for the next meeting.

[1669] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1670] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1671] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the headset type terminal 314.

[1672] [Fourth embodiment]

[1673] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[1674] 7, a data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1675] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1676] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[1677] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1678] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1679] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1680] The control object 443 includes a display device, LEDs in the eyes, and motors for driving the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[1681] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1682] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1683] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1684] In the robot 414, the processor 46 performs the reception output process. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1685] Next, a description will be given of the specific processing performed by the specific processing unit 290 of the data processing device 12. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1686] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system is composed of a server, terminals, and users who play the following roles.

[1687] Meeting Settings

[1688] 1. User (Organizer):

[1689] Enter the purpose of the meeting, agenda, and participant list into the input form.

[1690] 2. Terminal:

[1691] The entered conference information is sent to the server.

[1692] 3. Server:

[1693] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[1694] Starting a meeting

[1695] 1. User:

[1696] Participants click the provided meeting link to join the meeting.

[1697] 2. Terminal:

[1698] Connect to the real-time communication server and send the user ID and conference ID.

[1699] Display meeting information on the UI.

[1700] 3. Server:

[1701] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[1702] Real-time typing for meeting management

[1703] 1. User:

[1704] Each participant will enter their opinions and suggestions in text in line with the agenda.

[1705] 2. Terminal:

[1706] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[1707] 3. Server:

[1708] A generative AI model monitors meeting progress and facilitates the conversation as needed.

[1709] Ending the meeting and generating a summary

[1710] 1. User:

[1711] After all agenda items have been decided, the organizer clicks the end button.

[1712] 2. Terminal:

[1713] The conference end information is sent to the server.

[1714] 3. Server:

[1715] A generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[1716] The generated summaries are stored in a database and sent to the participants.

[1717] Specific examples

[1718] Example configuration:

[1719] User (Organizer): "New product marketing strategy meeting"

[1720] Objective: "Determine a marketing strategy for a new product"

[1721] Agenda:

[1722] 1. Selecting your target market

[1723] 2. Advertising campaign planning

[1724] 3. Budget allocation

[1725] Progress example:

[1726] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[1727] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[1728] End example:

[1729] User (Organizer): Complete all agenda items and click the End Meeting button.

[1730] Server: The generative AI model summarizes the meeting content and sends the summary to all participants.

[1731] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[1732] The processing flow will be explained below.

[1733] Step 1:

[1734] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[1735] Step 2:

[1736] Terminal: Sends the entered meeting information to the server in JSON format.

[1737] Step 3:

[1738] Server: Stores the received conference information in a database.

[1739] Step 4:

[1740] Server: The generative AI model sets up the meeting details based on the received information.

[1741] Step 5:

[1742] User: Participants join the meeting by clicking the provided meeting link.

[1743] Step 6:

[1744] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[1745] Step 7:

[1746] Server: Retrieves conference information from the database based on the conference ID.

[1747] Step 8:

[1748] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[1749] Step 9:

[1750] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[1751] Step 10:

[1752] Terminal: The meeting assistant displays the meeting opening message on the screen.

[1753] Step 11:

[1754] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[1755] Step 12:

[1756] Terminal: Sends text data entered by the user to the real-time communication server.

[1757] Step 13:

[1758] Server: Broadcasts the received text data from each participant to all user terminals.

[1759] Step 14:

[1760] Terminal: Text received from other users is displayed on the UI in real time.

[1761] Step 15:

[1762] Server: A generative AI model monitors the progress of the meeting and facilitates the conversation as needed.

[1763] Step 16:

[1764] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[1765] Step 17:

[1766] Terminal: Sends conference end information to the real-time communication server.

[1767] Step 18:

[1768] Server: The generative AI model analyzes all the text data recorded during the meeting and generates a summary.

[1769] Step 19:

[1770] Server: Stores the generated summaries in a database.

[1771] Step 20:

[1772] Server: Sends the generated summary to the participants.

[1773] Example 1

[1774] Next, a description will be given of Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1775] Meetings often involve unnecessary conversations, making it difficult to hold efficient discussions and make decisions. It also requires a lot of effort to quickly and accurately record the contents of meetings and provide a summary to participants. Furthermore, when multilingual support is required, communication becomes difficult when participants speak different languages.

[1776] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1777] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This allows for efficient conference progress, eliminates unnecessary conversation, and enables quick and accurate recording of conference content and provision of summaries, as well as multilingual support.

[1778] A "meeting organizer" is the person responsible for setting the purpose, agenda, and participant list for a meeting.

[1779] A "generative AI model" is an algorithm that uses artificial intelligence to automatically generate text and manage meeting progress.

[1780] An "input form" is an interface that allows a conference organizer to input information about the conference.

[1781] A "real-time communication server" is a server for transmitting and receiving data in real time.

[1782] A "user ID" is an identifier for uniquely identifying a user participating in a conference.

[1783] A "conference ID" is an identifier for uniquely identifying an individual conference.

[1784] An "agenda" is a list of items or topics to be discussed at a meeting.

[1785] "Text data" is character information input by the user.

[1786] "Display in real time" means that the input information is instantly displayed on the terminals of the other participants.

[1787] The "end conference button" is a button used by the conference organizer to instruct the end of the conference.

[1788] The "summary" is a shortened version of the meeting content generated based on the text data recorded during the meeting.

[1789] A "database" is a storage system for storing meeting details and generated summaries.

[1790] "Multilingual support" refers to translating the generated abstracts and meeting information into multiple languages.

[1791] This invention is a system that uses a generative AI model to guide meetings based on real-time typing and eliminates unnecessary conversations in order to ensure efficient meeting progress. This system operates in cooperation with the server, terminals, and users.

[1792] Meeting Settings

[1793] First, the user, who is the meeting organizer, enters the purpose, agenda, and participant list of the meeting into the system's input form. The device sends this information to the server. The server stores the received meeting information in a database, and a generative AI model (e.g., GPT-3) sets the meeting details.

[1794] Examples:

[1795] The organizer types in "New Product Marketing Strategy Meeting" and adds agenda items such as "Select Target Market," "Plan Advertising Campaign," and "Allocate Budget."

[1796] Starting a meeting

[1797] Users click on the designated meeting link to join the meeting. The device then connects to the real-time communication server and sends the user ID and meeting ID. The server retrieves the meeting information from the database, and the generative AI model displays the purpose and agenda of the meeting.

[1798] Examples:

[1799] Users click the link to access the meeting page, where the purpose and agenda of the meeting will be displayed on the screen.

[1800] Real-time typing for meeting management

[1801] Each participant, a user, enters their opinions and suggestions in text based on the agenda. The device sends this input text data to a real-time communication server, which also receives and displays the text data of other participants in real time. The server uses a generative AI model to monitor the progress of the conversation and facilitate it as needed.

[1802] Examples:

[1803] When a user types "advertising campaigns should be primarily conducted on social media," the content is instantly shared with other participants.

[1804] The server uses a generative AI model to display "May we move on to the next topic?"

[1805] Ending the meeting and generating a summary

[1806] When all agenda items have been completed, the organizer clicks the end button. The device sends the end-of-meeting information to the server. The server then uses a generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is stored in a database and sent to participants.

[1807] Examples:

[1808] After the meeting, the generative AI model generates a document summarizing the key points of the meeting and emails it to each participant.

[1809] Prompt Sentence Examples

[1810] "Please proceed to the next agenda item."

[1811] "Does anyone else have any opinions on this topic?"

[1812] In this way, the system of the present invention supports efficient and productive conference progress and eliminates unnecessary conversations, thereby improving conference productivity.

[1813] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1814] Step 1:

[1815] The user enters the meeting information.

[1816] The user enters the purpose of the meeting, the agenda, and a list of participants into an input form. The input includes the meeting name "New product marketing strategy meeting," and agenda items such as "Selection of target market," "Advertising campaign planning," and "Budget allocation." This information is generated as output in JSON format. The user enters this information into the input form on a web browser.

[1817] Step 2:

[1818] The terminal transmits the conference information.

[1819] The terminal sends the conference information entered in step 1 to the server. As input, it receives the JSON-formatted conference information entered by the user. As data processing, it converts this JSON data into an HTTP request and sends it as output to the server. Specifically, the terminal generates a request and sends a POST request to the server's API endpoint.

[1820] Step 3:

[1821] The server stores the meeting information and configures the generative AI model.

[1822] The server stores the meeting information received from the device in a database and configures the meeting details in the generative AI model. As input, it receives the JSON-formatted meeting information from the device. As data processing, it inserts this information into the database and configures the required settings for the generative AI model. As output, it notifies the generative AI model that the meeting details have been configured. The server executes an SQL query to store the information in the database and configures parameters through the AI ​​model's configuration interface.

[1823] Step 4:

[1824] A user joins a conference.

[1825] Users join a meeting by clicking a specified meeting link. The input is the act of clicking the meeting link. The output is the display of the meeting UI. The specific behavior is that the user clicks a link embedded in an email or calendar event they received, and the meeting page opens in a browser.

[1826] Step 5:

[1827] The terminal acquires and displays the conference information.

[1828] The terminal connects to the real-time communication server and sends the user ID and conference ID. It also displays the conference information on the UI. As input, it obtains the user ID and conference ID. As data processing, it connects to the real-time communication server using WebSocket and sends these IDs as authentication information. As output, the display of the conference information is completed. Specifically, the terminal displays the conference details information it received from the server in the UI component.

[1829] Step 6:

[1830] The server sends the meeting details.

[1831] The server retrieves meeting information from the database, and the generative AI model sends the meeting purpose and agenda to the device. As input, it receives the user ID and meeting ID. As data processing, it executes an SQL query to retrieve information from the database, and the generative AI model formats the meeting details. As output, it sends the generated meeting details to the device. Specifically, the server sends the output of the generative AI model to the device in real time.

[1832] Step 7:

[1833] The user inputs their opinion.

[1834] Each participant, a user, enters their opinions and suggestions in text along the agenda. As input, they enter the text of their opinions and suggestions related to the agenda. As output, that text data is generated. Specifically, they enter their opinions in the chat window and click the send button.

[1835] Step 8:

[1836] The device sends and receives text data.

[1837] The device sends the entered text data to the real-time communication server, receives text data from other participants, and displays it in real time. As input, it receives text data entered by the user. As data processing, it sends and receives this data via WebSocket, and as output, it displays it in the chat window in real time. In concrete terms, the device sends text data and immediately displays the received data.

[1838] Step 9:

[1839] The server facilitates the conversation.

[1840] The server uses a generative AI model to monitor the progress of the meeting and facilitate the conversation as needed. As input, it receives text data from each participant. As data processing, the generative AI model analyzes the content of the conversation and generates appropriate prompts. As output, a prompt for promotion is generated and sent to the terminal. Specifically, the generative AI model generates and displays a message such as "Shall we move on to the next agenda item?"

[1841] Step 10:

[1842] The user ends the conference.

[1843] When all agenda items have been completed, the user (organizer) clicks the end conference button. The input is the operation of clicking the end conference button. The output is the generation of conference end information. The specific operation is the user clicking the end conference button.

[1844] Step 11:

[1845] The terminal transmits the conference end information.

[1846] The terminal sends the conference end information to the server. As input, it receives information that the conference end button has been pressed. As data processing, it sends this information as an HTTP request. As output, it sends the conference end information to the server. Specifically, the terminal POSTs the conference end information to the server.

[1847] Step 12:

[1848] The server generates and transmits the meeting summary.

[1849] The server uses the generative AI model to analyze all text data recorded during the meeting and generate a summary. The generated summary is saved in a database and sent to participants. All text data is received as input. As data processing, the generative AI model analyzes the text data and automatically generates a summary. As output, the generated summary is saved and sent to participants by email. Specifically, the server uses the generative AI model to generate a summary and sends it to participants using the email sending function.

[1850] (Application example 1)

[1851] Next, a description will be given of Application Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[1852] With conventional conferencing systems, meeting progress management was often done manually, resulting in long meetings and frequent unnecessary conversations. Furthermore, there were limited ways for participants to share their opinions in real time, making meetings less efficient. In brick-and-mortar stores in particular, poor communication between store operations and staff can lead to a decline in work efficiency. Furthermore, organizing the content of meetings after they have ended was time-consuming, resulting in delays in summarizing and sharing results.

[1853] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1854] In this invention, the server includes: a means for a meeting organizer to input the purpose, agenda, and participant list of the meeting; a means for using a generative AI model to set and save the details of the meeting based on the input information; a means for receiving text input from the user via real-time typing and broadcasting it to other participants; a means for the generative AI model to monitor the progress of the meeting and promote conversation; a means for automatically analyzing the meeting content and generating a summary after the meeting ends; a means for saving the generated summary and sending it to participants; a means for staff in a physical store to join the meeting from a smart device and input their opinions in real time; a means for the generative AI model to generate a prompt comment during the meeting to encourage the next agenda item; a means for displaying the generated prompt comment on the staff member's smart device; and a means for sending a summary generated based on the meeting content to the staff member's smart device, thereby enabling efficient progress of meetings and smooth communication in a physical store.

[1855] A "meeting organizer" is a user whose role is to enter the purpose, agenda, and participant list of a meeting.

[1856] A "generative AI model" is an artificial intelligence that sets meeting details based on input information, monitors progress, and facilitates conversations.

[1857] "Real-time typing" is the process by which a user types text during a meeting, which is then shared with other participants in real time.

[1858] "Broadcast" is a communication method that distributes text input from a user to all participants simultaneously.

[1859] A "promotion comment" is a text message sent by the generative AI model during a meeting to prompt the participant to move on to the next agenda item.

[1860] "Smart devices" is a general term for portable electronic devices that can connect to the Internet, such as smartphones and tablets.

[1861] A "brick and mortar store" is a sales establishment that exists in a physical location and conducts business face-to-face with customers.

[1862] A "meeting summary" is a concise summary of what was discussed during the meeting.

[1863] "Means for inputting opinions in real time" refers to a method in which meeting participants use smart devices to instantly input their opinions in text.

[1864] "Means for automatic translation into multiple languages" refers to a function that automatically converts the generated summary into different languages.

[1865] This invention is a system that efficiently conducts meetings between staff in a physical store and eliminates unnecessary conversations.

[1866] Meeting Settings

[1867] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[1868] The terminal transmits the input conference information to the server.

[1869] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[1870] Starting a meeting

[1871] Users click on the provided meeting link to join the meeting.

[1872] The terminal connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI.

[1873] The server retrieves meeting information from a database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[1874] Real-time typing for meeting management

[1875] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[1876] The terminal transmits text data entered by the user to the real-time communication server, and receives text data from other participants and displays it in real time.

[1877] The server uses a generative AI model to monitor the progress of the meeting, generate prompts to move on to the next agenda item, and display them on participants' devices.

[1878] As a specific example, the following prompt sentence is generated: "Meeting agenda: Marketing strategy meeting for a new product. Participant's message: What should be the target market for the new product? We are considering a plan for an advertising campaign. Please generate a message to encourage whether we should proceed next."

[1879] Ending the meeting and generating a summary

[1880] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[1881] The terminal transmits the conference end information to the server.

[1882] The server uses a generative AI model to analyze all text data recorded during the meeting and generate a summary.

[1883] The generated summaries are stored in a database and sent to all participants.

[1884] It also provides a means for the generated summary to be automatically translated into multiple languages.

[1885] The system allows store staff to quickly and efficiently share opinions using smart devices (smartphones, tablets, etc.), eliminating unnecessary conversations. It also uses a generative AI model to monitor meeting progress in real time and generate appropriate prompts to guide the meeting.

[1886] A specific example is when staff at a brick-and-mortar store exchange opinions in real time based on the agenda for a "New Product Marketing Strategy Meeting." The generative AI model provides prompt comments such as "Shall we move on to the next agenda item?", allowing the meeting to proceed smoothly.

[1887] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1888] Processing Steps

[1889] Step 1:

[1890] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[1891] Input: Meeting purpose, agenda, participant list

[1892] Output: Entered meeting information

[1893] Specific action: A user enters text into a web or application form.

[1894] Step 2:

[1895] The terminal transmits the input conference information to the server.

[1896] Input: Meeting Information

[1897] Output: Meeting information transferred to the server

[1898] Specific operation: Obtain data from the input form and send it to the server via an HTTP request.

[1899] Step 3:

[1900] The server stores the received meeting information in a database, and the generative AI model sets the meeting details.

[1901] Input: Meeting Information

[1902] Output: Meeting information stored in the database, configured meeting details

[1903] Specific operation: The received data is stored in a database via an SQL query, and the AI ​​model uses that data to complete meeting details (purpose and agenda).

[1904] Step 4:

[1905] User clicks on the provided meeting link to join the meeting.

[1906] Input: Meeting link

[1907] Output: Meeting participation confirmation

[1908] What happens: A user clicks on a link they received in an email or message to access the meeting join page.

[1909] Step 5:

[1910] The device connects to the real-time communication server and sends the user ID and conference ID. The conference information is displayed on the UI.

[1911] Input: User ID, Meeting ID

[1912] Output: Connection status, displayed meeting information

[1913] Specific operation: User information is sent via WebSocket or HTTP request, and conference information is obtained and displayed on the screen.

[1914] Step 6:

[1915] The server retrieves meeting information from the database and sends it to the device, where the generative AI model displays the meeting purpose and agenda.

[1916] Input: Meeting ID

[1917] Output: Meeting information sent to the device and display content generated by the AI ​​model

[1918] How it works: Meeting information is retrieved through a database query and sent to the device. Based on that information, the AI ​​model displays the purpose and agenda in the UI.

[1919] Step 7:

[1920] Each participant (user) enters their opinions and suggestions in text in line with the agenda.

[1921] Input: User text input

[1922] Output: Real-time shared text data

[1923] Specific operation: The user enters opinions and suggestions in the text box, which are then sent to the real-time communication server.

[1924] Step 8:

[1925] The terminal transmits text data entered by the user to a real-time communication server, receives text data from other participants, and displays it in real time.

[1926] Input: User's text data

[1927] Output: Text data broadcast to other participants, text data displayed

[1928] Specific operation: Send data to other participants via a real-time communication protocol (e.g., WebSocket) and display the received data on the screen.

[1929] Step 9:

[1930] The server monitors the progress of the meeting using a generative AI model, generates prompts to move on to the next agenda item, and displays them on participants' devices.

[1931] Input: Meeting progress data

[1932] Output: Generated promotion comment, Displayed promotion comment

[1933] Specific operation: The AI ​​model analyzes the progress of the meeting, generates prompts such as "Shall we move on to the next agenda item?", and sends them to the device for display.

[1934] Step 10:

[1935] After all the agenda items have been decided, the user (organizer) clicks the end meeting button.

[1936] Input: End meeting instruction

[1937] Output:Confirmation of end of meeting

[1938] Specific operation: The terminal captures the click event of the end conference button and sends an end instruction to the server.

[1939] Step 11:

[1940] The terminal transmits conference end information to the server.

[1941] Input: Meeting end information

[1942] Output: Finished information sent to the server

[1943] Specific operation: Sends conference end information to the server via HTTP request or WebSocket.

[1944] Step 12:

[1945] The server uses the generative AI model to analyze all the text data recorded during the meeting and generate a summary.

[1946] Input: All text data during the meeting

[1947] Output: Generated meeting summary

[1948] Specific operation: Analyzes text data and generates summaries using natural language processing techniques.

[1949] Step 13:

[1950] The server stores the generated summaries in a database and sends them to all participants, and also provides a means for automatic translation into multiple languages.

[1951] Input: Generated meeting summary

[1952] Output: Summary stored in database, multilingual summary sent

[1953] Specific actions: The generated summary is saved and sent to each participant via email or notification. If necessary, the summary is translated into multiple languages ​​using an automatic translation API.

[1954] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1955] This invention is a system that uses a generative AI model and emotion engine to conduct meetings efficiently based on real-time typing and emotion information, eliminating unnecessary conversations. This system is composed of a server, terminals, and users who play the following roles.

[1956] Meeting Settings

[1957] 1. User (Organizer):

[1958] Enter the purpose of the meeting, agenda, and participant list into the input form.

[1959] 2. Terminal:

[1960] The entered conference information is sent to the server in JSON format.

[1961] 3. Server:

[1962] The received meeting information is stored in a database, and the generative AI model sets the meeting details.

[1963] Starting a meeting

[1964] 1. User:

[1965] Participants click the provided meeting link to join the meeting.

[1966] 2. Terminal:

[1967] Connect to the real-time communication server and send the user ID and conference ID.

[1968] Display the meeting information on the UI and activate the emotion engine.

[1969] 3. Server:

[1970] Meeting information is retrieved from a database and sent to the device, and the generative AI model displays the purpose and agenda of the meeting.

[1971] The emotion engine analyzes each user's emotional data in real time and feeds the results back into the generative AI model.

[1972] Real-time typing for meeting progress and emotion analysis

[1973] 1. User:

[1974] Each participant will enter their opinions and suggestions in text in line with the agenda.

[1975] 2. Terminal:

[1976] The text data entered by the user is transmitted to a real-time communication server, and the text data of other participants is received and displayed in real time.

[1977] The emotional engine analyzes each participant's emotional state and displays it to other participants as appropriate.

[1978] 3. Server:

[1979] A generative AI model monitors the progress of the meeting and drives the conversation based on the user's emotional data.

[1980] For example, if the user is nervous, the generative AI model will insert questions or comments to help them relax.

[1981] Ending the meeting and generating a summary

[1982] 1. User:

[1983] After all agenda items have been decided, the organizer clicks the end button.

[1984] 2. Terminal:

[1985] The conference end information is transmitted to the real-time communication server.

[1986] Data including the analysis results of the emotion engine is sent to the server.

[1987] 3. Server:

[1988] A generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[1989] The summary includes not only the conclusions and key statements for each topic, but also the emotional state of the meeting.

[1990] The generated summaries are stored in a database and sent to the participants.

[1991] Specific examples

[1992] Example configuration:

[1993] User (Organizer): "New product marketing strategy meeting"

[1994] Objective: "Determine a marketing strategy for a new product"

[1995] Agenda:

[1996] 1. Selecting your target market

[1997] 2. Advertising campaign planning

[1998] 3. Budget allocation

[1999] Progress example:

[2000] Users: Enter their opinions on each topic and the information is displayed to other participants in real time.

[2001] Server: The generative AI model stimulates discussion by displaying prompting text such as, "May we move on to the next topic?"

[2002] Terminal: The emotion engine analyzes the emotions of each participant and displays relaxing comments to users who are nervous.

[2003] End example:

[2004] User (Organizer): Complete all agenda items and click the End Meeting button.

[2005] Server: The generative AI model summarizes the meeting content and sends the summary and sentiment analysis results to all participants.

[2006] In this way, the system of the present invention not only supports efficient and productive meeting progress and eliminates unnecessary conversations, but also improves the quality of meetings by utilizing emotional data.

[2007] The processing flow will be explained below.

[2008] Step 1:

[2009] User: The meeting organizer enters the purpose of the meeting, the agenda, and the list of participants in a dedicated form.

[2010] Step 2:

[2011] Terminal: Sends the entered meeting information to the server in JSON format.

[2012] Step 3:

[2013] Server: Stores the received conference information in a database.

[2014] Step 4:

[2015] Server: The generative AI model sets up the meeting details based on the received information.

[2016] Step 5:

[2017] User: Participants join the meeting by clicking the provided meeting link.

[2018] Step 6:

[2019] Terminal: Connects to the real-time communication server and transmits the user ID and conference ID.

[2020] Step 7:

[2021] Server: Retrieves conference information from the database based on the conference ID.

[2022] Step 8:

[2023] Server: Sends the acquired meeting purpose and agenda to the terminal in JSON format.

[2024] Step 9:

[2025] Terminal: Based on the received meeting information, the purpose and agenda of the meeting are displayed on the UI.

[2026] Step 10:

[2027] Terminal: The meeting assistant displays the meeting opening message on the screen.

[2028] Step 11:

[2029] Users: Following the agenda, each participant enters their opinions and suggestions in text.

[2030] Step 12:

[2031] Terminal: Sends text data entered by the user to the real-time communication server.

[2032] Step 13:

[2033] Server: Broadcasts the received text data from each participant to all user terminals.

[2034] Step 14:

[2035] Terminal: Text received from other users is displayed on the UI in real time.

[2036] Step 15:

[2037] Terminal: The emotion engine analyzes the emotions of each participant and sends the results to the server in real time.

[2038] Step 16:

[2039] Server: Feedbacks emotional data from the emotion engine to the generative AI model and monitors the progress of the meeting.

[2040] Step 17:

[2041] Server: The generative AI model uses emotional data to insert comments and questions to encourage conversation.

[2042] Step 18:

[2043] Device: Analyzed emotion results are displayed in real time on the UI, providing feedback according to the situation.

[2044] Step 19:

[2045] User: After all the agenda items have been decided, the organizer clicks the end meeting button.

[2046] Step 20:

[2047] Terminal: Sends conference end information to the real-time communication server.

[2048] Step 21:

[2049] Server: The generative AI model analyzes all text and emotion data recorded during the meeting and generates a summary.

[2050] Step 22:

[2051] Server: Stores the generated summaries in a database.

[2052] Step 23:

[2053] Server: Sends summary results to participants.

[2054] Step 24:

[2055] Server: May automatically translate abstracts into multiple languages ​​and send each language version of the abstract to participants.

[2056] Example 2

[2057] Next, a description will be given of Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2058] The purpose of this invention is to improve the quality of meetings by efficiently conducting meetings, sharing information, and analyzing participants' emotional data. Conventional meeting systems have not adequately considered the efficiency of meeting progress or the emotional state of participants, resulting in problems such as participants losing concentration and discussion stalling. Furthermore, generating summaries of comments and discussions during meetings is often done manually, requiring time and effort.

[2059] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[2060] In this invention, the server

[2061] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[2062] a means for configuring and saving meeting details based on input information using a generative AI model;

[2063] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[2064] means for analyzing the emotional state of each user in real time using an emotion engine;

[2065] A generative AI model monitors the progress of the meeting and uses user emotional data to facilitate the conversation.

[2066] means for automatically analyzing the contents of a meeting after the meeting and generating a summary from the text data and emotion data;

[2067] and means for storing and transmitting the generated summary to the participants.

[2068] This will enable meetings to proceed efficiently, improve the quality of meetings by utilizing participants' emotional data, and automatically generate summaries.

[2069] A "meeting organizer" is a person who is responsible for setting the purpose, agenda, and participant list of a meeting and managing the overall progress of the meeting.

[2070] A "generative AI model" is an artificial intelligence model that uses technologies such as natural language processing to assist in the progress of meetings, providing appropriate feedback and support for progress based on input information.

[2071] An "emotion engine" is a system that analyzes participants' emotional state in real time from voice, facial expressions, text, etc., and provides feedback on the analysis results.

[2072] "Real-time typing" refers to the process by which a user types text in real time, which is instantly broadcast to other participants.

[2073] "Broadcast" refers to the simultaneous transmission of data (e.g., a text message) from one user to multiple recipients in real time.

[2074] "Meeting progress" refers to the state of the meeting, showing how well it is progressing according to the agenda, the flow of comments, progress on the agenda, etc.

[2075] "User emotional data" refers to data that indicates the emotional state of a participant, and is obtained from voice tone, text content, facial expressions, etc.

[2076] A "summary" refers to a concise report containing the overall conclusions and important statements of the meeting, generated based on all text and emotion data recorded during the meeting.

[2077] A "real-time communication server" refers to a server that manages real-time data communication between users using WebSocket or similar.

[2078] This invention is a system that uses a generative AI model and emotion engine to conduct meetings in real time based on typing and emotion information, eliminating unnecessary conversations, in order to ensure effective and productive meeting proceedings. This system is composed of three entities: a server, a terminal, and a user.

[2079] Meeting Settings

[2080] User (organizer):

[2081] The user enters the purpose, agenda, and participant list of the meeting into a dedicated input form. For example, the user can set the purpose, such as "New product marketing strategy meeting," and each agenda item.

[2082] Device:

[2083] The meeting information entered by the user is converted into JSON format and sent to the server using an HTTP POST request.

[2084] server:

[2085] The server stores the received JSON data in a database such as MySQL. The server then uses a generative AI model to set up the meeting details based on the received information. The generative AI model automatically calculates the optimal order and time allocation for the meeting.

[2086] Starting a meeting

[2087] User:

[2088] Participants click on the designated meeting link and log in to the system using a web browser or dedicated application.

[2089] Device:

[2090] The device connects to a real-time communication server such as a WebSocket server and transmits the user ID and conference ID. After the connection is complete, the device displays the conference information on the UI and starts the emotion engine, which starts analyzing the user's emotions.

[2091] server:

[2092] The server retrieves meeting information from the database and sends it to the device. The generative AI model displays the meeting's purpose and agenda on the screen and supports the meeting's progress. The emotion engine analyzes each user's emotional data in real time and feeds the results back to the generative AI model.

[2093] Real-time typing for meeting progress and emotion analysis

[2094] User:

[2095] Each participant enters their opinions and suggestions in text form along the agenda, using UI components such as a chat box.

[2096] Device:

[2097] The device sends the text data entered by the user to a real-time communication server using WebSocket or similar, and also receives and displays the text data of other participants in real time. The device also analyzes the emotional state of each participant using an emotion engine and displays the results on the UI. For example, if a user is judged to be "tense," the device will display "Please relax."

[2098] server:

[2099] The server monitors the progress of the meeting using a generative AI model. Based on participants' comments and emotional data, the generative AI model suggests the next agenda item and necessary questions. For example, if the discussion stalls, the generative AI model will insert a question such as, "Do you have any more specific suggestions for the current issue?"

[2100] Ending the meeting and generating a summary

[2101] User:

[2102] After all agenda items have been decided, the organizer clicks the end button.

[2103] Device:

[2104] The terminal transmits the conference end information to the real-time communication server, and also transmits the final analysis result of the emotion engine to the server.

[2105] server:

[2106] The server uses a generative AI model to analyze all text and emotion data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting. The summaries are stored in a database and sent to participants in real time via email or app notifications.

[2107] Prompt Sentence Examples

[2108] "Please provide your opinion on selecting the target market for the new product marketing strategy meeting."

[2109] "May we move on to the next topic?"

[2110] "Do you have any more specific suggestions for the current issue?"

[2111] As a result, the system of the present invention supports efficient and productive conference progress, eliminates unnecessary conversations, and improves the quality of conferences by utilizing emotion data.

[2112] The flow of the identification process in the second embodiment will be described with reference to FIG.

[2113] Step 1: Enter and submit meeting information

[2114] Input: The user (organizer) enters the purpose of the meeting, the agenda, and the participant list into an input form.

[2115] How it works: The terminal converts the input data into JSON format.

[2116] Output: The terminal sends the converted JSON data to the server using an HTTP POST request.

[2117] Step 2: Save meeting information and configure it with a generative AI model

[2118] Input: JSON data sent from the terminal.

[2119] How it works: The server stores the received JSON data in a database such as MySQL. It then launches a generative AI model to set up the details of the meeting based on the received information. The generative AI model automatically calculates the optimal order of the meeting and time allocation.

[2120] Output: The configured meeting details are saved in the database.

[2121] Step 3: Share the meeting link and connect

[2122] Input: User clicks on the provided meeting link.

[2123] Operation: The device connects to a real-time communication server (such as a WebSocket server) and sends the user ID and conference ID.

[2124] Output: The connection is authenticated and the meeting information is displayed in the UI.

[2125] Step 4: Displaying meeting information and launching the emotion engine

[2126] Input: Meeting information retrieved from the server.

[2127] Operation: The device displays the received conference information on the UI and starts the emotion engine, which starts emotion analysis.

[2128] Output: The meeting information is displayed on the screen and the emotion engine starts working.

[2129] Step 5: Real-time typing and sending and receiving data

[2130] Input: The user types their opinion or suggestion into the text box.

[2131] Operation: The device sends the user's input data to the real-time communication server, and simultaneously receives text data from other participants and displays it in real time. The emotion engine analyzes the emotional state of each participant, and the results are reflected in the UI.

[2132] Output: The input text data is broadcast to other participants in real time, and the analyzed emotional information is displayed.

[2133] Step 6: Monitor and facilitate meeting progress

[2134] Input: User utterances and emotional state data.

[2135] How it works: The server monitors the progress of the meeting using a generative AI model. The generative AI model uses participants' comments and emotional data to suggest the next agenda item and necessary questions. For example, if the discussion stagnates, the generative AI model inserts appropriate questions to stimulate the conversation.

[2136] Output: A prompt message based on feedback from the generative AI model is displayed.

[2137] Step 7: Ending the meeting and sending data

[2138] Input: Organizer clicks end meeting button.

[2139] Operation: The terminal sends the conference end information to the real-time communication server, and also sends the final analysis results from the emotion engine to the server.

[2140] Output: Information on the end of the conference and the final analysis results are sent to the server.

[2141] Step 8: Generate and send the summary

[2142] Input: All text and emotion data recorded during the meeting.

[2143] How it works: The server uses a generative AI model to automatically generate a summary of the meeting content, including conclusions for each topic, important comments, and the emotional state of the meeting.

[2144] Output: The generated summaries are stored in a database and sent to participants via email or a dedicated app notification.

[2145] By following the above steps, this system is able to conduct a conference effectively and efficiently.

[2146] (Application example 2)

[2147] Next, a description will be given of Application Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the robot 414 will be referred to as a "terminal."

[2148] In modern factory operations, improving meeting efficiency and speeding up decision-making are important challenges. However, conventional methods make meetings slow and prone to unnecessary conversation. Furthermore, there is no way to grasp the emotional state of meeting participants and provide appropriate feedback, leading to tension and stress that reduces the quality of meetings. This invention aims to use a generative AI model and emotion engine to efficiently conduct factory meetings and improve the quality of meetings by taking into account the emotional state of participants.

[2149] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[2150] In this invention, the server includes a means for the conference organizer to input the conference purpose, agenda, and participant list, a means for using a generative AI model to set and save the conference details based on the input information, a means for receiving text input from the user via real-time typing and broadcasting it to other participants, a means for the generative AI model to monitor the progress of the conference and promote conversation, a means for analyzing the user's emotional state and providing feedback, a means for automatically analyzing the conference content and generating a summary after the conference ends, and a means for saving the generated summary and sending it to the participants. This enables the conference to proceed more efficiently and the emotional states of the participants to be managed appropriately.

[2151] A "meeting organizer" is a person in charge of setting up a meeting by entering the purpose, agenda, participant list, etc. of the meeting.

[2152] A "generative AI model" is a system that uses artificial intelligence to support detailed meeting setup and progress based on input information.

[2153] An "agenda" refers to the items or topics to be discussed at a meeting.

[2154] "Real-time typing" is a system in which participants enter their opinions and comments in text during a meeting, and the content is instantly transmitted to other participants.

[2155] "Broadcast" means sending specific information to multiple recipients simultaneously.

[2156] The "Emotion Engine" is a system that analyzes the emotional state of participants in real time and reflects the results in the progress of the meeting.

[2157] "Emotional state" refers to the emotional state, such as tension, joy, or fatigue, that participants feel during the meeting.

[2158] "Feedback" refers to providing appropriate responses or comments to participants based on the emotional data analyzed by the emotion engine.

[2159] A "summary" is a short summary of the entire meeting and its important points, generated after the meeting is over.

[2160] "Storage" means recording the generated data in a database or the like so that it can be referenced later.

[2161] "Transmit" means transmitting information via email or other means to provide the generated summary and sentiment analysis results to participants.

[2162] This invention is a system that efficiently facilitates factory meetings and improves their quality by taking into account the emotional state of participants. The system utilizes a generative AI model and an emotion engine to analyze real-time text input and emotional data to facilitate meetings.

[2163] Program processing and hardware / software used

[2164] The system includes the following components:

[2165] 1. Server:

[2166] Generative AI models, such as OpenAI GPT-3, are used to set meeting details and facilitate the conversation. They use input to set meeting details, monitor progress, and facilitate the conversation.

[2167] Database: MongoDB or similar is used to store conference setting information and generated summaries.

[2168] Real-time communication server: Supports real-time data exchange between users using WebSockets, etc.

[2169] 2. Terminal:

[2170] User interface: HTML, CSS, and JavaScript are used to set up meetings, input real-time information, display emotional states, and display meeting summaries.

[2171] Emotion engine: Analyzes the user's emotional state in real time and generates feedback using a custom analysis module or an existing emotion analysis API (e.g., Microsoft Azure Emotion API).

[2172] 3. User:

[2173] Organizer: Set up the meeting by entering the purpose, agenda, and participant list.

[2174] Participants: Enter thoughts and comments in real time during the meeting, which are broadcast to other participants and displayed in real time.

[2175] Specific examples

[2176] Conference Settings:

[2177] The user (organizer) inputs the purpose, agenda, and participant list of the meeting. For example, if the purpose is a "marketing strategy meeting for a new product," the agenda may include "selecting the target market," "planning the advertising campaign," and "allocating the budget."

[2178] Meeting proceedings:

[2179] The device connects to the real-time communication server and transmits the user ID and conference ID. The conference information is displayed on the UI and the emotion engine is activated.

[2180] Each participant enters their opinions and suggestions in text based on the agenda, and the text data is displayed to other participants in real time. The emotion engine analyzes each participant's emotional state, and if the emotion analyzed is "tension," for example, the system displays a message such as "Please relax."

[2181] Conference Summary:

[2182] After the meeting, the generative AI model analyzes all the text and emotion data recorded during the meeting and generates a summary of the entire meeting, including the conclusions and key comments made on each topic, as well as the emotional state of the meeting.

[2183] The summaries will be stored in a database and communicated to participants.

[2184] Prompt Sentence Examples

[2185] Here are some examples of prompts:

[2186] Meeting Information:

[2187] Title: Improve work efficiency

[2188] Agenda:

[2189] 1. Review of work processes

[2190] 2. Reassignment of workers

[2191] 3. Introducing new tools

[2192] Real-time input:

[2193] User1: "I think there's a lot of overlap in our current workflow."

[2194] User 2: "I think introducing new tools will improve efficiency."

[2195] Prompt for GPT-3:

[2196] Summarize the conversation below and consider the sentiment data:

[2197] "I think there is a lot of overlap in the current work process." (Average)

[2198] "I think that introducing new tools will improve efficiency." (Optimistic)

[2199] As described above, the present invention can improve the efficiency of meetings and manage emotions, thereby improving the quality of meetings within a factory.

[2200] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[2201] Step 1:

[2202] The user (organizer) enters the purpose of the meeting, the agenda, and the participant list. The entered data is sent from the terminal to the server. The input data is sent to the server in JSON format and saved in the database as configuration information.

[2203] Step 2:

[2204] The server uses a generative AI model to set up detailed meetings based on the input meeting information. Specifically, it checks the purpose of the meeting and the order of the agenda, and creates an appropriate timetable. This information is also stored in a database.

[2205] Step 3:

[2206] When a conference starts, users (participants) click on the designated conference link from their devices to join the conference. The devices connect to the real-time communication server and send the user ID and conference ID. The server then sends the conference information to the devices and activates the emotion engine.

[2207] Step 4:

[2208] Each user inputs their opinions and suggestions in text format into their device, following the agenda. The input text data is sent to the real-time communication server and broadcast to the devices of other participants in real time. The input data is displayed immediately.

[2209] Step 5:

[2210] The emotion engine analyzes each user's input text and determines their emotional state. The device receives the analysis results and displays the user's emotional state (e.g., "tension" or "joy") on the screens of other participants. If necessary, it also displays a feedback message to encourage relaxation.

[2211] Step 6:

[2212] The server's generative AI model monitors the progress of the meeting and generates and displays text prompting the user to move on to the next agenda item as needed, such as "Shall we move on to the next item?"

[2213] Step 7:

[2214] After the meeting is over, the organizer clicks the end button. The device sends the end-of-meeting information to the real-time communication server, and all data, including the analysis results of the emotion engine, is sent to the server.

[2215] Step 8:

[2216] The server uses a generative AI model to analyze all text and emotional data recorded during the meeting and generate summaries, including conclusions for each topic, key comments, and the emotional state of the meeting.

[2217] Step 9:

[2218] The generated summary is stored in a database and sent from the server to all participants via email, etc. The summary is used as a review of the meeting and is also useful for preparing for the next meeting.

[2219] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[2220] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search <url: https: gemini.google.com ?hl="ja">) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[2221] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the robot 414.

[2222] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[2223] FIG. 9 is a diagram illustrating an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and actions arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion includes both affect and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[2224] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[2225] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[2226] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. Emotions can also be created for robots, automobiles, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is indicated, and when they approach the ideal, a state of pleasure is indicated. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on Voice Emotion Recognition and Emotional Brain Physiological Signal Analysis Systems, Tokushima University, Doctoral Dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the "reaction" domain, where sensation is dominant. The right half of the emotion map lists emotions belonging to the "situation" domain, where situational awareness is dominant.

[2227] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[2228] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[2229] The system according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the system according to the present disclosure is not necessarily implemented on a server. The system according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program running on a personal computer or an application running on a smartphone, etc. The method according to the present disclosure may be provided to users in the form of SaaS (Software as a Service).

[2230] In the above embodiment, an example was given in which the specific processing is performed by one computer 22, but the technology of the present disclosure is not limited to this, and the specific processing may be distributed and performed by a plurality of computers including the computer 22. For example, the data generation model 58 may be provided in an external device of the data processing device 12, and data may be generated in the external device in accordance with input data.

[2231] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[2232] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[2233] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[2234] The hardware resource for executing a specific process can be any of the following processors: An example of a processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. Another example of a processor is a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[2235] The hardware resource that executes the specific processing may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific processing may be a single processor.

[2236] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[2237] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[2238] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[2239] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference.

[2240] The following is further disclosed regarding the above embodiment.

[2241] (Claim 1)

[2242] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[2243] a means for configuring and saving meeting details based on input information using a generative AI model;

[2244] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[2245] A generative AI model monitors meeting progress and facilitates conversations;

[2246] After the meeting, a means for automatically analyzing the meeting contents and generating a summary is provided.

[2247] The system includes means for storing and transmitting the generated summaries to participants.

[2248] (Claim 2)

[2249] 10. The system of claim 1, further comprising means for displaying text entered by a user during a conference to other participants in real time.

[2250] (Claim 3)

[2251] 10. The system of claim 1, further comprising means for automatically translating the summary generated after the conference into multiple languages.

[2252] "Example 1"

[2253] (Claim 1)

[2254] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[2255] a means for configuring and saving meeting details based on input information using a generative AI model;

[2256] a means for users to join the meeting by clicking on a provided meeting link;

[2257] a means for connecting to a real-time communication server and transmitting a user ID and a conference ID;

[2258] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[2259] A generative AI model monitors meeting progress and facilitates conversations;

[2260] After the meeting, a means for automatically analyzing the meeting contents and generating a summary is provided.

[2261] The system includes means for storing and transmitting the generated summaries to participants.

[2262] (Claim 2)

[2263] 10. The system of claim 1, further comprising means for displaying text entered by a user during a conference to other participants in real time.

[2264] (Claim 3)

[2265] 10. The system of claim 1, further comprising means for automatically translating the summary generated after the conference into multiple languages.

[2266] "Application Example 1"

[2267] (Claim 1)

[2268] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[2269] a means for configuring and saving meeting details based on input information using a generative AI model;

[2270] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[2271] A generative AI model monitors meeting progress and facilitates conversations;

[2272] After the meeting, a means for automatically analyzing the meeting contents and generating a summary is provided.

[2273] a means for storing and transmitting the generated summaries to the participants;

[2274] In physical stores, staff can participate in meetings using smart devices and input their opinions in real time.

[2275] A means for the generative AI model to generate prompt comments during the meeting to move on to the next agenda item;

[2276] A means for displaying the generated promotional comments on the staff's smart devices;

[2277] The system includes a means for sending summaries generated from the meeting content to staff members' smart devices.

[2278] (Claim 2)

[2279] 10. The system of claim 1, further comprising means for displaying text entered by a user during a conference to other participants in real time.

[2280] (Claim 3)

[2281] 10. The system of claim 1, further comprising means for automatically translating the summary generated after the conference into multiple languages.

[2282] "Example 2: Combining Emotion Engines"

[2283] (Claim 1)

[2284] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[2285] a means for configuring and saving meeting details based on input information using a generative AI model;

[2286] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[2287] means for analyzing the emotional state of each user in real time using an emotion engine;

[2288] A generative AI model monitors the progress of the meeting and uses user emotional data to facilitate the conversation.

[2289] means for automatically analyzing the contents of a meeting after the meeting and generating a summary from the text data and emotion data;

[2290] The system includes means for storing and transmitting the generated summaries to participants.

[2291] (Claim 2)

[2292] 10. The system of claim 1, further comprising means for displaying text entered by a user during a conference to other participants in real time.

[2293] (Claim 3)

[2294] 10. The system of claim 1, further comprising means for automatically translating the summary generated after the conference into multiple languages.

[2295] "Application example 2 when combining emotion engines"

[2296] (Claim 1)

[2297] A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting;

[2298] a means for configuring and saving meeting details based on input information using a generative AI model;

[2299] means for receiving real-time typing text input from a user and broadcasting it to other participants;

[2300] A generative AI model monitors meeting progress and facilitates conversations;

[2301] means for analyzing and providing feedback to the user's emotional state;

[2302] After the meeting, a means for automatically analyzing the meeting contents and generating a summary is provided.

[2303] The system includes means for storing and transmitting the generated summaries to participants.

[2304] (Claim 2)

[2305] 10. The system according to claim 1, further comprising: means for displaying text entered by a user during a conference to other participants in real time; and means for analyzing and displaying an emotional state of the user in real time.

[2306] (Claim 3)

[2307] 10. The system of claim 1, further comprising means for automatically translating the summary generated after the conference into multiple languages. [Explanation of symbols]

[2308] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot< / url:> < / url:> < / url:> < / url:>

Claims

1. A means for the meeting organizer to enter the purpose, agenda, and participant list of the meeting; a means for configuring and saving meeting details based on input information using a generative AI model; means for receiving real-time typing text input from a user and broadcasting it to other participants; A generative AI model monitors meeting progress and facilitates conversations; After the meeting, a means for automatically analyzing the meeting contents and generating a summary is provided. The system includes means for storing and transmitting the generated summaries to participants.

2. 10. The system of claim 1, further comprising means for displaying text entered by a user during a conference to other participants in real time.

3. 10. The system of claim 1, further comprising means for automatically translating the summary generated after the conference into multiple languages.

Images