System

Abstract

To provide a system.SOLUTION: A system includes: means that recognizes voices of participants in a conference and makes the voices into a text; emotion analysis means that analyzes the emotions of the participants from text information of the voices of the participants that has been made into the text; conference progress support means that creates a comment for supporting the progress of the conference on the basis of a result of the analysis of the emotions of the participants, and present the created comment; and means that summarizes the details of utterances of the participants to create a minute book at the end of the conference.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional meetings require a lot of time and effort to create an agenda, proceed with the meeting, create minutes, etc. Furthermore, when it comes to proceeding with a meeting, it is difficult to grasp the appropriate timing for each speaker and to summarize the contents within the allotted time. [Means for solving the problem]

[0005] The present invention solves the above problems by providing a means for creating a draft agenda before a meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, and a means for compiling the contents of what was said in the meeting and creating minutes at the end of the meeting. In particular, the means for supporting the progress of the meeting during the meeting includes a means for coming up with comments to encourage participants to speak next, a means for coming up with comments for the facilitator to encourage speaking, and a means for providing a summary or closing remarks before the end of the meeting so that the meeting can be completed within the allotted time. In addition, the means for recognizing the speaker's voice and converting it into text uses speech recognition technology. [Brief explanation of the drawings]

[0006] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 11] FIG. 2 is a sequence diagram showing a flow of processing in the data processing system according to the first embodiment of the first form example. [Figure 12] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1. [Figure 13] FIG. 10 is a sequence diagram showing a processing flow of a data processing system in a second embodiment of the second form example. [Figure 14] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Embodiment Example 2. [Figure 15] FIG. 10 is a sequence diagram showing the flow of processing in a data processing system according to a third embodiment of the third embodiment. [Figure 16] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Embodiment 3. [Figure 17] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the first embodiment of the first form example when an emotion engine is combined. [Figure 18] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 1 of Embodiment 1 when an emotion engine is combined. [Figure 19] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the second embodiment of the second form example when an emotion engine is combined. [Figure 20] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 2 of Form Example 2 when an emotion engine is combined. [Figure 21] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in the third embodiment of the third form example when an emotion engine is combined. [Figure 22] FIG. 10 is a sequence diagram showing the flow of processing in the data processing system in Application Example 3 of Form Example 3 when an emotion engine is combined. DETAILED DESCRIPTION OF THE INVENTION

[0007] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0008] First, the terms used in the following description will be explained.

[0009] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (TENSOR PROCESSING UNIT (registered trademark)).

[0010] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0011] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0012] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0013] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0014] [First embodiment]

[0015] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0016] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0017] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0018] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0019] The reception device 38 includes a touch panel 38A, a microphone 38B, and the like, and receives user input. The touch panel 38A detects contact with an indicator (for example, a pen or a finger) to receive user input by the touch of the indicator. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0020] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form of expression that the user 20 can perceive (for example, audio and / or text). The display 40A displays visible information such as text and images in accordance with instructions from the processor 46. The speaker 40B outputs audio in accordance with instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0021] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0022] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0023] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0024] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0025] In the smart device 14, the processor 46 performs the reception output process. The storage 50 stores a reception output program 60. The reception output program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0026] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0027] "Example 1"

[0028] This embodiment of the present invention provides an interface for inputting the purpose and agenda of a meeting as a means for creating a draft agenda before the meeting. Specifically, a user inputs the purpose and agenda of the meeting as text, and a draft agenda is automatically generated based on that.

[0029] "Example 2"

[0030] In addition, speech recognition technology is used to recognize the speaker's voice and convert it into text. Specifically, what is said during a meeting is picked up by a microphone and converted into text in real time using speech recognition software. This converted text is used to support the progress of the meeting and to create minutes.

[0031] "Example 3"

[0032] As a means of supporting the progress of ongoing meetings, the system provides a means for suggesting comments to encourage participants to speak next, a means for suggesting comments for the facilitator to encourage speaking, and a means for providing a summary of the content and closing content before the end of the meeting so that the meeting can be wrapped up within the allotted time. Specifically, the system analyzes the progress of the meeting and the content of comments, and displays on the screen suggested comments to encourage speaking at the appropriate time. In addition, before the end of the meeting, it summarizes what has been said so far and provides suggested closing content.

[0033] "Example 4"

[0034] At the end of a meeting, we provide a means to compile meeting minutes by summarizing what was said during the meeting. Specifically, minutes are automatically generated based on the text of what was said during the meeting. These minutes can be used for reference after the meeting or to prepare for the next meeting.

[0035] The processing flow of each embodiment will be described below.

[0036] "Example 1"

[0037] Step 1: The user enters the purpose and agenda of the meeting.

[0038] Step 2: The system analyzes the input information and automatically generates a draft agenda.

[0039] Step 3: The generated agenda proposal is displayed to the user, who can make any necessary corrections or additions.

[0040] "Example 2"

[0041] Step 1: When the meeting starts, the microphone picks up what is being said during the meeting.

[0042] Step 2: The captured voice data is sent to speech recognition software and converted into text in real time.

[0043] Step 3: The transcribed remarks are used to support the progress of the meeting and to create minutes.

[0044] "Example 3"

[0045] Step 1: As the meeting progresses, the system analyzes the progress of the meeting and what is being said in real time.

[0046] Step 2: Based on the analysis results, the system displays suggested comments on the screen to encourage people to speak at the appropriate time.

[0047] Step 3: Before the meeting ends, the system summarizes what has been said so far and provides suggestions for closing remarks.

[0048] "Example 4"

[0049] Step 1: Once the meeting is over, the system automatically generates minutes based on the text of what was said during the meeting.

[0050] Step 2: The generated minutes are displayed to the user, who can make corrections or additions as needed.

[0051] Step 3: The final minutes are saved for post-meeting reference and to help prepare for the next meeting.

[0052] Example 1

[0053] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0054] In conventional conference systems, creating an agenda before a meeting was a time-consuming and difficult task, making it difficult to do efficiently. Supporting the progress of the meeting and creating minutes after the meeting also required manual work, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to automatically generate an appropriate agenda based on the purpose and topic of the meeting, which meant a lot of time was spent preparing for the meeting.

[0055] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0056] In this invention, the server includes means for a user to input the purpose and agenda of a meeting, means for transmitting the input data to the server, means for the server to analyze the received data and create and input prompt sentences to the generative AI model, means for the generative AI model to generate a draft agenda, means for the server to return the generated draft agenda to the terminal, and means for the terminal to display the draft agenda to the user, thereby enabling users to easily automatically generate draft agendas for meetings.

[0057] The "means for creating a draft agenda before a meeting" is a means for automatically generating a draft agenda based on the purpose and agenda of the meeting.

[0058] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[0059] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to ensure the smooth progress of meetings.

[0060] "Methods for summarizing what was said at a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said at a meeting and compiling it into minutes after the meeting has ended.

[0061] The "means for users to input the purpose and agenda of the meeting" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[0062] The "means for transmitting input data to a server" refers to a communication means for transmitting data input by a user to a server.

[0063] "Means for analyzing data received by the server and creating and inputting prompt sentences to the generative AI model" refers to means for analyzing data received by the server and creating prompt sentences to be input to the generative AI model based on that data.

[0064] "Means by which a generative AI model generates a draft agenda" refers to means by which a generative AI model automatically generates a draft agenda based on a prompt sentence.

[0065] The "means for the server to return the generated agenda proposal to the terminal" refers to the means for the server to transmit the generated agenda proposal to the terminal.

[0066] The "means by which the terminal displays the agenda proposal to the user" refers to the means by which the terminal visually displays the received agenda proposal to the user.

[0067] This invention is a system that automatically generates a draft agenda before a meeting, supports the progress of the meeting during the meeting, and creates minutes after the meeting. This system includes: a means for a user to input the purpose and agenda of the meeting; a means for transmitting the input data to a server; a means for the server to analyze the received data and create and input prompt sentences to a generative AI model; a means for the generative AI model to generate a draft agenda; a means for the server to return the generated draft agenda to a terminal; and a means for the terminal to display the draft agenda to the user.

[0068] Users access a dedicated interface using a web browser and enter the purpose and agenda of the meeting in text format. This interface is implemented as an HTML form, making it easy for users to enter information. For example, a user can enter the purpose of the meeting, "Determine a market launch strategy for a new product," and the agenda, "Market analysis, competitive research, marketing strategy, and sales plan."

[0069] The device sends the data entered by the user to the server as an HTTP POST request. The data is sent in JSON format. The server analyzes the received data and creates a prompt to input into the generative AI model. For example, it generates a prompt such as, "The purpose of the meeting is to decide on a market launch strategy for a new product, and the agenda will include market analysis, competitive research, marketing strategy, and sales plan. Please create a draft agenda based on this."

[0070] The server inputs prompt sentences into the generative AI model to generate a draft agenda. The generated draft agenda is temporarily stored on the server, and the server sends it back to the device. The device analyzes the received draft agenda and displays it in a format that is easy for the user to view. For example, it is displayed on a web page in the following format:

[0071] Proposed agenda:

[0072] 1. Market analysis (10:00 - 10:30)

[0073] Review of current market conditions

[0074] Analysis of key market trends

[0075] 2. Competitive Research (10:30 - 11:00)

[0076] Strategies and trends of major competitors

[0077] Analysis of the strengths and weaknesses of competing products

[0078] 3. Marketing Strategy (11:00 - 11:30)

[0079] Identifying your target market

[0080] Marketing campaign planning

[0081] 4. Sales Planning (11:30 - 12:00)

[0082] Selection of sales channels

[0083] Setting sales targets

[0084] In this way, users can easily automatically generate meeting agendas. Support for meeting progress and the creation of meeting minutes after the meeting are also automated, allowing for efficient preparation, progress, and post-meeting processes.

[0085] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0086] Step 1:

[0087] The user inputs the purpose and agenda of the meeting.

[0088] A user opens a web browser and accesses a dedicated interface. The interface has a text input field where the user inputs the purpose and agenda of the meeting. For example, the purpose is to "decide on a market launch strategy for a new product" and the agenda is to "market analysis, competitive research, marketing strategy, and sales plan." The input data is in text format.

[0089] Step 2:

[0090] The terminal sends the input data to the server.

[0091] When the user completes the input and clicks the submit button, the device sends the input data to the server as an HTTP POST request. The data sent is in JSON format, and looks something like this:

[0092] json

[0093] {

[0094] "Objective": "Determine a new product launch strategy",

[0095] "Agenda": "Market analysis, competitive research, marketing strategy, sales plan"

[0096] }

[0097] Step 3:

[0098] The server receives the data and creates and inputs prompts to the generative AI model.

[0099] The server analyzes the received data and creates a prompt to be input to the generative AI model. For example, it generates the following prompt:

[0100] "The purpose of the meeting is to decide on a market launch strategy for a new product, and the topics to be discussed will include market analysis, competitive research, marketing strategy, and sales plan. Please create a draft agenda based on this."

[0101] The input data is in JSON format, and the output data is a prompt in text format.

[0102] Step 4:

[0103] A generative AI model generates a draft agenda.

[0104] The server inputs a prompt to the generative AI model, which then generates a draft agenda based on the prompt. For example, the following draft agenda is generated:

[0105] 1. Market analysis (10:00 - 10:30)

[0106] Review of current market conditions

[0107] Analysis of key market trends

[0108] 2. Competitive Research (10:30 - 11:00)

[0109] Strategies and trends of major competitors

[0110] Analysis of the strengths and weaknesses of competing products

[0111] 3. Marketing Strategy (11:00 - 11:30)

[0112] Identifying your target market

[0113] Marketing campaign planning

[0114] 4. Sales Planning (11:30 - 12:00)

[0115] Selection of sales channels

[0116] Setting sales targets

[0117] The input data is a prompt, and the output data is a draft agenda in text format.

[0118] Step 5:

[0119] The server returns the generated agenda plan to the terminal.

[0120] The server returns the generated agenda in JSON format to the terminal. For example, the following JSON data is returned:

[0121] json

[0122] {

[0123] "agenda": [

[0124] {"Time": "10:00 - 10:30", "Topic": "Market Analysis", "Details": ["Review of the current market situation", "Analysis of key market trends"]},

[0125] {"Time": "10:30 - 11:00", "Topic": "Competitive Research", "Details": ["Strategies and Trends of Major Competitors", "Analysis of Strengths and Weaknesses of Competitive Products"]},

[0126] {"Time": "11:00 - 11:30", "Topic": "Marketing Strategy", "Details": ["Identifying Target Market", "Planning a Marketing Campaign"]},

[0127] {"Time": "11:30 - 12:00", "Item": "Sales plan", "Details": ["Sales channel selection", "Sales target setting"]}

[0128] ]

[0129] }

[0130] The input data is a draft agenda in text format, and the output data is a draft agenda in JSON format.

[0131] Step 6:

[0132] The terminal displays the proposed agenda to the user.

[0133] The device parses the received agenda proposal and displays it in a user-friendly format, for example on a web page in the following format:

[0134] Proposed agenda:

[0135] 1. Market analysis (10:00 - 10:30)

[0136] Review of current market conditions

[0137] Analysis of key market trends

[0138] 2. Competitive Research (10:30 - 11:00)

[0139] Strategies and trends of major competitors

[0140] Analysis of the strengths and weaknesses of competing products

[0141] 3. Marketing Strategy (11:00 - 11:30)

[0142] Identifying your target market

[0143] Marketing campaign planning

[0144] 4. Sales Planning (11:30 - 12:00)

[0145] Selection of sales channels

[0146] Setting sales targets

[0147] The input data is a draft agenda in JSON format, and the output data is a visual representation of the draft agenda.

[0148] (Application example 1)

[0149] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0150] In conventional conference systems, creating the agenda before the meeting, providing support during the meeting, and creating minutes after the meeting were often done manually, which was time-consuming and labor-intensive. It was also difficult to create an agenda efficiently based on the purpose and topic of the meeting, which sometimes prevented the meeting from proceeding smoothly.

[0151] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0152] In this invention, the server includes a means for creating a draft agenda before the meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for compiling the contents of the meeting remarks and creating minutes at the end of the meeting, an interface means for the user to input the purpose and agenda of the meeting as text, a means for automatically generating a draft agenda using a generative AI model, and a means for displaying and sharing the generated draft agenda with the user. This makes it possible to efficiently carry out everything from preparation to progress of the meeting and follow-up after it ends.

[0153] The "means for creating a draft agenda before a meeting" is a means for automatically creating a draft agenda for planning the progress of a meeting based on the purpose and agenda of the meeting.

[0154] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[0155] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[0156] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said during the meeting and compiling it into minutes after the meeting has ended.

[0157] The "interface means for users to input the purpose and agenda of the meeting as text" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[0158] "Means for automatically generating a draft agenda using a generative AI model" refers to means for automatically generating a draft agenda based on the input meeting purpose and agenda items using a generative AI model.

[0159] The "means for displaying and sharing the generated agenda draft with the user" refers to a means for displaying the generated agenda draft to the user and sharing it with other participants as necessary.

[0160] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what was said in the meeting and creating minutes at the end of the meeting, an interface means for users to input the purpose and agenda of the meeting as text, a means for automatically generating a draft agenda using a generative AI model, and a means for displaying and sharing the generated draft agenda with users.

[0161] System program processing explanation

[0162] Hardware used

[0163] Smartphone

[0164] server

[0165] Software used

[0166] Python (registered trademark)

[0167] OpenAI(R) API

[0168] Data processing and calculation

[0169] 1. User Input:

[0170] The user opens the smartphone application and inputs the purpose and agenda of the meeting in text format, thereby functioning as an interface means for the user to input the purpose and agenda of the meeting in text format.

[0171] 2. Use of generative AI models:

[0172] Based on the input information, a prompt is sent to a generative AI model (OpenAI's GPT-3 (registered trademark)). This is where the means to automatically generate a draft agenda using the generative AI model comes into play.

[0173] 3. Generate a draft agenda:

[0174] GPT-3 generates a draft agenda based on the prompt and returns it to the application, which then provides a way to display and share the draft agenda with the user.

[0175] 4. Speech Recognition:

[0176] During a meeting, the speaker's voice is recognized in real time and converted into text data. This is where the means for recognizing the speaker's voice and converting it into text comes into play.

[0177] 5. Progression support:

[0178] To facilitate smooth progress in the meeting, appropriate comments and instructions are provided to participants and facilitators. This is where ongoing meeting support measures come into play.

[0179] 6. Minutes Preparation:

[0180] After the meeting, the contents of what was said during the meeting are organized and compiled as minutes. This is where the means of compiling the contents of what was said during the meeting and creating minutes at the end of the meeting comes into play.

[0181] Specific examples

[0182] A store manager holds a sales strategy meeting for a new product, and uses a smartphone app to input the purpose and agenda of the meeting. For example, he might enter a prompt like this:

[0183] Meeting purpose: New product sales strategy

[0184] agenda:

[0185] market analysis

[0186] Competitive Research

[0187] Selection of sales channels

[0188] Promotion Plan

[0189] Create a draft agenda based on this information.

[0190] The application automatically generates an agenda that can be shared with all staff members, improving meeting efficiency. In this way, meetings can be efficiently conducted from preparation to progress, and follow-up after the meeting has ended.

[0191] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0192] Step 1:

[0193] A user opens a smartphone application and inputs the purpose and agenda of the meeting in text format. The input text data is generated as a prompt sentence including the purpose and agenda of the meeting. The input data is the "purpose of the meeting" and "agenda," and the output data is the prompt sentence.

[0194] Step 2:

[0195] The device sends the generated prompt to the server. The server receives the prompt and sends it to the generative AI model (OpenAI's GPT-3). The input data is the prompt, and the output data is a request to the generative AI model.

[0196] Step 3:

[0197] The server receives the response from the generative AI model. The generative AI model generates a draft agenda based on the prompt and returns the result to the server. The input data is the prompt, and the output data is the generated draft agenda.

[0198] Step 4:

[0199] The server sends the generated agenda plan to the terminal, which receives the agenda plan and displays it to the user. The input data is the generated agenda plan, and the output data is the agenda plan displayed to the user.

[0200] Step 5:

[0201] During a meeting, the device recognizes the speaker's voice in real time and converts it into text data. Speech recognition technology is used to convert the speaker's voice into text. The input data is the speaker's voice, and the output data is the text of what was said.

[0202] Step 6:

[0203] The server receives the textual content of speech and generates appropriate comments and instructions to support the progress of the meeting. This provides appropriate comments and instructions to participants and the facilitator. The input data is the textual content of speech, and the output data is the generated comments and instructions.

[0204] Step 7:

[0205] After the meeting ends, the server organizes the content of the meeting and compiles it into minutes. The generated minutes are sent to the terminal and displayed to the user. The input data is the text of the comments, and the output data is the generated minutes.

[0206] Example 2

[0207] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0208] With conventional conferencing systems, it was difficult to convert statements made during a meeting into text in real time and use it to support the progress of the meeting or to create minutes. Furthermore, there was a lack of efficient means to carry out a series of processes, such as collecting, transmitting, converting, displaying, and saving audio data. This resulted in reduced meeting efficiency and the time it took to create minutes.

[0209] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0210] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of what was said at the meeting and creating minutes at the end of the meeting, means for collecting voice data, means for transmitting the voice data, means for converting the voice data into text data, means for displaying the text data in real time, and means for saving the text data. This makes it possible to convert what is said during the meeting into text in real time, and to efficiently support the progress of the meeting and create minutes.

[0211] A "method for drafting an agenda before a meeting" is a method for planning the agenda and progress of a meeting and sharing it with participants before the meeting begins.

[0212] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text in real time using voice recognition technology.

[0213] "Means for supporting the progress of a meeting in progress" are means for encouraging people to speak and supporting the progress of the agenda in order to ensure the smooth progress of the meeting.

[0214] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a method for organizing what was said during the meeting and saving it as an official record when the meeting ends.

[0215] The "means for collecting voice data" refers to a means for collecting speeches made during a meeting as voice data using a device such as a microphone.

[0216] The "means for transmitting voice data" refers to a means for transmitting collected voice data to a server or other device.

[0217] The "means for converting voice data into text data" refers to a means for converting voice data into text data using voice recognition technology.

[0218] The "means for displaying text data in real time" refers to a means for displaying the converted text data in real time during a conference so that participants can check it.

[0219] The "means for saving text data" refers to a means for saving text data generated during a conference so that it can be referenced later.

[0220] MODE FOR CARRYING OUT THE INVENTION

[0221] The present invention relates to a system that converts statements made during a meeting into text in real time and uses the text to support the progress of the meeting and to create minutes. A specific embodiment of this system will be described below.

[0222] Hardware and software used

[0223] Hardware: microphone, device (computer, tablet, smartphone, etc.)

[0224] Software: speech recognition software (e.g., Google® Speech-to-Text API), conferencing applications

[0225] System Overview

[0226] The system includes the following main means:

[0227] 1. A means for creating agendas before a meeting: A means for users to plan the agenda and progress of a meeting and share it with participants.

[0228] 2. A means of recognizing the speaker's voice and converting it into text: A means of converting statements made during a meeting into text in real time using voice recognition technology.

[0229] 3. Support measures for ongoing meetings: These are measures to encourage speaking and help move the agenda forward in order to ensure smooth progress in the meeting.

[0230] 4. A means of summarizing what was said during the meeting and creating minutes at the end of the meeting: This is a means of organizing what was said during the meeting and saving it as an official record when the meeting ends.

[0231] 5. Means of collecting audio data: A means of collecting speech during a meeting as audio data using a device such as a microphone.

[0232] 6. Means for transmitting audio data: A means for transmitting collected audio data to a server or other device.

[0233] 7. Means for converting voice data into text data: A means for converting voice data into text data using voice recognition technology.

[0234] 8. A means for displaying text data in real time: A means for displaying converted text data in real time during a meeting so that participants can check it.

[0235] 9. A means of saving text data: A means of saving text data generated during a meeting for later reference.

[0236] Specific examples

[0237] For example, if someone says "Let's move on to the next topic" during a meeting, the following processing will occur:

[0238] The audio is picked up by a microphone connected to the device.

[0239] The terminal transmits the voice data to the server.

[0240] The server converts the voice data into text data using speech recognition software (Google Speech-to-Text API).

[0241] The server displays text data such as "Let's move on to the next topic" in the conference application in real time.

[0242] The user checks the displayed text and supports the progress of the meeting.

[0243] After the meeting ends, the server saves the text data as minutes, allowing users to access them later.

[0244] Prompt Sentence Examples

[0245] Examples of prompts to be input to a generative AI model include:

[0246] "Please explain the process flow of a system that converts statements made during a meeting into text in real time and is used to support the progress of the meeting and create meeting minutes. Specifically, please clearly state what hardware and software are used and what data processing and calculations are performed for each step from the start of the meeting to collecting audio, transmitting the audio data, converting the audio into text, and using the text data."

[0247] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[0248] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0249] Step 1:

[0250] A user starts a conference.

[0251] Input: A user launches a conferencing application and clicks the "Start Conference" button.

[0252] Data processing / calculation: The terminal notifies the server of the start of the conference and generates a conference ID.

[0253] Output: A conference ID is generated and the conference is started.

[0254] Specific operation: When a user launches a conferencing application and clicks the "Start conference" button, the terminal sends a request to start the conference to the server, and the server generates a conference ID.

[0255] Step 2:

[0256] The device collects the audio.

[0257] Input: Speeches made during the meeting are input into the microphone.

[0258] Data processing / computation: The microphone converts the voice into digital data, which the device stores in a buffer.

[0259] Output: Digital audio data is stored in a buffer.

[0260] How it works: A microphone connected to the device picks up what is being said during the meeting, converts the audio into digital data, and stores it in a buffer.

[0261] Step 3:

[0262] The terminal transmits the audio data.

[0263] Input: Buffered digital audio data.

[0264] Data processing / calculation: The terminal divides the voice data into packets and sends them to the server over the network.

[0265] Output: The audio data is sent to the server.

[0266] Specific operation: The device divides the audio data stored in the buffer into packets and sends them to the server over the network.

[0267] Step 4:

[0268] The server converts the audio into text.

[0269] Input: The audio data received by the server.

[0270] Data processing / calculation: The server converts the voice data into text data using speech recognition software (e.g., Google Speech-to-Text API).

[0271] Output: Text data is generated.

[0272] Specific operation: The server passes the received voice data to the voice recognition software, which converts the voice data into text data.

[0273] Step 5:

[0274] The server displays the text data in real time.

[0275] Input: Generated text data.

[0276] Data processing / calculation: The server sends the text data to the conferencing application and displays it in real time.

[0277] Output: The text data is displayed in the conferencing application.

[0278] Specific behavior: The server sends the generated text data to the conferencing application and displays it so that the user can view it in real time.

[0279] Step 6:

[0280] The server stores the text data.

[0281] Input: Generated text data.

[0282] Data processing / calculation: The server stores the text data in a database.

[0283] Output: Text data is saved.

[0284] Specific operation: The server stores the generated text data in a database so that users can access it after the meeting ends.

[0285] (Application example 2)

[0286] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0287] Conventional conferencing systems can convert statements made during a meeting into text in real time to support the progress of the meeting, but they cannot receive voice instructions at work sites such as factories and automatically carry out work based on those instructions. This reduces work efficiency at the site and increases the possibility of human error.

[0288] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for summarizing the contents of the meeting remarks and creating minutes at the end of the meeting, and means for receiving work instructions by voice, converting them into text in real time, and automating work based on the text. This makes it possible not only to support the progress of meetings and create minutes, but also to receive work instructions by voice at work sites such as factories and automatically perform work based on those instructions.

[0289] A "method for creating a draft agenda before a meeting" is a method for creating an agenda and a schedule before a meeting begins.

[0290] "Means for recognizing a speaker's voice and converting it into text" refers to means for converting a speaker's speech into text data in real time using voice recognition technology.

[0291] The "means for supporting the progress of the ongoing conference" are means for supporting the smooth progress of the conference.

[0292] "Means for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a means for summarizing what was said during the meeting and compiling it into minutes when the meeting ends.

[0293] "Means for receiving work instructions via voice, converting them into text in real time, and automating work based on that text" refers to means for converting work instructions given via voice into text in real time, and automatically performing work based on that text.

[0294] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what has been said at the meeting and creating minutes at the end of the meeting, and a means for receiving work instructions by voice, converting them into text in real time, and automating work based on that text.

[0295] The server uses speech recognition technology to convert the speaker's voice into text in real time. Specifically, it collects voice data using a microphone and converts it into text data using a speech recognition library. This text data is used to support the progress of the meeting and to create minutes.

[0296] The server also receives work instructions via voice at factories and other work sites, converts them into text in real time, and automatically performs tasks based on the text. For example, tasks such as assembling, inspecting, and transporting parts can be performed by voice command. This improves work efficiency at the site and reduces the occurrence of human error.

[0297] For example, if a voice command such as "Please assemble the parts" is given in a factory, the server will recognize the command and begin assembling the parts. At this time, voice recognition technology will be used to convert the command into text, and the robot will automatically carry out the work based on that text.

[0298] Examples of prompts to input to a generative AI model include:

[0299] Create a program that recognizes the voice command "Please assemble the parts" and causes the robot to begin assembling the parts.

[0300] In this way, by combining voice recognition technology and automation technology, the server can not only support the progress of meetings and create minutes, but also efficiently process and automate work instructions on-site.

[0301] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0302] Step 1:

[0303] The user inputs voice instructions through a microphone, and the input voice data is sent to the server.

[0304] Step 2:

[0305] The server converts the received voice data into text data using a voice recognition library. Specifically, it analyzes the voice data and generates corresponding text using voice recognition technology. The input is voice data and the output is text data.

[0306] Step 3:

[0307] The server analyzes the generated text data and understands the instructions. For example, it checks whether the instruction "Please assemble the parts" is included. The input is the text data, and the output is the analysis result.

[0308] Step 4:

[0309] The server selects an appropriate task based on the analysis results. For example, if the keyword "assembly" is included, it selects the task of assembling parts. The input is the analysis results, and the output is the selected task.

[0310] Step 5:

[0311] The server generates commands to execute the selected tasks and sends them to the robot. Specifically, it generates commands containing specific operation instructions for assembly tasks. The input is the selected task, and the output is the command to the robot.

[0312] Step 6:

[0313] The robot performs a task based on the command it receives, for example, starting an assembly task. The input is the command to the robot, and the output is the task performed.

[0314] Step 7:

[0315] The server monitors the progress of the work and issues further instructions as needed, for example, when the work is completed it issues the next instruction. The input is the progress of the work and the output is further instructions.

[0316] In this way, a series of steps is realized: the user issues instructions by voice, the server converts the instructions into text, analyzes them, selects an appropriate task, and the robot carries out that task.

[0317] Example 3

[0318] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0319] Conventional conferencing systems lacked sufficient support for smoothly advancing meetings, and in particular, the timing of encouraging participants to speak, summarizing the content, and providing closing remarks were often done manually. Furthermore, creating meeting minutes after the end of a meeting was time-consuming and inefficient. This led to a decrease in meeting productivity and the risk of important points being overlooked.

[0320] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0321] In this invention, the server includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for compiling the contents of the meeting comments and creating minutes at the end of the meeting, a means for collecting the progress of the meeting and the contents of the comments, a means for analyzing the text of the comments, a means for generating draft comments to encourage participants to speak, a means for generating draft comments for the facilitator, and a means for generating a summary of the contents and closing content before the end of the meeting.This enables smooth support for the progress of the meeting and encourages participants to speak at the appropriate time.In addition, by providing a summary of the contents and closing content before the end of the meeting and automatically creating minutes at the end of the meeting, the productivity of the meeting can be improved.

[0322] The "means for creating a draft agenda before a meeting" is a means for automatically creating an agenda and a schedule before a meeting begins.

[0323] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text data in real time using voice recognition technology.

[0324] "Means for supporting the progress of ongoing meetings" are means for providing comments and advice to participants and facilitators at appropriate times to ensure that meetings proceed smoothly.

[0325] "Means for summarizing the contents of statements made during a meeting and creating minutes at the end of the meeting" refers to a means for automatically summarizing the contents of statements made during a meeting and compiling them into minutes when the meeting ends.

[0326] The "means for collecting the progress of the meeting and the content of comments" is a means for collecting the progress of the meeting and the content of comments in real time.

[0327] "Means for text analysis of speech content" refers to means for analyzing collected text data of speech content and performing keyword extraction and sentiment analysis.

[0328] The "means for generating a comment proposal to encourage a participant to make a speech" is a means for generating a comment proposal to encourage a participant to make the next speech according to the progress of the conference.

[0329] The "means for generating a proposal for a facilitator's remarks" is a means for generating an appropriate proposal for a facilitator's remarks so that the facilitator can smoothly proceed with the meeting.

[0330] "Means for generating a summary or closing content before the end of the meeting" refers to a means for summarizing the content discussed so far and providing the information necessary for closing the meeting before the end of the meeting.

[0331] The present invention provides a system for smoothly supporting the progress of a meeting and automating the creation of minutes after the meeting. A specific embodiment of this system will be described below.

[0332] Hardware and software used

[0333] Hardware: Servers, devices (PCs, tablets, etc.)

[0334] Software: generative AI models (e.g., GPT-4®), meeting management systems, speech recognition software (e.g., Google Speech-to-Text API), text analysis tools (e.g., spaCy)

[0335] System configuration

[0336] 1. How to prepare an agenda before a meeting

[0337] The server automatically generates a draft agenda based on the purpose of the meeting and the list of participants, allowing the meeting to proceed smoothly.

[0338] 2. A means of recognizing the speaker's voice and converting it into text

[0339] The server uses speech recognition software to convert speech into text in real time, for example, using the Google Speech-to-Text API.

[0340] 3. Support for ongoing meetings

[0341] The server collects information about the progress of the meeting and the content of comments made through the meeting management system, and analyzes it using a text analysis tool. Based on the analysis results, it generates suggested comments to encourage participants to make their next comments, as well as suggested comments for the facilitator.

[0342] The device displays the generated comment proposal on the screen. For example, a comment such as "Please give your opinion on the following agenda item" may be displayed.

[0343] 4. A way to generate a summary or closing statement before the meeting ends

[0344] The server summarizes what has been said so far based on the progress of the meeting and the content of what has been said, and generates a proposal for a closing speech, which allows the meeting content to be summarized effectively.

[0345] The device displays the generated summary and closing statement on the screen. For example, a summary such as "To summarize the discussion so far, the following points are important."

[0346] 5. A way to summarize what was said at the meeting and create minutes at the end of the meeting

[0347] The server automatically generates minutes based on the text of what was said during the meeting, organizing the content in chronological order and highlighting the important points.

[0348] The device saves the generated minutes so that they can be referenced after the meeting. For example, a message such as "Meeting minutes have been automatically generated. Please use them to prepare for your next meeting" is displayed.

[0349] Specific examples

[0350] For example, if you want to generate a comment suggestion during a meeting such as "Please give your opinion on the next agenda item," you can input the following prompt sentence into the generative AI model:

[0351] Example prompt sentence:

[0352] Now that we have all expressed our opinions on the current topic, please give your opinion on the next topic.

[0353] Based on this prompt, the generative AI model generates appropriate comment proposals and displays them to participants and the facilitator. Before the meeting ends, the model provides a summary such as, "To summarize the discussion so far, the following points are important," and automatically generates minutes to save at the end of the meeting.

[0354] In this way, the present invention can smoothly support the progress of the meeting and improve the productivity of the meeting by automating the creation of minutes after the meeting. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[0355] Step 1:

[0356] Meeting progress and collection of comments

[0357] The server collects the progress of the meeting and the content of comments in real time through the meeting management system. Specifically, it converts the voice data spoken during the meeting into text using voice recognition software (e.g., Google Speech-to-Text API).

[0358] Input: Audio data during the meeting

[0359] Output: Text of what was said

[0360] Specific operation: The server calls the voice recognition software and converts the voice data into text data, which is then stored in the conference management system.

[0361] Step 2:

[0362] Text analysis of speech content

[0363] The server analyzes the collected text data using a text analysis tool (e.g., spaCy). Specifically, it extracts keywords from the comments and analyzes their sentiment.

[0364] Input: Translated speech

[0365] Output: Analysis results (keywords, sentiment analysis results, etc.)

[0366] Specific operation: The server uses text analysis tools to analyze the text data and extract important keywords and sentiment trends. The analysis results are used in the next step.

[0367] Step 3:

[0368] Generate comment suggestions to encourage participation

[0369] The server uses a generative AI model (e.g., GPT-4) to generate a comment proposal that prompts the participant to make the next statement. Specifically, the analysis results are input into the generative AI model as a prompt sentence.

[0370] Input: Analysis result, prompt statement

[0371] Output: Comment proposal

[0372] Specific operation: The server inputs the prompt sentence into the generative AI model to generate appropriate comment suggestions, which are then displayed on the device in the next step.

[0373] Step 4:

[0374] Generate comments for the facilitator

[0375] The server uses a generative AI model to generate suggested comments for the facilitator, specifically, advice and questions that will help the facilitator smoothly conduct the meeting.

[0376] Input: Analysis result, prompt statement

[0377] Output: Draft comments for facilitator

[0378] Specific operation: The server inputs the prompt sentence into the generative AI model and generates a comment proposal for the facilitator. The generated comment proposal is displayed on the facilitator's device.

[0379] Step 5:

[0380] Generate a summary and closing statement before the meeting ends

[0381] The server summarizes what has been said so far based on the progress of the meeting and the content of the remarks, and generates a proposal for closing remarks.

[0382] Input: Analysis results, history of comments

[0383] Output: Summary, closing contents

[0384] Specific operation: The server extracts important points based on the speech history and generates a summary and closing remarks, which are then displayed on the terminal before the end of the meeting.

[0385] Step 6:

[0386] Automatic generation of meeting minutes

[0387] The server automatically generates minutes based on the text of what was said during the meeting, organizing the content in chronological order and emphasizing the important points.

[0388] Input: Translated speech

[0389] Output: Auto-generated meeting transcript

[0390] Specific operation: The server organizes the transcribed remarks and generates minutes that highlight the important points. The generated minutes are saved on the device after the meeting and can be referenced.

[0391] (Application example 3)

[0392] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0393] In order for a meeting to proceed smoothly, it is important for participants and facilitators to speak at the appropriate time and to summarize the contents of the meeting efficiently. However, it is not easy to come up with ideas for comments to encourage the next person to speak during the meeting, or to summarize the contents and prepare closing remarks before the end of the meeting. In addition, creating minutes after the meeting takes time and effort. Effective methods to solve these problems are needed.

[0394] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[0395] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for summarizing what was said in the meeting and creating minutes at the end of the meeting, means for analyzing the progress of the meeting and the content of the comments using a generative AI model and generating draft comments to encourage the next comment, means for generating a summary of the content and closing content before the end of the meeting, and means for automatically generating minutes based on the content of the comments made in the meeting.This allows the meeting to proceed smoothly, enables participants and the facilitator to speak at the appropriate time, and enables the content of the meeting to be summarized efficiently.

[0396] The "means for creating a draft agenda before a meeting" is a means for automatically creating an agenda and a schedule before a meeting begins.

[0397] "Means for recognizing the speaker's voice and converting it into text" refers to means for recognizing the voice of a speaker during a conference in real time and converting it into text information.

[0398] "Means for supporting the progress of ongoing meetings" are means for providing participants and facilitators with suggested comments that encourage them to speak at appropriate times in order to ensure smooth progress in the meeting.

[0399] The "means for summarizing the contents of statements made during a meeting and creating minutes at the end of the meeting" is a means for automatically summarizing the contents of statements made during the meeting and creating minutes at the end of the meeting.

[0400] A "generative AI model" is a model that uses artificial intelligence technology to generate appropriate output from input data.

[0401] "A means for analyzing the progress of a meeting and the content of comments, and generating suggested comments to encourage the next comment" is a means for analyzing the progress of a meeting and the content of comments, and proposing what should be said next.

[0402] "Means for generating a summary of the content and closing content before the end of the meeting" refers to means for summarizing what has been said so far and generating content necessary to conclude the meeting before the end of the meeting.

[0403] "Means for automatically generating minutes based on the content of statements made during a meeting" refers to a means for automatically creating minutes based on the content of statements made during a meeting.

[0404] The system for carrying out the present invention includes a plurality of means for supporting the progress of a meeting and efficiently creating minutes. Specific embodiments are described below.

[0405] First, the server provides a means for automatically creating a draft agenda before a meeting is held. This means generates the meeting agenda and schedule based on past meeting data and pre-entered information.

[0406] Next, we provide a means to recognize the speaker's voice during a meeting and convert it into text in real time. This means uses speech recognition technology to convert the speaker's utterances into text information. Specifically, we can use speech recognition services such as Google Cloud Speech-to-Text API and IBM Watson (registered trademark) Speech to Text.

[0407] Furthermore, to support the progress of ongoing meetings, the company provides a means to use a generative AI model to analyze the progress of the meeting and the content of comments, and generate proposed comments to encourage the next comment. This means uses a generative AI model such as OpenAI's GPT-3 to generate proposed comments to facilitate the progress of the meeting.

[0408] Before the end of the meeting, we provide a means to summarize what has been said so far and generate a closing statement, which also uses a generative AI model to efficiently summarize the meeting content.

[0409] At the end of the meeting, we provide a means to automatically generate minutes based on what was said during the meeting. This means uses a generative AI model to organize what was said and compile it into minutes.

[0410] As a concrete example, imagine a meeting to discuss the introduction of a new production line in a factory. In this case, the server inputs the following prompt sentence into the generative AI model to generate a proposed comment that will prompt the next statement.

[0411] Example prompt sentence:

[0412] Analyze what was said in the meeting below and generate suggested comments to encourage the next one.

[0413] Participant A: Today we would like to discuss the introduction of a new production line.

[0414] Participant B: First, let's identify the problems with the current production line.

[0415] Participant C: The issues are inefficiency and inconsistent quality.

[0416] Suggested comments:

[0417] Using this prompt, the generative AI model can generate proposed comments to encourage the next person to speak, helping the meeting proceed smoothly. Furthermore, by summarizing what has been said so far and generating closing remarks before the end of the meeting, the content of the meeting can be efficiently summarized. At the end of the meeting, minutes can be automatically generated based on what has been said, significantly reducing post-meeting work.

[0418] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0419] Step 1:

[0420] The server creates a draft agenda before the meeting. It uses past meeting data and pre-entered information as input and generates a draft agenda based on this data. As output, it provides a draft agenda that includes the meeting topic and schedule. Specifically, it retrieves past meeting data from a database and generates a draft agenda using natural language processing technology.

[0421] Step 2:

[0422] The server recognizes the speaker's voice during the meeting and converts it into text in real time. It uses the speaker's voice data as input and converts it into text data using speech recognition technology. It provides the text of what was said as output. Specifically, it uses speech recognition services such as Google Cloud Speech-to-Text API and IBM Watson Speech to Text.

[0423] Step 3:

[0424] The server uses a generative AI model to analyze the progress of the meeting and the content of comments, and generates suggested comments to encourage the next remark. As input, it uses the textual content of the comments and inputs a prompt into the generative AI model. As output, it provides suggested comments to encourage the next remark. Specifically, it uses a generative AI model such as OpenAI's GPT-3 to generate suggested comments based on the prompt.

[0425] Step 4:

[0426] The server generates a summary of the meeting content and closing remarks before the end of the meeting. It uses the remarks made during the meeting as input and inputs prompts into the generative AI model. It provides a summary of the meeting and closing remarks as output. Specifically, it uses the generative AI model to summarize the remarks and generate closing remarks.

[0427] Step 5:

[0428] At the end of the meeting, the server automatically generates minutes based on what was said during the meeting. It uses what was said during the meeting as input and inputs prompts into the generative AI model. The output provides the automatically generated minutes. Specifically, the generative AI model is used to organize what was said and compile it into minutes.

[0429] Step 6:

[0430] The user checks the proposed agenda, textual comments, proposed comments encouraging the next speaker, the meeting summary, closing remarks, and minutes provided by the server, and makes corrections or additions as necessary. The various data provided by the server is used as input, and the final meeting materials are created as output. Specifically, the quality of the meeting is improved by the user checking the various data and making any necessary corrections or additions.

[0431] Furthermore, an emotion engine that estimates the user's emotion may be combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion.

[0432] "Example 1"

[0433] One embodiment of the present invention is a conference support system incorporating an emotion engine. This system recognizes a user's voice and extracts emotions from that voice. Specifically, the system analyzes the tone, volume, and speaking rate of the user's voice and estimates the user's emotional state from the results. For example, if the user's voice is loud and fast, it is determined that the user is likely excited.

[0434] "Example 2"

[0435] The emotion engine also adjusts the operation of the meeting progress support means based on the recognized emotions. For example, if it determines that the emotional state of all meeting participants is excited, the progress support means will suggest comments to encourage calm. Conversely, if it determines that the meeting has become monotonous, it will suggest comments to stimulate discussion.

[0436] "Example 3"

[0437] Furthermore, the emotion engine is also used to create minutes at the end of a meeting. Specifically, it records the emotional state of each speaker in the minutes and analyzes how that emotional state influenced the discussion. This allows for a more detailed understanding of the progress of the meeting.

[0438] "Example 4"

[0439] As a concrete example, imagine a meeting to discuss the progress of a project. If the project leader's tone of voice sounds discouraged when reporting progress, the emotion engine will capture that information and provide feedback to the progress support tool. Based on that information, the progress support tool will suggest encouraging comments to other members. This will lighten the mood in the meeting and promote productive discussion.

[0440] The processing flow of each embodiment will be described below.

[0441] "Example 1"

[0442] Step 1: Recognize the user's voice.

[0443] Step 2: Analyze the audio for tone, volume, speaking rate, etc.

[0444] Step 3: Estimate the user's emotional state from the analysis results.

[0445] "Example 2"

[0446] Step 1: The emotion engine determines the emotional state of all meeting participants.

[0447] Step 2: The progress support tool suggests comments based on the emotional state.

[0448] "Example 3"

[0449] Step 1: At the end of the meeting, the emotion engine records the emotional state of each speaker.

[0450] Step 2: Analyze how your emotional state influenced the discussion.

[0451] "Example 4"

[0452] Step 1: Project leader reports on progress.

[0453] Step 2: If the emotion engine senses disappointment in the tone of voice, it picks up on that information.

[0454] Step 3: The progress support tool uses this information to suggest encouraging comments to other members.

[0455] Example 1

[0456] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0457] Conventional meeting support systems have difficulty efficiently reflecting the purpose and topics of a meeting in the agenda, and lack support for proceedings that take into account the emotional state of participants during the meeting. As a result, meetings do not proceed smoothly, and creating minutes is a time-consuming process.

[0458] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0459] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of the meeting remarks and creating minutes at the end of the meeting, means for the user to input the purpose and agenda of the meeting, means for analyzing the input information and generating a draft agenda, means for analyzing the voice data and estimating the emotional state, and means for updating the agenda based on the emotional state. This makes it possible to efficiently reflect the purpose and agenda of the meeting in the agenda and to support the progress of the meeting while taking into account the emotional state of the participants.

[0460] The "means for creating a draft agenda before a meeting" is a means for automatically generating items necessary for the progress of a meeting based on the purpose and agenda of the meeting.

[0461] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[0462] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[0463] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said during the meeting and compiling it into minutes after the meeting has ended.

[0464] The "means for users to input the purpose and agenda of the meeting" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[0465] The "means for analyzing input information and generating a draft agenda" is a means for analyzing the purpose and agenda of the meeting input by the user and automatically generating a draft agenda based on the information.

[0466] The "means for analyzing voice data and estimating emotional states" is a means for analyzing voice data of speakers during a conference and estimating the emotional states of the speakers from the tone, volume, speaking speed, and the like.

[0467] The "means for updating the agenda based on emotional state" is a means for dynamically updating the meeting agenda based on the estimated emotional state.

[0468] MODE FOR CARRYING OUT THE INVENTION

[0469] This invention is a system for supporting the efficient running of meetings, creating a draft agenda before the meeting, supporting the progress of the meeting during the meeting, and creating minutes at the end of the meeting. A specific embodiment of this system will be described below.

[0470] 1. System Overview

[0471] This system works in cooperation with three parties: a server, a terminal, and a user. The server provides an interface for inputting the purpose and agenda of the meeting, and analyzes the input information to generate a draft agenda. It also recognizes the user's voice during the meeting, analyzes the voice data to estimate the user's emotional state, and dynamically updates the agenda. The terminal is responsible for sending the user's input and voice data to the server.

[0472] 2. Hardware and Software Used

[0473] The server uses the following hardware and software:

[0474] Generative AI models using natural language processing techniques (e.g., OpenAI's GPT-4)

[0475] Voice recognition software (e.g., Google Cloud Speech-to-Text)

[0476] Emotion engine (e.g. IBM Watson Tone Analyzer)

[0477] The terminal provides an interface for capturing user input and voice data and transmitting it to a server. Specifically, a computer or smartphone equipped with a microphone and keyboard is used.

[0478] 3. Program Processing

[0479] The user uses the device's interface to input the purpose and agenda of the meeting in text format, for example, "The purpose of the next project meeting is to review progress and share issues." The device then sends this information to the server, which uses natural language processing technology to analyze the input text and generate appropriate agenda items.

[0480] During a meeting, a user uses the device's microphone to input voice, such as saying, "This part is very important." The device captures the voice data in real time and sends it to the server. The server then uses speech recognition software to convert the voice into text and uses an emotion engine to analyze the user's tone, volume, speaking rate, etc. to estimate the user's emotional state. Based on the estimated emotional state, the server dynamically updates the agenda.

[0481] 4. Examples of concrete examples and prompts

[0482] For example, if a user types "The purpose of the next project meeting is to review progress and share issues," and during the meeting emphasizes "This part is very important," the server will generate and update the following agenda:

[0483] 1. Opening Remarks

[0484] 2. Project progress reports

[0485] 3. Sharing current issues

[0486] 4. Check important points (parts highlighted by the user)

[0487] 5. Question and Answer Session

[0488] 6. Confirm the next meeting date

[0489] 7. Closing Remarks

[0490] Examples of prompts to input to a generative AI model might include:

[0491] "The user has entered the purpose of the next project meeting as reviewing progress and sharing issues. Please create a draft agenda based on this."

[0492] In this way, a system is realized in which the server, terminals, and users work together to support efficient conference management.

[0493] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0494] Step 1:

[0495] The user inputs the purpose and agenda of the meeting.

[0496] The user uses the device's interface to input the purpose and agenda of the meeting in text format. For example, the user might input, "The purpose of the next project meeting is to review progress and share issues." The input data is in text format, and the device then sends this data to the server.

[0497] Step 2:

[0498] The terminal transmits the input information to the server.

[0499] The terminal sends the text data entered by the user to the server using an HTTP request. The input is the text data entered by the user, and the output is the text data sent to the server.

[0500] Step 3:

[0501] The server analyzes the input information and generates a draft agenda.

[0502] The server analyzes the received text data using natural language processing technology. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-4) to analyze the input text and generate appropriate agenda items. The input is the text data entered by the user, and the output is the generated agenda proposal. For example, it generates the following agenda proposal:

[0503] 1. Opening Remarks

[0504] 2. Project progress reports

[0505] 3. Sharing current issues

[0506] 4. Q&A

[0507] 5. Confirm the next meeting date

[0508] 6. Closing Remarks

[0509] Step 4:

[0510] A user inputs audio during a conference.

[0511] During a meeting, a user inputs voice using the device's microphone, for example, saying, "This part is very important." The input is the user's voice data, and the device captures this data in real time.

[0512] Step 5:

[0513] The terminal transmits the voice data to the server.

[0514] The device sends the captured audio data to the server using an HTTP request. The input is the user's audio data, and the output is the audio data sent to the server.

[0515] Step 6:

[0516] The server analyzes the voice data and estimates the emotional state.

[0517] The server converts the speech into text using speech recognition software (e.g., Google Cloud Speech-to-Text). It then uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the tone, volume, and speaking rate of the speech and estimate the user's emotional state. The input is the speech data, and the output is the estimated emotional state. For example, if the voice is loud and the speaking rate is fast, it is determined that the user is excited.

[0518] Step 7:

[0519] The server updates the agenda based on the emotional state.

[0520] The server dynamically updates the agenda based on the estimated emotional state. The input is the estimated emotional state and the existing agenda proposal, and the output is the updated agenda proposal. For example, if a user gets excited and says, "This part is very important," the server adds that part to the agenda as a particularly important topic.

[0521] (Application example 1)

[0522] Next, a description will be given of Application Example 1 of Embodiment Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0523] Conventional meeting support systems were unable to take into account the emotional state of participants when creating meeting agendas or supporting progress, which could result in a decline in meeting efficiency and quality. Furthermore, they lacked the functionality to automatically generate agendas based on the purpose and topic of the meeting, which resulted in the problem of requiring a lot of time and effort to prepare for meetings.

[0524] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for summarizing the contents of the meeting remarks and creating minutes at the end of the meeting, a means for extracting emotions from the participants' voices and providing feedback in real time, and a means for automatically generating a draft agenda using a generative AI model. This makes meeting preparation more efficient, enables feedback based on the participants' emotional states, and improves the quality and efficiency of meetings.

[0525] The "means for creating a draft agenda before a meeting" is a means for automatically generating a draft agenda for a meeting by inputting the purpose and subject of the meeting.

[0526] The "means for recognizing and converting the speaker's voice into text" refers to a means for collecting the voice of the speaker during a conference and converting it into text data.

[0527] "Means for supporting the progress of an ongoing meeting" are means for encouraging participants to speak and providing a summary of the meeting in order to ensure smooth progress of the meeting.

[0528] "Means for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a means for organizing what was said during the meeting and compiling it into minutes after the meeting has ended.

[0529] The "means for extracting emotions from the participants' voices and providing feedback in real time" refers to a means for analyzing the participants' voice data to estimate their emotional state and providing feedback on the results in real time.

[0530] "Means for automatically generating a draft agenda using a generative AI model" means means for automatically generating a draft agenda based on the input meeting purpose and agenda items using a generative AI model.

[0531] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for summarizing the contents of the meeting remarks and creating minutes at the end of the meeting, a means for extracting emotions from the participants' voices and providing feedback in real time, and a means for automatically generating a draft agenda using a generative AI model.

[0532] Hardware and Software Configuration

[0533] 1. Hardware:

[0534] Robot body: Meeting support robot (e.g., Pepper (registered trademark))

[0535] Microphone: For voice input

[0536] Speaker: For audio output

[0537] Display: for displaying agendas and feedback

[0538] 2. Software:

[0539] Speech recognition engine: Google Cloud Speech-to-Text

[0540] Sentiment analysis engine: IBM Watson Tone Analyzer

[0541] Generative AI model: OpenAI GPT-4

[0542] Conference Management System: MICROSOFT® TEAMS® API

[0543] Data processing and calculation

[0544] 1. Voice input and recognition:

[0545] The robot collects the user's voice through its microphone and converts it into text using a speech recognition engine.

[0546] Example: A user says, "The purpose of our next meeting is to improve the quality of our new product. The agenda will include reviewing our manufacturing process and strengthening quality inspections."

[0547] 2. Sentiment analysis:

[0548] The converted text and voice data is sent to a sentiment analysis engine to estimate the user's emotional state.

[0549] Example: If the voice is loud and speaking quickly, we determine that the user is excited.

[0550] 3. Agenda generation:

[0551] The purpose and agenda of the meeting are input into a generative AI model to automatically generate a draft agenda.

[0552] Example prompt: "The purpose of the next meeting is to improve the quality of our new product. The agenda will include reviewing the manufacturing process and strengthening quality inspections. Please create a draft agenda based on this."

[0553] 4. Real-time feedback:

[0554] It analyzes audio data collected during meetings in real time and provides feedback based on emotional state.

[0555] For example: If a participant is agitated, provide feedback such as, "Let's calm down a bit."

[0556] Specific examples

[0557] 1. Before the meeting:

[0558] The user speaks to the robot, saying, "The purpose of the next meeting is to improve the quality of our new products. The agenda will include reviewing the manufacturing process and strengthening quality inspections."

[0559] The robot converts the speech into text, which is then fed into a generative AI model to generate a draft agenda.

[0560] The proposed agenda will be displayed on the screen and the user will be asked to confirm it.

[0561] 2. During the meeting:

[0562] The robot will assist in the meeting and analyze participants' voices in real time.

[0563] For example, if a participant is agitated, provide feedback such as, "Please calm down a bit."

[0564] In this way, meeting preparation can be streamlined, feedback based on participants' emotional states can be provided, and the quality and efficiency of meetings can be improved.

[0565] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0566] Step 1:

[0567] The user speaks to the robot to describe the purpose and agenda of the meeting. The robot's microphone collects the voice data and sends it to a speech recognition engine (Google Cloud Speech-to-Text). The speech recognition engine converts the voice data into text data and sends the text data to the server.

[0568] Input: User's voice data

[0569] Output: Text data

[0570] Step 2:

[0571] The server inputs the received text data into a generative AI model (OpenAI GPT-4) to automatically generate a draft agenda. The generated draft agenda is sent from the server to the robot's display and displayed to the user.

[0572] Input: Text data

[0573] Output: Agenda proposal

[0574] Step 3:

[0575] During the meeting, the robot's microphone collects participants' voices in real time and sends them to a speech recognition engine, which converts the voice data into text data and sends the text data to a server.

[0576] Input: Participant's voice data

[0577] Output: Text data

[0578] Step 4:

[0579] The server sends the received text and voice data to a sentiment analysis engine (IBM Watson Tone Analyzer) to estimate the participants' emotional state. The sentiment analysis engine analyzes the emotional state and sends the results to the server.

[0580] Input: Text data, audio data

[0581] Output: Emotional state data

[0582] Step 5:

[0583] The server generates real-time feedback based on the emotional state data and provides it to participants through the robot's speaker. For example, if a participant is excited, the server will provide feedback such as "Please calm down a bit when you speak."

[0584] Input: Emotional state data

[0585] Output: Feedback audio

[0586] Step 6:

[0587] At the end of the meeting, the server organizes the text data collected during the meeting and automatically generates minutes, which are then sent from the server to the robot's display and displayed to the user.

[0588] Input: Text data during the meeting

[0589] Output:Minutes

[0590] In this way, meeting preparation can be streamlined, feedback based on participants' emotional states can be provided, and the quality and efficiency of meetings can be improved.

[0591] Example 2

[0592] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0593] Conventional conference systems have difficulty converting statements made during a meeting into text in real time and generating comments to support the progress of the meeting. It is also difficult to analyze the emotional state of meeting participants and provide appropriate comments based on that. This has led to problems such as reduced efficiency and effectiveness of meetings.

[0594] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[0595] In this invention, the server includes means for creating a draft agenda before the meeting, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of the meeting comments and creating minutes at the end of the meeting, means for analyzing emotions, means for generating comments to support the progress of the meeting based on the results of the emotion analysis, and means for presenting the generated comments. This makes it possible to convert comments made during the meeting into text in real time and provide appropriate comments based on the emotional state.

[0596] A "method for drafting an agenda before a meeting" is a method for planning the agenda and progress of a meeting and sharing it with participants before the meeting begins.

[0597] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text data in real time using voice recognition technology.

[0598] "Means for supporting the progress of an ongoing meeting" refers to means for providing comments and instructions at appropriate times to ensure that the meeting proceeds smoothly.

[0599] "Methods for summarizing what has been said at a meeting and creating minutes at the end of the meeting" refers to a method for organizing what has been said at a meeting and saving it as an official record after the meeting has ended.

[0600] The "emotion analysis means" is a means for analyzing text data obtained by speech recognition and identifying the emotional state of the speaker and conference participants.

[0601] The "means for generating comments to support the progress of a meeting based on the result of emotion analysis" is a means for automatically generating comments to facilitate the progress of a meeting based on the result of emotion analysis.

[0602] The "means for presenting the generated comments" is a means for displaying the generated comments to the conference participants and supporting the progress of the conference.

[0603] This invention is a system for improving the efficiency and effectiveness of meetings, and includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what has been said at the meeting and creating minutes at the end of the meeting, a means for analyzing emotions, a means for generating comments that support the progress of the meeting based on the results of the emotion analysis, and a means for presenting the generated comments.

[0604] Hardware and software used

[0605] Hardware

[0606] Device: A computer or smartphone with a microphone

[0607] Server: A server with high-performance data processing capabilities

[0608] software

[0609] Speech recognition software: Google Speech-to-Text API

[0610] Sentiment analysis software: IBM Watson Tone Analyzer

[0611] Generative AI model: OpenAI GPT-3

[0612] Explanation of program processing

[0613] Acquiring voice input

[0614] When a user speaks during a conference, the microphone connected to the device picks up the voice, and the device transmits the captured voice data to the server.

[0615] Speech to text

[0616] The server passes the received voice data to speech recognition software (Google Speech-to-Text API) and converts it into text in real time, which is then passed on to the next processing step.

[0617] Sentiment analysis of text data

[0618] The server sends the converted speech to emotion analysis software (IBM Watson Tone Analyzer) to analyze the emotional state. The analysis results are used to support the progress of the meeting.

[0619] Generate comments to support the progress of the meeting

[0620] The server generates comments to support the progress of the meeting based on the results of sentiment analysis. Appropriate comments are generated using a generative AI model (OpenAI GPT-3).

[0621] Comment suggestion

[0622] The server transmits the generated comments to the terminal, and the terminal presents the comments to the user, who then proceeds with the conference by referring to the presented comments.

[0623] Specific examples

[0624] Specific examples of voice recognition

[0625] When a user says, "How is this project going?", the device's microphone picks up the speech. The server uses the Google Speech-to-Text API to convert this speech into text: "How is this project going?"

[0626] Specific examples of sentiment analysis

[0627] The server then sends the converted speech to the IBM Watson Tone Analyzer, which analyzes the user's emotional state. For example, emotions such as "excitement" or "anxiety" can be identified from the speech.

[0628] Examples of meeting support

[0629] Based on the results of the emotion analysis, the server generates comments to support the progress of the meeting. For example, if it determines that meeting participants are excited, it sends a prompt to the generative AI model saying, "Please generate a comment to encourage meeting participants to calm down when they are excited," and obtains a comment such as, "Everyone, let's calm down a bit."

[0630] Prompt Sentence Examples

[0631] "Generate comments to calm meeting participants when they get excited."

[0632] Using this prompt, the generative AI model can generate comments to effectively support the progress of the meeting.

[0633] The flow of the identification process in the second embodiment will be described with reference to FIG.

[0634] Step 1:

[0635] When a user speaks during a conference, the microphone connected to the device picks up the voice, and the device transmits the captured voice data to the server.

[0636] Input: User's voice

[0637] Output: Audio data sent to the server

[0638] Specific operation: When a user says, "How is this project progressing?", the device's microphone captures the voice and sends it to the server as audio data.

[0639] Step 2:

[0640] The server passes the received voice data to speech recognition software (Google Speech-to-Text API) and converts it into text in real time, which is then passed on to the next processing step.

[0641] Input: Audio data

[0642] Output: Text data

[0643] Specific operation: The server sends the voice data to the Google Speech-to-Text API and receives the text data, "How is the progress on this project?"

[0644] Step 3:

[0645] The server sends the converted speech to emotion analysis software (IBM Watson Tone Analyzer) to analyze the emotional state. The analysis results are used to support the progress of the meeting.

[0646] Input: Text data

[0647] Output: Emotion analysis results

[0648] Specific operation: The server sends the text "How is the progress on this project?" to the IBM Watson Tone Analyzer and obtains emotional states such as "excited" or "anxious."

[0649] Step 4:

[0650] The server generates comments to support the progress of the meeting based on the results of sentiment analysis. Appropriate comments are generated using a generative AI model (OpenAI GPT-3).

[0651] Input: Sentiment analysis results

[0652] Output: Generated comments

[0653] Specific operation: If the server determines that the emotional state is "excited," it sends a prompt to the generative AI model saying, "Please generate a comment to encourage meeting participants to calm down when they are excited," and obtains a comment such as, "Everyone, let's calm down a bit."

[0654] Step 5:

[0655] The server transmits the generated comments to the terminal, and the terminal presents the comments to the user, who then proceeds with the conference by referring to the presented comments.

[0656] Input: Generated comment

[0657] Output: Comments presented to the user

[0658] Specific operation: The server sends a comment to the terminal saying, "Everyone, let's calm down a bit." The terminal then displays the comment to the user. The user then uses the displayed comment as a reference to proceed with the meeting.

[0659] (Application example 2)

[0660] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0661] Conventional conference systems lack the ability to convert statements made during a meeting into text in real time or to support the progress of the meeting. Furthermore, because the progress of the meeting is not adjusted taking into account the emotional state of the participants, the meeting can become stagnant or lack vitality. This reduces the efficiency of the meeting and takes time to create minutes.

[0662] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[0663] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of what was said in the meeting and creating minutes at the end of the meeting, means for recognizing the emotional states of the meeting participants, and means for adjusting the progress of the meeting based on the emotional states. This makes it possible to convert what is said during the meeting into text in real time and adjust the progress of the meeting based on the emotional states.

[0664] A "method for creating an agenda before a meeting" is a method for planning the agenda and progress of a meeting and providing it to participants before the meeting begins.

[0665] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text information in real time using voice recognition technology.

[0666] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[0667] "Methods for summarizing what was said at a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said at the end of a meeting and compiling it into an official record.

[0668] The "means for recognizing the emotional state of the conference participants" is a means for analyzing the emotions of the participants during the conference and grasping their state.

[0669] The "means for adjusting the progress of a meeting based on emotional states" refers to a means for appropriately changing the way the meeting is conducted in accordance with the recognized emotional states.

[0670] A "generative AI model" is a model that uses artificial intelligence technology to generate data and perform specific tasks.

[0671] A "prompt" is an instruction given to a generative AI model to obtain a specific output.

[0672] A system for implementing this invention has the following configuration: A server includes means for creating a draft agenda before a meeting begins, means for recognizing and converting the speaker's voice into text, means for supporting the progress of the meeting during the meeting, means for summarizing what has been said at the meeting and creating minutes at the end of the meeting, means for recognizing the emotional states of the meeting participants, and means for adjusting the progress of the meeting based on their emotional states.

[0673] Hardware and Software Configuration

[0674] Microphone: Used to pick up audio during the meeting.

[0675] Server: Performs various processes such as speech recognition, emotion recognition, progress support, and minutes creation.

[0676] Speech recognition software: Uses speech recognition libraries to convert speech to text.

[0677] Emotion recognition software: Analyzes emotions from text using the pipeline function from the transformers library.

[0678] Generative AI model: Used to generate progress support messages based on emotional state.

[0679] Processing flow

[0680] 1. Speech recognition: The server converts the voice data acquired from the microphone into text in real time using a speech recognition library.

[0681] 2. Emotion recognition: The server performs emotion analysis on the textual content of the speech using the pipeline function of the transformers library.

[0682] 3. Progression support: Based on the results of the sentiment analysis, the server inputs prompt sentences into the generative AI model to generate appropriate progress support messages.

[0683] 4. Creating minutes: At the end of the meeting, the server creates minutes by compiling the text of the remarks and the support messages for the meeting.

[0684] Specific examples

[0685] For example, if someone says "This project is going well" during a meeting, the server converts this speech into text and the result of the sentiment analysis is "POSITIVE." Based on this, the following prompt sentence is input into the generative AI model:

[0686] This is a lively meeting. Let's move on to the next item on the agenda.

[0687] The generated progress support message is "The meeting is lively. Let's move on to the next agenda item." and is presented to the meeting participants.

[0688] In this way, the server can facilitate the smooth progress of the meeting and realize efficient minutes creation.

[0689] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[0690] Step 1:

[0691] The server acquires voice data from the microphone. When a user speaks during a conference, the voice is sent to the server through the microphone. The input is voice data, and the output is the acquisition of voice data.

[0692] Step 2:

[0693] The server converts the acquired voice data into text using a voice recognition library. Voice recognition technology is used to convert voice data into text data. The input is voice data, and the output is text data.

[0694] Step 3:

[0695] The server performs sentiment analysis on text data using the pipeline function of the transformers library. The text data is input into a sentiment analysis model to obtain the emotional state. The input is text data, and the output is the emotional state (e.g., POSITIVE, NEGATIVE, NEUTRAL).

[0696] Step 4:

[0697] Based on the results of the emotion analysis, the server inputs a prompt sentence into the generative AI model to generate a progress support message.The server inputs a prompt sentence according to the emotional state into the generative AI model to obtain an appropriate progress support message.The input is the emotional state and the prompt sentence, and the output is a progress support message.

[0698] Step 5:

[0699] The server presents the generated progress support message to the conference participants. The progress support message is communicated to the participants via a display device or an audio output device. The input is the progress support message, and the output is the message presented to the participants.

[0700] Step 6:

[0701] At the end of the meeting, the server compiles the textual content of the remarks and the support messages to create minutes. The server organizes the text data and support messages collected during the meeting and saves them in a file as minutes. The input is the text data and support messages, and the output is a minutes file.

[0702] Example 3

[0703] Next, a description will be given of a third embodiment of the third embodiment. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0704] Conventional meeting support systems lacked specific support for smoothly progressing meetings, and did not provide sufficient means to encourage participants to speak or to effectively summarize the meeting content. Furthermore, they lacked the functionality to analyze the content of comments made during the meeting and the emotional state of participants in real time and generate appropriate comments based on that analysis, which could result in a decrease in meeting productivity. Furthermore, when automatically generating minutes at the end of a meeting, the emotional state of the speakers could not be reflected, making it difficult to understand the progress of the meeting in detail.

[0705] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[0706] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of the meeting comments and creating minutes at the end of the meeting, means for analyzing the contents of the comments and emotional state, means for generating draft comments to encourage comments, and means for generating a summary of the contents and closing content before the end of the meeting. This makes it possible to smoothly proceed with the meeting and provide an environment in which all participants can actively participate.

[0707] A "method for creating a draft agenda before a meeting" is a method for planning the agenda and order of proceedings before a meeting and providing them to participants.

[0708] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text data in real time using voice recognition technology.

[0709] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[0710] "A means for summarizing what has been said at a meeting and creating minutes at the end of the meeting" is a means for automatically generating minutes based on the text of what has been said at the end of the meeting.

[0711] The "means for analyzing speech content and emotional state" is a means for analyzing the textual content of speech and audio data and evaluating the emotional state of each speaker.

[0712] The "means for generating proposed comments to encourage speech" is a means for generating proposed comments to encourage participants and the facilitator to make the next statement according to the progress of the meeting.

[0713] "Means for generating a summary of the content and closing content before the end of the meeting" refers to a means for summarizing what has been said so far and generating closing content before the end of the meeting.

[0714] This invention is a system for smoothly progressing a conference and providing an environment in which all participants can actively participate. This system includes elements of a server, a terminal, and a user, and operates as follows.

[0715] Hardware and software used

[0716] Hardware: Servers, devices (PCs, tablets, smartphones)

[0717] Software: Generative AI models (e.g., GPT-4), emotion engines, speech recognition software (e.g., Google Speech-to-Text)

[0718] System Operation Overview

[0719] A method for drafting agendas before a meeting

[0720] The server plans the agenda and order of the meeting beforehand and generates a draft agenda, which is then provided to participants via their terminals.

[0721] A means of recognizing a speaker's voice and converting it into text

[0722] The server uses voice recognition software to convert speech during the meeting into text data in real time. The voice data is collected from microphones installed in the meeting room and participants' devices.

[0723] Support for ongoing meetings

[0724] The server uses a generative AI model to generate suggested comments that encourage participants to speak according to the progress of the meeting. The generated suggested comments are displayed on the device screen.

[0725] The server uses an emotion engine to analyze the textual content of comments and the audio data, and evaluates the emotional state of each speaker. The evaluation results are fed back to the progress support means.

[0726] A way to generate a summary or closing statement before the end of a meeting

[0727] Before the end of the meeting, the server summarizes what has been said so far and generates a closing statement. The summary and closing statement are displayed on the terminal screen.

[0728] A method for summarizing what was said at a meeting and creating minutes at the end of the meeting

[0729] The server automatically generates minutes based on the text of the remarks at the end of the meeting. The emotion engine records the emotional state of each speaker in the minutes and analyzes how those emotions influenced the discussion. The generated minutes are stored in a database and can be accessed by users.

[0730] Examples of specific examples and prompts

[0731] Examples:

[0732] Imagine a meeting to discuss the progress of a project. If the project leader's tone of voice sounds discouraged when reporting progress, the emotion engine will capture that information and provide feedback to the progress support tool. Based on that information, the progress support tool will suggest encouraging comments to other members. This will lighten the mood in the meeting and promote productive discussion.

[0733] Example prompt sentence:

[0734] "If a project leader sounds discouraged while reporting progress, what encouraging comments should you suggest to other team members?"

[0735] This system aims to facilitate smooth progress of the conference and provide an environment in which all participants can actively participate. The flow of the identification process in the third embodiment will be described with reference to FIG.

[0736] Step 1:

[0737] Collecting meeting audio data

[0738] The server collects audio data from the conference in real time, obtained from microphones installed in the conference room and from participants' devices.

[0739] Input: Audio data from the conference room microphone and participants' devices

[0740] Output: Audio data stored on the server

[0741] Specific operation: The server starts the audio stream at the beginning of the conference and collects the data.

[0742] Step 2:

[0743] Converting audio data to text

[0744] The server converts the collected voice data into text using voice recognition software (e.g., Google Speech-to-Text).

[0745] Input: Audio data

[0746] Output: Text data

[0747] Specific operation: The server converts the voice data into text in real time and stores the text data in a database.

[0748] Step 3:

[0749] Analysis of speech content and emotional state

[0750] The server analyzes the textual content of the comments and evaluates the emotional state of each speaker using an emotion engine.

[0751] Input: Text data, audio data

[0752] Output: Emotional state assessment result

[0753] Specific operation: The server inputs text data into the emotion engine and analyzes the emotional state (e.g., joy, sadness, anger). The evaluation results are stored in a database.

[0754] Step 4:

[0755] Generate comment suggestions to encourage participation

[0756] The server uses a generative AI model (e.g., GPT-4) to generate suggested comments that encourage participants to speak according to the progress of the meeting.

[0757] Input: Analysis results, meeting progress

[0758] Output: Comment proposal

[0759] Specific operation: Based on the analysis results and the progress of the meeting, the server inputs prompt sentences into the generative AI model to generate comment proposals, which are then sent to the device and displayed on the screen.

[0760] Step 5:

[0761] Generate summary and closing content

[0762] Before the end of the meeting, the server summarizes what has been said so far and generates a proposal for closing remarks.

[0763] Input: Text data

[0764] Output: Summary, closing contents

[0765] Specific operation: The server inputs the text data into the summarization algorithm to generate a summary, inputs the proposed closing sentence into the generative AI model as a prompt sentence, and sends the generated sentence to the device.

[0766] Step 6:

[0767] Automatic generation of meeting minutes

[0768] The server automatically generates minutes based on the text of the remarks at the end of the meeting.

[0769] Input: Text data, sentiment analysis results

[0770] Output: Meeting minutes

[0771] Specific operation: The server generates minutes based on the text data and the results of sentiment analysis. The generated minutes are stored in a database and made accessible to users.

[0772] (Application example 3)

[0773] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart device 14 will be referred to as a "terminal."

[0774] Conventional meeting support systems lacked sufficient support for smooth meeting progress, particularly in terms of support that took into account the emotional state of the speaker, and lacked the ability to summarize the meeting content or generate closing comments. Furthermore, the functionality for automatically creating minutes at the end of a meeting was limited, creating challenges for improving meeting efficiency and productivity.

[0775] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for summarizing the contents of the meeting and creating minutes at the end of the meeting, a means for analyzing the emotional state of the speaker, a means for generating a summary of the meeting content, and a means for generating closing comments. This makes it possible to smoothly progress the meeting, support the progress taking into account the emotional state of the speaker, and generate a summary of the meeting content and closing comments. In addition, automatically creating minutes at the end of the meeting makes it possible to improve the efficiency and productivity of the meeting.

[0776] The "means for creating a draft agenda before a meeting" is a means for automatically creating an agenda and a schedule before a meeting begins.

[0777] "Means for recognizing the speaker's voice and converting it into text" refers to means for converting statements made during a meeting into text data using voice recognition technology.

[0778] The "means for supporting the progress of an ongoing meeting" are means for providing comments and summaries that encourage people to speak up in order to facilitate the smooth progress of the meeting.

[0779] The "means for summarizing the contents of statements made at a meeting and creating minutes at the end of the meeting" is a means for automatically summarizing the contents of statements made at the end of a meeting and creating minutes.

[0780] The "means for analyzing the emotional state of a speaker" is a means for analyzing the emotions from the speaker's voice or text and recording that state.

[0781] The "means for generating a summary of the contents of a meeting" is a means for summarizing the contents of what was said during the meeting and providing it in a concise form.

[0782] The "means for generating closing comments" is a means for automatically generating appropriate closing comments at the end of a meeting.

[0783] A system for carrying out this invention has the following configuration: A server includes means for creating a draft agenda before a meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for summarizing the contents of the meeting and creating minutes at the end of the meeting, means for analyzing the emotional state of the speaker, means for generating a summary of the meeting contents, and means for generating closing comments.

[0784] Hardware and Software Configuration

[0785] Hardware: microphone, speaker, robot body

[0786] Software: Python, speech recognition library, OpenAI API, TextBlob library

[0787] Processing flow

[0788] 1. Speech recognition: The server collects the audio during the meeting through a microphone and converts the audio into text using a speech recognition library.

[0789] 2. Summary generation: The server uses the OpenAI API to generate a summary of the meeting content. Specifically, based on the text data collected during the meeting, the server inputs prompt sentences into the generative AI model to obtain the summary.

[0790] 3. Sentiment Analysis: The server uses the TextBlob library to analyze the emotional state of the speaker from the text data, thereby understanding how the speaker's emotions affect the progress of the conference.

[0791] 4. Closing comment generation: The server again uses the OpenAI API to generate a closing comment based on the meeting content. It inputs a prompt and retrieves an appropriate closing comment.

[0792] 5. Creation of minutes: At the end of the meeting, the server automatically generates minutes based on the collected text data.

[0793] Specific examples

[0794] For example, imagine a meeting is being held to discuss improvements to a production line in a factory. During the meeting, when an engineer makes a new proposal, the emotion engine analyzes how other members feel about the proposal. If there are many negative emotions toward the proposal, the server uses that information to suggest a comment to the facilitator, such as, "Could you please explain this proposal in more detail?"

[0795] Prompt Sentence Examples

[0796] Please summarize the following meeting:

[0797] [Text of the meeting]

[0798] Please generate closing comments based on the following meeting content:

[0799] [Text of the meeting]

[0800] In this way, the system can smoothly advance meetings, provide support that takes into account the emotional state of the speakers, and generate a summary of the meeting content and closing comments. Furthermore, by automatically creating minutes at the end of a meeting, it is possible to improve meeting efficiency and productivity.

[0801] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[0802] Step 1:

[0803] The server collects audio during the conference via a microphone. The collected audio data is converted into text data using a speech recognition library. The input is audio data, and the output is text data. Through this process, the content of speech during the conference is recorded in text format.

[0804] Step 2:

[0805] The server generates a summary of the meeting content based on the text data. Specifically, it uses the OpenAI API to input prompts into the generative AI model and obtains the summary. The input is the text data collected during the meeting, and the output is the summary text. This process concisely summarizes the meeting content.

[0806] Step 3:

[0807] The server analyzes the emotional state of the speaker from the text data using the TextBlob library. The input is text data, and the output is the emotion analysis results. This processing allows us to understand how the speaker's emotions are affecting the progress of the conference.

[0808] Step 4:

[0809] The server suggests appropriate comments to the facilitator and participants based on the results of the sentiment analysis. For example, if there are a lot of negative emotions, it suggests a comment to the facilitator such as, "Could you please explain this proposal in more detail?" The input is the sentiment analysis results, and the output is the suggested comment. This process helps meetings proceed smoothly.

[0810] Step 5:

[0811] The server generates closing comments based on the meeting content. It again uses the OpenAI API to input prompts into the generative AI model and obtains the closing comments. The input is the text data collected during the meeting, and the output is the closing comments. This process ensures that the meeting is properly concluded.

[0812] Step 6:

[0813] The server automatically generates minutes based on the text data collected at the end of the meeting. The input is the text data collected during the meeting, and the output is the minutes text. This process records the details of the meeting and allows for later reference.

[0814] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0815] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (registered trademark) (Internet search engine).<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0816] Another example of generative AI is Gemini (registered trademark) (Internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[0817] In the above embodiment, an example in which the specific process is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific process may be performed by the smart device 14.

[0818] [Second embodiment]

[0819] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0820] 3, the data processing system 210 includes the data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0821] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0822] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0823] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[0824] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0825] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0826] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0827] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0828] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0829] In the smart glasses 214, the reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0830] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[0831] "Example 1"

[0832] This embodiment of the present invention provides an interface for inputting the purpose and agenda of a meeting as a means for creating a draft agenda before the meeting. Specifically, a user inputs the purpose and agenda of the meeting as text, and a draft agenda is automatically generated based on that.

[0833] "Example 2"

[0834] In addition, speech recognition technology is used to recognize the speaker's voice and convert it into text. Specifically, what is said during a meeting is picked up by a microphone and converted into text in real time using speech recognition software. This converted text is used to support the progress of the meeting and to create minutes.

[0835] "Example 3"

[0836] As a means of supporting the progress of ongoing meetings, the system provides a means for suggesting comments to encourage participants to speak next, a means for suggesting comments for the facilitator to encourage speaking, and a means for providing a summary of the content and closing content before the end of the meeting so that the meeting can be wrapped up within the allotted time. Specifically, the system analyzes the progress of the meeting and the content of comments, and displays on the screen suggested comments to encourage speaking at the appropriate time. In addition, before the end of the meeting, it summarizes what has been said so far and provides suggested closing content.

[0837] "Example 4"

[0838] At the end of a meeting, we provide a means to compile meeting minutes by summarizing what was said during the meeting. Specifically, minutes are automatically generated based on the text of what was said during the meeting. These minutes can be used for reference after the meeting or to prepare for the next meeting.

[0839] The processing flow of each embodiment will be described below.

[0840] "Example 1"

[0841] Step 1: The user enters the purpose and agenda of the meeting.

[0842] Step 2: The system analyzes the input information and automatically generates a draft agenda.

[0843] Step 3: The generated agenda is displayed to the user, who can make any necessary corrections or additions.

[0844] It will be held.

[0845] "Example 2"

[0846] Step 1: When the meeting starts, the microphone picks up what is being said during the meeting.

[0847] Step 2: The captured voice data is sent to speech recognition software and converted into text in real time.

[0848] Step 3: The transcribed remarks are used to support the progress of the meeting and to create minutes.

[0849] "Example 3"

[0850] Step 1: As the meeting progresses, the system analyzes the progress of the meeting and what is being said in real time.

[0851] Step 2: Based on the analysis results, the system displays suggested comments on the screen to encourage people to speak at the appropriate time.

[0852] Step 3: Before the meeting ends, the system summarizes what has been said so far and provides suggestions for closing remarks.

[0853] "Example 4"

[0854] Step 1: Once the meeting is over, the system automatically generates minutes based on the text of what was said during the meeting.

[0855] Step 2: The generated minutes are displayed to the user, who can make corrections or additions as needed.

[0856] Step 3: The final minutes are saved for post-meeting reference and to help prepare for the next meeting.

[0857] Example 1

[0858] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0859] In conventional conference systems, creating an agenda before a meeting was a time-consuming and difficult task, making it difficult to do efficiently. Supporting the progress of the meeting and creating minutes after the meeting also required manual work, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to automatically generate an appropriate agenda based on the purpose and topic of the meeting, which meant a lot of time was spent preparing for the meeting.

[0860] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[0861] In this invention, the server includes means for a user to input the purpose and agenda of a meeting, means for transmitting the input data to the server, means for the server to analyze the received data and create and input prompt sentences to the generative AI model, means for the generative AI model to generate a draft agenda, means for the server to return the generated draft agenda to the terminal, and means for the terminal to display the draft agenda to the user, thereby enabling users to easily automatically generate draft agendas for meetings.

[0862] The "means for creating a draft agenda before a meeting" is a means for automatically generating a draft agenda based on the purpose and agenda of the meeting.

[0863] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[0864] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to ensure the smooth progress of meetings.

[0865] "Methods for summarizing what was said at a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said at a meeting and compiling it into minutes after the meeting has ended.

[0866] The "means for users to input the purpose and agenda of the meeting" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[0867] The "means for transmitting input data to a server" refers to a communication means for transmitting data input by a user to a server.

[0868] "Means for analyzing data received by the server and creating and inputting prompt sentences to the generative AI model" refers to means for analyzing data received by the server and creating prompt sentences to be input to the generative AI model based on that data.

[0869] "Means by which a generative AI model generates a draft agenda" refers to means by which a generative AI model automatically generates a draft agenda based on a prompt sentence.

[0870] The "means for the server to return the generated agenda proposal to the terminal" refers to the means for the server to transmit the generated agenda proposal to the terminal.

[0871] The "means by which the terminal displays the agenda proposal to the user" refers to the means by which the terminal visually displays the received agenda proposal to the user.

[0872] This invention is a system that automatically generates a draft agenda before a meeting, supports the progress of the meeting during the meeting, and creates minutes after the meeting. This system includes: a means for a user to input the purpose and agenda of the meeting; a means for transmitting the input data to a server; a means for the server to analyze the received data and create and input prompt sentences to a generative AI model; a means for the generative AI model to generate a draft agenda; a means for the server to return the generated draft agenda to a terminal; and a means for the terminal to display the draft agenda to the user.

[0873] Users access a dedicated interface using a web browser and enter the purpose and agenda of the meeting in text format. This interface is implemented as an HTML form, making it easy for users to enter information. For example, a user can enter the purpose of the meeting, "Determine a market launch strategy for a new product," and the agenda, "Market analysis, competitive research, marketing strategy, and sales plan."

[0874] The device sends the data entered by the user to the server as an HTTP POST request. The data is sent in JSON format. The server analyzes the received data and creates a prompt to input into the generative AI model. For example, it generates a prompt such as, "The purpose of the meeting is to decide on a market launch strategy for a new product, and the agenda will include market analysis, competitive research, marketing strategy, and sales plan. Please create a draft agenda based on this."

[0875] The server inputs prompt sentences into the generative AI model to generate a draft agenda. The generated draft agenda is temporarily stored on the server, and the server sends it back to the device. The device analyzes the received draft agenda and displays it in a format that is easy for the user to view. For example, it is displayed on a web page in the following format:

[0876] Proposed agenda:

[0877] 1. Market analysis (10:00 - 10:30)

[0878] Review of current market conditions

[0879] Analysis of key market trends

[0880] 2. Competitive Research (10:30 - 11:00)

[0881] Strategies and trends of major competitors

[0882] Analysis of the strengths and weaknesses of competing products

[0883] 3. Marketing Strategy (11:00 - 11:30)

[0884] Identifying your target market

[0885] Marketing campaign planning

[0886] 4. Sales Planning (11:30 - 12:00)

[0887] Selection of sales channels

[0888] Setting sales targets

[0889] In this way, users can easily automatically generate meeting agendas. Support for meeting progress and the creation of meeting minutes after the meeting are also automated, allowing for efficient preparation, progress, and post-meeting processes.

[0890] The flow of the identification process in the first embodiment will be described with reference to FIG.

[0891] Step 1:

[0892] The user inputs the purpose and agenda of the meeting.

[0893] A user opens a web browser and accesses a dedicated interface. The interface has a text input field where the user inputs the purpose and agenda of the meeting. For example, the purpose is to "decide on a market launch strategy for a new product" and the agenda is to "market analysis, competitive research, marketing strategy, and sales plan." The input data is in text format.

[0894] Step 2:

[0895] The terminal sends the input data to the server.

[0896] When the user completes the input and clicks the submit button, the device sends the input data to the server as an HTTP POST request. The data sent is in JSON format, and looks something like this:

[0897] json

[0898] {

[0899] "Objective": "Determine a new product launch strategy",

[0900] "Agenda": "Market analysis, competitive research, marketing strategy, sales plan"

[0901] }

[0902] Step 3:

[0903] The server receives the data and creates and inputs prompts to the generative AI model.

[0904] The server analyzes the received data and creates a prompt to be input to the generative AI model. For example, it generates the following prompt:

[0905] "The purpose of the meeting is to decide on a market launch strategy for a new product, and the topics to be discussed will include market analysis, competitive research, marketing strategy, and sales plan. Please create a draft agenda based on this."

[0906] The input data is in JSON format, and the output data is a prompt in text format.

[0907] Step 4:

[0908] A generative AI model generates a draft agenda.

[0909] The server inputs a prompt to the generative AI model, which then generates a draft agenda based on the prompt. For example, the following draft agenda is generated:

[0910] 1. Market analysis (10:00 - 10:30)

[0911] Review of current market conditions

[0912] Analysis of key market trends

[0913] 2. Competitive Research (10:30 - 11:00)

[0914] Strategies and trends of major competitors

[0915] Analysis of the strengths and weaknesses of competing products

[0916] 3. Marketing Strategy (11:00 - 11:30)

[0917] Identifying your target market

[0918] Marketing campaign planning

[0919] 4. Sales Planning (11:30 - 12:00)

[0920] Selection of sales channels

[0921] Setting sales targets

[0922] The input data is a prompt, and the output data is a draft agenda in text format.

[0923] Step 5:

[0924] The server returns the generated agenda plan to the terminal.

[0925] The server returns the generated agenda in JSON format to the terminal. For example, the following JSON data is returned:

[0926] json

[0927] {

[0928] "agenda": [

[0929] {"Time": "10:00 - 10:30", "Topic": "Market Analysis", "Details": ["Review of the current market situation", "Analysis of key market trends"]},

[0930] {"Time": "10:30 - 11:00", "Topic": "Competitive Research", "Details": ["Strategies and Trends of Major Competitors", "Analysis of Strengths and Weaknesses of Competitive Products"]},

[0931] {"Time": "11:00 - 11:30", "Topic": "Marketing Strategy", "Details": ["Identifying Target Market", "Planning a Marketing Campaign"]},

[0932] {"Time": "11:30 - 12:00", "Item": "Sales plan", "Details": ["Sales channel selection", "Sales target setting"]}

[0933] ]

[0934] }

[0935] The input data is a draft agenda in text format, and the output data is a draft agenda in JSON format.

[0936] Step 6:

[0937] The terminal displays the proposed agenda to the user.

[0938] The device parses the received agenda proposal and displays it in a user-friendly format, for example on a web page in the following format:

[0939] Proposed agenda:

[0940] 1. Market analysis (10:00 - 10:30)

[0941] Review of current market conditions

[0942] Analysis of key market trends

[0943] 2. Competitive Research (10:30 - 11:00)

[0944] Strategies and trends of major competitors

[0945] Analysis of the strengths and weaknesses of competing products

[0946] 3. Marketing Strategy (11:00 - 11:30)

[0947] Identifying your target market

[0948] Marketing campaign planning

[0949] 4. Sales Planning (11:30 - 12:00)

[0950] Selection of sales channels

[0951] Setting sales targets

[0952] The input data is a draft agenda in JSON format, and the output data is a visual representation of the draft agenda.

[0953] (Application example 1)

[0954] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[0955] In conventional conference systems, creating the agenda before the meeting, providing support during the meeting, and creating minutes after the meeting were often done manually, which was time-consuming and labor-intensive. It was also difficult to create an agenda efficiently based on the purpose and topic of the meeting, which sometimes prevented the meeting from proceeding smoothly.

[0956] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[0957] In this invention, the server includes a means for creating a draft agenda before the meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for compiling the contents of the meeting remarks and creating minutes at the end of the meeting, an interface means for the user to input the purpose and agenda of the meeting as text, a means for automatically generating a draft agenda using a generative AI model, and a means for displaying and sharing the generated draft agenda with the user. This makes it possible to efficiently carry out everything from preparation to progress of the meeting and follow-up after it ends.

[0958] The "means for creating a draft agenda before a meeting" is a means for automatically creating a draft agenda for planning the progress of a meeting based on the purpose and agenda of the meeting.

[0959] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[0960] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[0961] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said during the meeting and compiling it into minutes after the meeting has ended.

[0962] The "interface means for users to input the purpose and agenda of the meeting as text" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[0963] "Means for automatically generating a draft agenda using a generative AI model" refers to means for automatically generating a draft agenda based on the input meeting purpose and agenda items using a generative AI model.

[0964] The "means for displaying and sharing the generated agenda draft with the user" refers to a means for displaying the generated agenda draft to the user and sharing it with other participants as necessary.

[0965] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what was said in the meeting and creating minutes at the end of the meeting, an interface means for users to input the purpose and agenda of the meeting as text, a means for automatically generating a draft agenda using a generative AI model, and a means for displaying and sharing the generated draft agenda with users.

[0966] System program processing explanation

[0967] Hardware used

[0968] Smartphone

[0969] server

[0970] Software used

[0971] Python

[0972] OpenAI API

[0973] Data processing and calculation

[0974] 1. User Input:

[0975] The user opens the smartphone application and inputs the purpose and agenda of the meeting in text format, thereby functioning as an interface means for the user to input the purpose and agenda of the meeting in text format.

[0976] 2. Use of generative AI models:

[0977] Based on the input information, a prompt is sent to a generative AI model (OpenAI's GPT-3). This is where the means to automatically generate a draft agenda using the generative AI model comes into play.

[0978] 3. Generate a draft agenda:

[0979] GPT-3 generates a draft agenda based on the prompt and returns it to the application, which then provides a way to display and share the draft agenda with the user.

[0980] 4. Speech Recognition:

[0981] During a meeting, the speaker's voice is recognized in real time and converted into text data. This is where the means for recognizing the speaker's voice and converting it into text comes into play.

[0982] 5. Progression support:

[0983] To facilitate smooth progress in the meeting, appropriate comments and instructions are provided to participants and facilitators. This is where ongoing meeting support measures come into play.

[0984] 6. Minutes Preparation:

[0985] After the meeting, the contents of what was said during the meeting are organized and compiled as minutes. This is where the means of compiling the contents of what was said during the meeting and creating minutes at the end of the meeting comes into play.

[0986] Specific examples

[0987] A store manager holds a sales strategy meeting for a new product, and uses a smartphone app to input the purpose and agenda of the meeting. For example, he might enter a prompt like this:

[0988] Meeting purpose: New product sales strategy

[0989] agenda:

[0990] market analysis

[0991] Competitive Research

[0992] Selection of sales channels

[0993] Promotion Plan

[0994] Create a draft agenda based on this information.

[0995] The application automatically generates an agenda that can be shared with all staff members, improving meeting efficiency. In this way, meetings can be efficiently conducted from preparation to progress, and follow-up after the meeting has ended.

[0996] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[0997] Step 1:

[0998] A user opens a smartphone application and inputs the purpose and agenda of the meeting in text format. The input text data is generated as a prompt sentence including the purpose and agenda of the meeting. The input data is the "purpose of the meeting" and "agenda," and the output data is the prompt sentence.

[0999] Step 2:

[1000] The device sends the generated prompt to the server. The server receives the prompt and sends it to the generative AI model (OpenAI's GPT-3). The input data is the prompt, and the output data is a request to the generative AI model.

[1001] Step 3:

[1002] The server receives the response from the generative AI model. The generative AI model generates a draft agenda based on the prompt and returns the result to the server. The input data is the prompt, and the output data is the generated draft agenda.

[1003] Step 4:

[1004] The server sends the generated agenda plan to the terminal, which receives the agenda plan and displays it to the user. The input data is the generated agenda plan, and the output data is the agenda plan displayed to the user.

[1005] Step 5:

[1006] During a meeting, the device recognizes the speaker's voice in real time and converts it into text data. Speech recognition technology is used to convert the speaker's voice into text. The input data is the speaker's voice, and the output data is the text of what was said.

[1007] Step 6:

[1008] The server receives the textual content of speech and generates appropriate comments and instructions to support the progress of the meeting. This provides appropriate comments and instructions to participants and the facilitator. The input data is the textual content of speech, and the output data is the generated comments and instructions.

[1009] Step 7:

[1010] After the meeting ends, the server organizes the content of the meeting and compiles it into minutes. The generated minutes are sent to the terminal and displayed to the user. The input data is the text of the comments, and the output data is the generated minutes.

[1011] Example 2

[1012] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1013] With conventional conferencing systems, it was difficult to convert statements made during a meeting into text in real time and use it to support the progress of the meeting or to create minutes. Furthermore, there was a lack of efficient means to carry out a series of processes, such as collecting, transmitting, converting, displaying, and saving audio data. This resulted in reduced meeting efficiency and the time it took to create minutes.

[1014] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1015] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of what was said at the meeting and creating minutes at the end of the meeting, means for collecting voice data, means for transmitting the voice data, means for converting the voice data into text data, means for displaying the text data in real time, and means for saving the text data. This makes it possible to convert what is said during the meeting into text in real time, and to efficiently support the progress of the meeting and create minutes.

[1016] A "method for drafting an agenda before a meeting" is a method for planning the agenda and progress of a meeting and sharing it with participants before the meeting begins.

[1017] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text in real time using voice recognition technology.

[1018] "Means for supporting the progress of a meeting in progress" are means for encouraging people to speak and supporting the progress of the agenda in order to ensure the smooth progress of the meeting.

[1019] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a method for organizing what was said during the meeting and saving it as an official record when the meeting ends.

[1020] The "means for collecting voice data" refers to a means for collecting speeches made during a meeting as voice data using a device such as a microphone.

[1021] The "means for transmitting voice data" refers to a means for transmitting collected voice data to a server or other device.

[1022] The "means for converting voice data into text data" refers to a means for converting voice data into text data using voice recognition technology.

[1023] The "means for displaying text data in real time" refers to a means for displaying the converted text data in real time during a conference so that participants can check it.

[1024] The "means for saving text data" refers to a means for saving text data generated during a conference so that it can be referenced later.

[1025] MODE FOR CARRYING OUT THE INVENTION

[1026] The present invention relates to a system that converts statements made during a meeting into text in real time and uses the text to support the progress of the meeting and to create minutes. A specific embodiment of this system will be described below.

[1027] Hardware and software used

[1028] Hardware: microphone, device (computer, tablet, smartphone, etc.)

[1029] Software: speech recognition software (e.g., Google Speech-to-Text API), conferencing applications

[1030] System Overview

[1031] The system includes the following main means:

[1032] 1. A means for creating agendas before a meeting: A means for users to plan the agenda and progress of a meeting and share it with participants.

[1033] 2. A means of recognizing the speaker's voice and converting it into text: A means of converting statements made during a meeting into text in real time using voice recognition technology.

[1034] 3. Support measures for ongoing meetings: These are measures to encourage speaking and help move the agenda forward in order to ensure smooth progress in the meeting.

[1035] 4. A means of summarizing what was said during the meeting and creating minutes at the end of the meeting: This is a means of organizing what was said during the meeting and saving it as an official record when the meeting ends.

[1036] 5. Means of collecting audio data: A means of collecting speech during a meeting as audio data using a device such as a microphone.

[1037] 6. Means for transmitting audio data: A means for transmitting collected audio data to a server or other device.

[1038] 7. Means for converting voice data into text data: A means for converting voice data into text data using voice recognition technology.

[1039] 8. A means for displaying text data in real time: A means for displaying converted text data in real time during a meeting so that participants can check it.

[1040] 9. A means of saving text data: A means of saving text data generated during a meeting for later reference.

[1041] Specific examples

[1042] For example, if someone says "Let's move on to the next topic" during a meeting, the following processing will occur:

[1043] The audio is picked up by a microphone connected to the device.

[1044] The terminal transmits the voice data to the server.

[1045] The server converts the voice data into text data using speech recognition software (Google Speech-to-Text API).

[1046] The server displays text data such as "Let's move on to the next topic" in the conference application in real time.

[1047] The user checks the displayed text and supports the progress of the meeting.

[1048] After the meeting ends, the server saves the text data as minutes, allowing users to access them later.

[1049] Prompt Sentence Examples

[1050] Examples of prompts to be input to a generative AI model include:

[1051] "Please explain the process flow of a system that converts statements made during a meeting into text in real time and is used to support the progress of the meeting and create meeting minutes. Specifically, please clearly state what hardware and software are used and what data processing and calculations are performed for each step from the start of the meeting to collecting audio, transmitting the audio data, converting the audio into text, and using the text data."

[1052] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[1053] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1054] Step 1:

[1055] A user starts a conference.

[1056] Input: A user launches a conferencing application and clicks the "Start Conference" button.

[1057] Data processing / calculation: The terminal notifies the server of the start of the conference and generates a conference ID.

[1058] Output: A conference ID is generated and the conference is started.

[1059] Specific operation: When a user launches a conferencing application and clicks the "Start conference" button, the terminal sends a request to start the conference to the server, and the server generates a conference ID.

[1060] Step 2:

[1061] The device collects the audio.

[1062] Input: Speeches made during the meeting are input into the microphone.

[1063] Data processing / computation: The microphone converts the voice into digital data, which the device stores in a buffer.

[1064] Output: Digital audio data is stored in a buffer.

[1065] How it works: A microphone connected to the device picks up what is being said during the meeting, converts the audio into digital data, and stores it in a buffer.

[1066] Step 3:

[1067] The terminal transmits the audio data.

[1068] Input: Buffered digital audio data.

[1069] Data processing / calculation: The terminal divides the voice data into packets and sends them to the server over the network.

[1070] Output: The audio data is sent to the server.

[1071] Specific operation: The device divides the audio data stored in the buffer into packets and sends them to the server over the network.

[1072] Step 4:

[1073] The server converts the audio into text.

[1074] Input: The audio data received by the server.

[1075] Data processing / calculation: The server converts the voice data into text data using speech recognition software (e.g., Google Speech-to-Text API).

[1076] Output: Text data is generated.

[1077] Specific operation: The server passes the received voice data to the voice recognition software, which converts the voice data into text data.

[1078] Step 5:

[1079] The server displays the text data in real time.

[1080] Input: Generated text data.

[1081] Data processing / calculation: The server sends the text data to the conferencing application and displays it in real time.

[1082] Output: The text data is displayed in the conferencing application.

[1083] Specific behavior: The server sends the generated text data to the conferencing application and displays it so that the user can view it in real time.

[1084] Step 6:

[1085] The server stores the text data.

[1086] Input: Generated text data.

[1087] Data processing / calculation: The server stores the text data in a database.

[1088] Output: Text data is saved.

[1089] Specific operation: The server stores the generated text data in a database so that users can access it after the meeting ends.

[1090] (Application example 2)

[1091] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1092] Conventional conferencing systems can convert statements made during a meeting into text in real time to support the progress of the meeting, but they cannot receive voice instructions at work sites such as factories and automatically carry out work based on those instructions. This reduces work efficiency at the site and increases the possibility of human error.

[1093] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for summarizing the contents of the meeting remarks and creating minutes at the end of the meeting, and means for receiving work instructions by voice, converting them into text in real time, and automating work based on the text. This makes it possible not only to support the progress of meetings and create minutes, but also to receive work instructions by voice at work sites such as factories and automatically perform work based on those instructions.

[1094] A "method for creating a draft agenda before a meeting" is a method for creating an agenda and a schedule before a meeting begins.

[1095] "Means for recognizing a speaker's voice and converting it into text" refers to means for converting a speaker's speech into text data in real time using voice recognition technology.

[1096] The "means for supporting the progress of the ongoing conference" are means for supporting the smooth progress of the conference.

[1097] "Means for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a means for summarizing what was said during the meeting and compiling it into minutes when the meeting ends.

[1098] "Means for receiving work instructions via voice, converting them into text in real time, and automating work based on that text" refers to means for converting work instructions given via voice into text in real time, and automatically performing work based on that text.

[1099] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what has been said at the meeting and creating minutes at the end of the meeting, and a means for receiving work instructions by voice, converting them into text in real time, and automating work based on that text.

[1100] The server uses speech recognition technology to convert the speaker's voice into text in real time. Specifically, it collects voice data using a microphone and converts it into text data using a speech recognition library. This text data is used to support the progress of the meeting and to create minutes.

[1101] The server also receives work instructions via voice at factories and other work sites, converts them into text in real time, and automatically performs tasks based on the text. For example, tasks such as assembling, inspecting, and transporting parts can be performed by voice command. This improves work efficiency at the site and reduces the occurrence of human error.

[1102] For example, if a voice command such as "Please assemble the parts" is given in a factory, the server will recognize the command and begin assembling the parts. At this time, voice recognition technology will be used to convert the command into text, and the robot will automatically carry out the work based on that text.

[1103] Examples of prompts to input to a generative AI model include:

[1104] Create a program that recognizes the voice command "Please assemble the parts" and causes the robot to begin assembling the parts.

[1105] In this way, by combining voice recognition technology and automation technology, the server can not only support the progress of meetings and create minutes, but also efficiently process and automate work instructions on-site.

[1106] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1107] Step 1:

[1108] The user inputs voice instructions through a microphone, and the input voice data is sent to the server.

[1109] Step 2:

[1110] The server converts the received voice data into text data using a voice recognition library. Specifically, it analyzes the voice data and generates corresponding text using voice recognition technology. The input is voice data and the output is text data.

[1111] Step 3:

[1112] The server analyzes the generated text data and understands the instructions. For example, it checks whether the instruction "Please assemble the parts" is included. The input is the text data, and the output is the analysis result.

[1113] Step 4:

[1114] The server selects an appropriate task based on the analysis results. For example, if the keyword "assembly" is included, it selects the task of assembling parts. The input is the analysis results, and the output is the selected task.

[1115] Step 5:

[1116] The server generates commands to execute the selected tasks and sends them to the robot. Specifically, it generates commands containing specific operation instructions for assembly tasks. The input is the selected task, and the output is the command to the robot.

[1117] Step 6:

[1118] The robot performs a task based on the command it receives, for example, starting an assembly task. The input is the command to the robot, and the output is the task performed.

[1119] Step 7:

[1120] The server monitors the progress of the work and issues further instructions as needed, for example, when the work is completed it issues the next instruction. The input is the progress of the work and the output is further instructions.

[1121] In this way, a series of steps is realized: the user issues instructions by voice, the server converts the instructions into text, analyzes them, selects an appropriate task, and the robot carries out that task.

[1122] Example 3

[1123] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1124] Conventional conferencing systems lacked sufficient support for smoothly advancing meetings, and in particular, the timing of encouraging participants to speak, summarizing the content, and providing closing remarks were often done manually. Furthermore, creating meeting minutes after the end of a meeting was time-consuming and inefficient. This led to a decrease in meeting productivity and the risk of important points being overlooked.

[1125] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1126] In this invention, the server includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for compiling the contents of the meeting comments and creating minutes at the end of the meeting, a means for collecting the progress of the meeting and the contents of the comments, a means for analyzing the text of the comments, a means for generating draft comments to encourage participants to speak, a means for generating draft comments for the facilitator, and a means for generating a summary of the contents and closing content before the end of the meeting.This enables smooth support for the progress of the meeting and encourages participants to speak at the appropriate time.In addition, by providing a summary of the contents and closing content before the end of the meeting and automatically creating minutes at the end of the meeting, the productivity of the meeting can be improved.

[1127] The "means for creating a draft agenda before a meeting" is a means for automatically creating an agenda and a schedule before a meeting begins.

[1128] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text data in real time using voice recognition technology.

[1129] "Means for supporting the progress of ongoing meetings" are means for providing comments and advice to participants and facilitators at appropriate times to ensure that meetings proceed smoothly.

[1130] "Means for summarizing the contents of statements made during a meeting and creating minutes at the end of the meeting" refers to a means for automatically summarizing the contents of statements made during a meeting and compiling them into minutes when the meeting ends.

[1131] The "means for collecting the progress of the meeting and the content of comments" is a means for collecting the progress of the meeting and the content of comments in real time.

[1132] "Means for text analysis of speech content" refers to means for analyzing collected text data of speech content and performing keyword extraction and sentiment analysis.

[1133] The "means for generating a comment proposal to encourage a participant to make a speech" is a means for generating a comment proposal to encourage a participant to make the next speech according to the progress of the conference.

[1134] The "means for generating a proposal for a facilitator's remarks" is a means for generating an appropriate proposal for a facilitator's remarks so that the facilitator can smoothly proceed with the meeting.

[1135] "Means for generating a summary or closing content before the end of the meeting" refers to a means for summarizing the content discussed so far and providing the information necessary for closing the meeting before the end of the meeting.

[1136] The present invention provides a system for smoothly supporting the progress of a meeting and automating the creation of minutes after the meeting. A specific embodiment of this system will be described below.

[1137] Hardware and software used

[1138] Hardware: Servers, devices (PCs, tablets, etc.)

[1139] Software: generative AI models (e.g., GPT-4), meeting management systems, speech recognition software (e.g., Google Speech-to-Text API), text analysis tools (e.g., spaCy)

[1140] System configuration

[1141] 1. How to prepare an agenda before a meeting

[1142] The server automatically generates a draft agenda based on the purpose of the meeting and the list of participants, allowing the meeting to proceed smoothly.

[1143] 2. A means of recognizing the speaker's voice and converting it into text

[1144] The server uses speech recognition software to convert speech into text in real time, for example, using the Google Speech-to-Text API.

[1145] 3. Support for ongoing meetings

[1146] The server collects information about the progress of the meeting and the content of comments made through the meeting management system, and analyzes it using a text analysis tool. Based on the analysis results, it generates suggested comments to encourage participants to make their next comments, as well as suggested comments for the facilitator.

[1147] The device displays the generated comment proposal on the screen. For example, a comment such as "Please give your opinion on the following agenda item" may be displayed.

[1148] 4. A way to generate a summary or closing statement before the meeting ends

[1149] The server summarizes what has been said so far based on the progress of the meeting and the content of what has been said, and generates a proposal for a closing speech, which allows the meeting content to be summarized effectively.

[1150] The device displays the generated summary and closing statement on the screen. For example, a summary such as "To summarize the discussion so far, the following points are important."

[1151] 5. A way to summarize what was said at the meeting and create minutes at the end of the meeting

[1152] The server automatically generates minutes based on the text of what was said during the meeting, organizing the content in chronological order and highlighting the important points.

[1153] The device saves the generated minutes so that they can be referenced after the meeting. For example, a message such as "Meeting minutes have been automatically generated. Please use them to prepare for your next meeting" is displayed.

[1154] Specific examples

[1155] For example, if you want to generate a comment suggestion during a meeting such as "Please give your opinion on the next agenda item," you can input the following prompt sentence into the generative AI model:

[1156] Example prompt sentence:

[1157] Now that we have all expressed our opinions on the current topic, please give your opinion on the next topic.

[1158] Based on this prompt, the generative AI model generates appropriate comment proposals and displays them to participants and the facilitator. Before the meeting ends, the model provides a summary such as, "To summarize the discussion so far, the following points are important," and automatically generates minutes to save at the end of the meeting.

[1159] In this way, the present invention can smoothly support the progress of the meeting and improve the productivity of the meeting by automating the creation of minutes after the meeting. The flow of the specific processing in the third embodiment will be described with reference to FIG.

[1160] Step 1:

[1161] Meeting progress and collection of comments

[1162] The server collects the progress of the meeting and the content of comments in real time through the meeting management system. Specifically, it converts the voice data spoken during the meeting into text using voice recognition software (e.g., Google Speech-to-Text API).

[1163] Input: Audio data during the meeting

[1164] Output: Text of what was said

[1165] Specific operation: The server calls the voice recognition software and converts the voice data into text data, which is then stored in the conference management system.

[1166] Step 2:

[1167] Text analysis of speech content

[1168] The server analyzes the collected text data using a text analysis tool (e.g., spaCy). Specifically, it extracts keywords from the comments and analyzes their sentiment.

[1169] Input: Translated speech

[1170] Output: Analysis results (keywords, sentiment analysis results, etc.)

[1171] Specific operation: The server uses text analysis tools to analyze the text data and extract important keywords and sentiment trends. The analysis results are used in the next step.

[1172] Step 3:

[1173] Generate comment suggestions to encourage participation

[1174] The server uses a generative AI model (e.g., GPT-4) to generate a comment proposal that prompts the participant to make the next statement. Specifically, the analysis results are input into the generative AI model as a prompt sentence.

[1175] Input: Analysis result, prompt statement

[1176] Output: Comment proposal

[1177] Specific operation: The server inputs the prompt sentence into the generative AI model to generate appropriate comment suggestions, which are then displayed on the device in the next step.

[1178] Step 4:

[1179] Generate comments for the facilitator

[1180] The server uses a generative AI model to generate suggested comments for the facilitator, specifically, advice and questions that will help the facilitator smoothly conduct the meeting.

[1181] Input: Analysis result, prompt statement

[1182] Output: Draft comments for facilitator

[1183] Specific operation: The server inputs the prompt sentence into the generative AI model and generates a comment proposal for the facilitator. The generated comment proposal is displayed on the facilitator's device.

[1184] Step 5:

[1185] Generate a summary and closing statement before the meeting ends

[1186] The server summarizes what has been said so far based on the progress of the meeting and the content of the remarks, and generates a proposal for closing remarks.

[1187] Input: Analysis results, history of comments

[1188] Output: Summary, closing contents

[1189] Specific operation: The server extracts important points based on the speech history and generates a summary and closing remarks, which are then displayed on the terminal before the end of the meeting.

[1190] Step 6:

[1191] Automatic generation of meeting minutes

[1192] The server automatically generates minutes based on the text of what was said during the meeting, organizing the content in chronological order and emphasizing the important points.

[1193] Input: Translated speech

[1194] Output: Auto-generated meeting transcript

[1195] Specific operation: The server organizes the transcribed remarks and generates minutes that highlight the important points. The generated minutes are saved on the device after the meeting and can be referenced.

[1196] (Application example 3)

[1197] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1198] In order for a meeting to proceed smoothly, it is important for participants and facilitators to speak at the appropriate time and to summarize the contents of the meeting efficiently. However, it is not easy to come up with ideas for comments to encourage the next person to speak during the meeting, or to summarize the contents and prepare closing remarks before the end of the meeting. In addition, creating minutes after the meeting takes time and effort. Effective methods to solve these problems are needed.

[1199] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 3 is realized by the following means.

[1200] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for summarizing what was said in the meeting and creating minutes at the end of the meeting, means for analyzing the progress of the meeting and the content of the comments using a generative AI model and generating draft comments to encourage the next comment, means for generating a summary of the content and closing content before the end of the meeting, and means for automatically generating minutes based on the content of the comments made in the meeting.This allows the meeting to proceed smoothly, enables participants and the facilitator to speak at the appropriate time, and enables the content of the meeting to be summarized efficiently.

[1201] The "means for creating a draft agenda before a meeting" is a means for automatically creating an agenda and a schedule before a meeting begins.

[1202] "Means for recognizing the speaker's voice and converting it into text" refers to means for recognizing the voice of a speaker during a conference in real time and converting it into text information.

[1203] "Means for supporting the progress of ongoing meetings" are means for providing participants and facilitators with suggested comments that encourage them to speak at appropriate times in order to ensure smooth progress in the meeting.

[1204] The "means for summarizing the contents of statements made during a meeting and creating minutes at the end of the meeting" is a means for automatically summarizing the contents of statements made during the meeting and creating minutes at the end of the meeting.

[1205] A "generative AI model" is a model that uses artificial intelligence technology to generate appropriate output from input data.

[1206] "A means for analyzing the progress of a meeting and the content of comments, and generating suggested comments to encourage the next comment" is a means for analyzing the progress of a meeting and the content of comments, and proposing what should be said next.

[1207] "Means for generating a summary of the content and closing content before the end of the meeting" refers to means for summarizing what has been said so far and generating content necessary to conclude the meeting before the end of the meeting.

[1208] "Means for automatically generating minutes based on the content of statements made during a meeting" refers to a means for automatically creating minutes based on the content of statements made during a meeting.

[1209] The system for carrying out the present invention includes a plurality of means for supporting the progress of a meeting and efficiently creating minutes. Specific embodiments are described below.

[1210] First, the server provides a means for automatically creating a draft agenda before a meeting is held. This means generates the meeting agenda and schedule based on past meeting data and pre-entered information.

[1211] Next, we provide a means to recognize the speaker's voice during the meeting and convert it into text in real time. This means using speech recognition technology to convert the speaker's speech into text information. Specifically, we can use speech recognition services such as Google Cloud Speech-to-Text API and IBM Watson Speech to Text.

[1212] Furthermore, to support the progress of ongoing meetings, the company provides a means to use a generative AI model to analyze the progress of the meeting and the content of comments, and generate proposed comments to encourage the next comment. This means uses a generative AI model such as OpenAI's GPT-3 to generate proposed comments to facilitate the progress of the meeting.

[1213] Before the end of the meeting, we provide a means to summarize what has been said so far and generate a closing statement, which also uses a generative AI model to efficiently summarize the meeting content.

[1214] At the end of the meeting, we provide a means to automatically generate minutes based on what was said during the meeting. This means uses a generative AI model to organize what was said and compile it into minutes.

[1215] As a concrete example, imagine a meeting to discuss the introduction of a new production line in a factory. In this case, the server inputs the following prompt sentence into the generative AI model to generate a proposed comment that will prompt the next statement.

[1216] Example prompt sentence:

[1217] Analyze what was said in the meeting below and generate suggested comments to encourage the next one.

[1218] Participant A: Today we would like to discuss the introduction of a new production line.

[1219] Participant B: First, let's identify the problems with the current production line.

[1220] Participant C: The issues are inefficiency and inconsistent quality.

[1221] Suggested comments:

[1222] Using this prompt, the generative AI model can generate proposed comments to encourage the next person to speak, helping the meeting proceed smoothly. Furthermore, by summarizing what has been said so far and generating closing remarks before the end of the meeting, the content of the meeting can be efficiently summarized. At the end of the meeting, minutes can be automatically generated based on what has been said, significantly reducing post-meeting work.

[1223] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1224] Step 1:

[1225] The server creates a draft agenda before the meeting. It uses past meeting data and pre-entered information as input and generates a draft agenda based on this data. As output, it provides a draft agenda that includes the meeting topic and schedule. Specifically, it retrieves past meeting data from a database and generates a draft agenda using natural language processing technology.

[1226] Step 2:

[1227] The server recognizes the speaker's voice during the meeting and converts it into text in real time. It uses the speaker's voice data as input and converts it into text data using speech recognition technology. It provides the text of what was said as output. Specifically, it uses speech recognition services such as Google Cloud Speech-to-Text API and IBM Watson Speech to Text.

[1228] Step 3:

[1229] The server uses a generative AI model to analyze the progress of the meeting and the content of comments, and generates suggested comments to encourage the next remark. As input, it uses the textual content of the comments and inputs a prompt into the generative AI model. As output, it provides suggested comments to encourage the next remark. Specifically, it uses a generative AI model such as OpenAI's GPT-3 to generate suggested comments based on the prompt.

[1230] Step 4:

[1231] The server generates a summary of the meeting content and closing remarks before the end of the meeting. It uses the remarks made during the meeting as input and inputs prompts into the generative AI model. It provides a summary of the meeting and closing remarks as output. Specifically, it uses the generative AI model to summarize the remarks and generate closing remarks.

[1232] Step 5:

[1233] At the end of the meeting, the server automatically generates minutes based on what was said during the meeting. It uses what was said during the meeting as input and inputs prompts into the generative AI model. The output provides the automatically generated minutes. Specifically, the generative AI model is used to organize what was said and compile it into minutes.

[1234] Step 6:

[1235] The user checks the proposed agenda, textual comments, proposed comments encouraging the next speaker, the meeting summary, closing remarks, and minutes provided by the server, and makes corrections or additions as necessary. The various data provided by the server is used as input, and the final meeting materials are created as output. Specifically, the quality of the meeting is improved by the user checking the various data and making any necessary corrections or additions.

[1236] Furthermore, an emotion engine that estimates the user's emotion may be further combined. That is, the identification processing unit 290 may estimate the user's emotion using the emotion identification model 59, and perform identification processing using the user's emotion.

[1237] "Example 1"

[1238] One embodiment of the present invention is a conference support system incorporating an emotion engine. This system recognizes a user's voice and extracts emotions from that voice. Specifically, the system analyzes the tone, volume, and speaking rate of the user's voice and estimates the user's emotional state from the results. For example, if the user's voice is loud and fast, it is determined that the user is likely excited.

[1239] "Example 2"

[1240] The emotion engine also adjusts the operation of the meeting progress support means based on the recognized emotions. For example, if it determines that the emotional state of all meeting participants is excited, the progress support means will suggest comments to encourage calm. Conversely, if it determines that the meeting has become monotonous, it will suggest comments to stimulate discussion.

[1241] "Example 3"

[1242] Furthermore, the emotion engine is also used to create minutes at the end of a meeting. Specifically, it records the emotional state of each speaker in the minutes and analyzes how that emotional state influenced the discussion. This allows for a more detailed understanding of the progress of the meeting.

[1243] "Example 4"

[1244] As a concrete example, imagine a meeting to discuss the progress of a project. If the project leader's tone of voice sounds discouraged when reporting progress, the emotion engine will capture that information and provide feedback to the progress support tool. Based on that information, the progress support tool will suggest encouraging comments to other members. This will lighten the mood in the meeting and promote productive discussion.

[1245] The processing flow of each embodiment will be described below.

[1246] "Example 1"

[1247] Step 1: Recognize the user's voice.

[1248] Step 2: Analyze the audio for tone, volume, speaking rate, etc.

[1249] Step 3: Estimate the user's emotional state from the analysis results.

[1250] "Example 2"

[1251] Step 1: The emotion engine determines the emotional state of all meeting participants.

[1252] Step 2: The progress support tool suggests comments based on the emotional state.

[1253] "Example 3"

[1254] Step 1: At the end of the meeting, the emotion engine records the emotional state of each speaker.

[1255] Step 2: Analyze how your emotional state influenced the discussion.

[1256] "Example 4"

[1257] Step 1: Project leader reports on progress.

[1258] Step 2: If the emotion engine senses disappointment in the tone of voice, it picks up on that information.

[1259] Step 3: The progress support tool uses this information to suggest encouraging comments to other members.

[1260] Example 1

[1261] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1262] Conventional meeting support systems have difficulty efficiently reflecting the purpose and topics of a meeting in the agenda, and lack support for proceedings that take into account the emotional state of participants during the meeting. As a result, meetings do not proceed smoothly, and creating minutes is a time-consuming process.

[1263] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1264] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of the meeting remarks and creating minutes at the end of the meeting, means for the user to input the purpose and agenda of the meeting, means for analyzing the input information and generating a draft agenda, means for analyzing the voice data and estimating the emotional state, and means for updating the agenda based on the emotional state. This makes it possible to efficiently reflect the purpose and agenda of the meeting in the agenda and to support the progress of the meeting while taking into account the emotional state of the participants.

[1265] The "means for creating a draft agenda before a meeting" is a means for automatically generating items necessary for the progress of a meeting based on the purpose and agenda of the meeting.

[1266] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[1267] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[1268] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said during the meeting and compiling it into minutes after the meeting has ended.

[1269] The "means for users to input the purpose and agenda of the meeting" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[1270] The "means for analyzing input information and generating a draft agenda" is a means for analyzing the purpose and agenda of the meeting input by the user and automatically generating a draft agenda based on the information.

[1271] The "means for analyzing voice data and estimating emotional states" is a means for analyzing voice data of speakers during a conference and estimating the emotional states of the speakers from the tone, volume, speaking speed, and the like.

[1272] The "means for updating the agenda based on emotional state" is a means for dynamically updating the meeting agenda based on the estimated emotional state.

[1273] MODE FOR CARRYING OUT THE INVENTION

[1274] This invention is a system for supporting the efficient running of meetings, creating a draft agenda before the meeting, supporting the progress of the meeting during the meeting, and creating minutes at the end of the meeting. A specific embodiment of this system will be described below.

[1275] 1. System Overview

[1276] This system works in cooperation with three parties: a server, a terminal, and a user. The server provides an interface for inputting the purpose and agenda of the meeting, and analyzes the input information to generate a draft agenda. It also recognizes the user's voice during the meeting, analyzes the voice data to estimate the user's emotional state, and dynamically updates the agenda. The terminal is responsible for sending the user's input and voice data to the server.

[1277] 2. Hardware and Software Used

[1278] The server uses the following hardware and software:

[1279] Generative AI models using natural language processing techniques (e.g., OpenAI's GPT-4)

[1280] Voice recognition software (e.g., Google Cloud Speech-to-Text)

[1281] Emotion engine (e.g. IBM Watson Tone Analyzer)

[1282] The terminal provides an interface for capturing user input and voice data and transmitting it to a server. Specifically, a computer or smartphone equipped with a microphone and keyboard is used.

[1283] 3. Program Processing

[1284] The user uses the device's interface to input the purpose and agenda of the meeting in text format, for example, "The purpose of the next project meeting is to review progress and share issues." The device then sends this information to the server, which uses natural language processing technology to analyze the input text and generate appropriate agenda items.

[1285] During a meeting, a user uses the device's microphone to input voice, such as saying, "This part is very important." The device captures the voice data in real time and sends it to the server. The server then uses speech recognition software to convert the voice into text and uses an emotion engine to analyze the user's tone, volume, speaking rate, etc. to estimate the user's emotional state. Based on the estimated emotional state, the server dynamically updates the agenda.

[1286] 4. Examples of concrete examples and prompts

[1287] For example, if a user types "The purpose of the next project meeting is to review progress and share issues," and during the meeting emphasizes "This part is very important," the server will generate and update the following agenda:

[1288] 1. Opening Remarks

[1289] 2. Project progress reports

[1290] 3. Sharing current issues

[1291] 4. Check important points (parts highlighted by the user)

[1292] 5. Question and Answer Session

[1293] 6. Confirm the next meeting date

[1294] 7. Closing Remarks

[1295] Examples of prompts to input to a generative AI model might include:

[1296] "The user has entered the purpose of the next project meeting as reviewing progress and sharing issues. Please create a draft agenda based on this."

[1297] In this way, a system is realized in which the server, terminals, and users work together to support efficient conference management.

[1298] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1299] Step 1:

[1300] The user inputs the purpose and agenda of the meeting.

[1301] The user uses the device's interface to input the purpose and agenda of the meeting in text format. For example, the user might input, "The purpose of the next project meeting is to review progress and share issues." The input data is in text format, and the device then sends this data to the server.

[1302] Step 2:

[1303] The terminal transmits the input information to the server.

[1304] The terminal sends the text data entered by the user to the server using an HTTP request. The input is the text data entered by the user, and the output is the text data sent to the server.

[1305] Step 3:

[1306] The server analyzes the input information and generates a draft agenda.

[1307] The server analyzes the received text data using natural language processing technology. Specifically, it uses a generative AI model (e.g., OpenAI's GPT-4) to analyze the input text and generate appropriate agenda items. The input is the text data entered by the user, and the output is the generated agenda proposal. For example, it generates the following agenda proposal:

[1308] 1. Opening Remarks

[1309] 2. Project progress reports

[1310] 3. Sharing current issues

[1311] 4. Q&A

[1312] 5. Confirm the next meeting date

[1313] 6. Closing Remarks

[1314] Step 4:

[1315] A user inputs audio during a conference.

[1316] During a meeting, a user inputs voice using the device's microphone, for example, saying, "This part is very important." The input is the user's voice data, and the device captures this data in real time.

[1317] Step 5:

[1318] The terminal transmits the voice data to the server.

[1319] The device sends the captured audio data to the server using an HTTP request. The input is the user's audio data, and the output is the audio data sent to the server.

[1320] Step 6:

[1321] The server analyzes the voice data and estimates the emotional state.

[1322] The server converts the speech into text using speech recognition software (e.g., Google Cloud Speech-to-Text). It then uses an emotion engine (e.g., IBM Watson Tone Analyzer) to analyze the tone, volume, and speaking rate of the speech and estimate the user's emotional state. The input is the speech data, and the output is the estimated emotional state. For example, if the voice is loud and the speaking rate is fast, it is determined that the user is excited.

[1323] Step 7:

[1324] The server updates the agenda based on the emotional state.

[1325] The server dynamically updates the agenda based on the estimated emotional state. The input is the estimated emotional state and the existing agenda proposal, and the output is the updated agenda proposal. For example, if a user gets excited and says, "This part is very important," the server adds that part to the agenda as a particularly important topic.

[1326] (Application example 1)

[1327] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1328] Conventional meeting support systems were unable to take into account the emotional state of participants when creating meeting agendas or supporting progress, which could result in a decline in meeting efficiency and quality. Furthermore, they lacked the functionality to automatically generate agendas based on the purpose and topic of the meeting, which resulted in the problem of requiring a lot of time and effort to prepare for meetings.

[1329] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means. In this invention, the server includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for summarizing the contents of the meeting remarks and creating minutes at the end of the meeting, a means for extracting emotions from the participants' voices and providing feedback in real time, and a means for automatically generating a draft agenda using a generative AI model. This makes meeting preparation more efficient, enables feedback based on the participants' emotional states, and improves the quality and efficiency of meetings.

[1330] The "means for creating a draft agenda before a meeting" is a means for automatically generating a draft agenda for a meeting by inputting the purpose and subject of the meeting.

[1331] The "means for recognizing and converting the speaker's voice into text" refers to a means for collecting the voice of the speaker during a conference and converting it into text data.

[1332] "Means for supporting the progress of an ongoing meeting" are means for encouraging participants to speak and providing a summary of the meeting in order to ensure smooth progress of the meeting.

[1333] "Means for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a means for organizing what was said during the meeting and compiling it into minutes after the meeting has ended.

[1334] The "means for extracting emotions from the participants' voices and providing feedback in real time" refers to a means for analyzing the participants' voice data to estimate their emotional state and providing feedback on the results in real time.

[1335] "Means for automatically generating a draft agenda using a generative AI model" means means for automatically generating a draft agenda based on the input meeting purpose and agenda items using a generative AI model.

[1336] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for summarizing the contents of the meeting remarks and creating minutes at the end of the meeting, a means for extracting emotions from the participants' voices and providing feedback in real time, and a means for automatically generating a draft agenda using a generative AI model.

[1337] Hardware and Software Configuration

[1338] 1. Hardware:

[1339] Robot body: Meeting support robot (e.g. Pepper)

[1340] Microphone: For voice input

[1341] Speaker: For audio output

[1342] Display: for displaying agendas and feedback

[1343] 2. Software:

[1344] Speech recognition engine: Google Cloud Speech-to-Text

[1345] Sentiment analysis engine: IBM Watson Tone Analyzer

[1346] Generative AI model: OpenAI GPT-4

[1347] Conference management system: Microsoft Teams API

[1348] Data processing and calculation

[1349] 1. Voice input and recognition:

[1350] The robot collects the user's voice through its microphone and converts it into text using a speech recognition engine.

[1351] Example: A user says, "The purpose of our next meeting is to improve the quality of our new product. The agenda will include reviewing our manufacturing process and strengthening quality inspections."

[1352] 2. Sentiment analysis:

[1353] The converted text and voice data is sent to a sentiment analysis engine to estimate the user's emotional state.

[1354] Example: If the voice is loud and speaking quickly, we determine that the user is excited.

[1355] 3. Agenda generation:

[1356] The purpose and agenda of the meeting are input into a generative AI model to automatically generate a draft agenda.

[1357] Example prompt: "The purpose of the next meeting is to improve the quality of our new product. The agenda will include reviewing the manufacturing process and strengthening quality inspections. Please create a draft agenda based on this."

[1358] 4. Real-time feedback:

[1359] It analyzes audio data collected during meetings in real time and provides feedback based on emotional state.

[1360] For example: If a participant is agitated, provide feedback such as, "Let's calm down a bit."

[1361] Specific examples

[1362] 1. Before the meeting:

[1363] The user speaks to the robot, saying, "The purpose of the next meeting is to improve the quality of our new products. The agenda will include reviewing the manufacturing process and strengthening quality inspections."

[1364] The robot converts the speech into text, which is then fed into a generative AI model to generate a draft agenda.

[1365] The proposed agenda will be displayed on the screen and the user will be asked to confirm it.

[1366] 2. During the meeting:

[1367] The robot will assist in the meeting and analyze participants' voices in real time.

[1368] For example, if a participant is agitated, provide feedback such as, "Please calm down a bit."

[1369] In this way, meeting preparation can be streamlined, feedback based on participants' emotional states can be provided, and the quality and efficiency of meetings can be improved.

[1370] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1371] Step 1:

[1372] The user speaks to the robot to describe the purpose and agenda of the meeting. The robot's microphone collects the voice data and sends it to a speech recognition engine (Google Cloud Speech-to-Text). The speech recognition engine converts the voice data into text data and sends the text data to the server.

[1373] Input: User's voice data

[1374] Output: Text data

[1375] Step 2:

[1376] The server inputs the received text data into a generative AI model (OpenAI GPT-4) to automatically generate a draft agenda. The generated draft agenda is sent from the server to the robot's display and displayed to the user.

[1377] Input: Text data

[1378] Output: Agenda proposal

[1379] Step 3:

[1380] During the meeting, the robot's microphone collects participants' voices in real time and sends them to a speech recognition engine, which converts the voice data into text data and sends the text data to a server.

[1381] Input: Participant's voice data

[1382] Output: Text data

[1383] Step 4:

[1384] The server sends the received text and voice data to a sentiment analysis engine (IBM Watson Tone Analyzer) to estimate the participants' emotional state. The sentiment analysis engine analyzes the emotional state and sends the results to the server.

[1385] Input: Text data, audio data

[1386] Output: Emotional state data

[1387] Step 5:

[1388] The server generates real-time feedback based on the emotional state data and provides it to participants through the robot's speaker. For example, if a participant is excited, the server will provide feedback such as "Please calm down a bit when you speak."

[1389] Input: Emotional state data

[1390] Output: Feedback audio

[1391] Step 6:

[1392] At the end of the meeting, the server organizes the text data collected during the meeting and automatically generates minutes, which are then sent from the server to the robot's display and displayed to the user.

[1393] Input: Text data during the meeting

[1394] Output:Minutes

[1395] In this way, meeting preparation can be streamlined, feedback based on participants' emotional states can be provided, and the quality and efficiency of meetings can be improved.

[1396] Example 2

[1397] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1398] Conventional conference systems have difficulty converting statements made during a meeting into text in real time and generating comments to support the progress of the meeting. It is also difficult to analyze the emotional state of meeting participants and provide appropriate comments based on that. This has led to problems such as reduced efficiency and effectiveness of meetings.

[1399] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1400] In this invention, the server includes means for creating a draft agenda before the meeting, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of the meeting comments and creating minutes at the end of the meeting, means for analyzing emotions, means for generating comments to support the progress of the meeting based on the results of the emotion analysis, and means for presenting the generated comments. This makes it possible to convert comments made during the meeting into text in real time and provide appropriate comments based on the emotional state.

[1401] A "method for drafting an agenda before a meeting" is a method for planning the agenda and progress of a meeting and sharing it with participants before the meeting begins.

[1402] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text data in real time using voice recognition technology.

[1403] "Means for supporting the progress of an ongoing meeting" refers to means for providing comments and instructions at appropriate times to ensure that the meeting proceeds smoothly.

[1404] "Methods for summarizing what has been said at a meeting and creating minutes at the end of the meeting" refers to a method for organizing what has been said at a meeting and saving it as an official record after the meeting has ended.

[1405] The "emotion analysis means" is a means for analyzing text data obtained by speech recognition and identifying the emotional state of the speaker and conference participants.

[1406] The "means for generating comments to support the progress of a meeting based on the result of emotion analysis" is a means for automatically generating comments to facilitate the progress of a meeting based on the result of emotion analysis.

[1407] The "means for presenting the generated comments" is a means for displaying the generated comments to the conference participants and supporting the progress of the conference.

[1408] This invention is a system for improving the efficiency and effectiveness of meetings, and includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what has been said at the meeting and creating minutes at the end of the meeting, a means for analyzing emotions, a means for generating comments that support the progress of the meeting based on the results of the emotion analysis, and a means for presenting the generated comments.

[1409] Hardware and software used

[1410] Hardware

[1411] Device: A computer or smartphone with a microphone

[1412] Server: A server with high-performance data processing capabilities

[1413] software

[1414] Speech recognition software: Google Speech-to-Text API

[1415] Sentiment analysis software: IBM Watson Tone Analyzer

[1416] Generative AI model: OpenAI GPT-3

[1417] Explanation of program processing

[1418] Acquiring voice input

[1419] When a user speaks during a conference, the microphone connected to the device picks up the voice, and the device transmits the captured voice data to the server.

[1420] Speech to text

[1421] The server passes the received voice data to speech recognition software (Google Speech-to-Text API) and converts it into text in real time, which is then passed on to the next processing step.

[1422] Sentiment analysis of text data

[1423] The server sends the converted speech to emotion analysis software (IBM Watson Tone Analyzer) to analyze the emotional state. The analysis results are used to support the progress of the meeting.

[1424] Generate comments to support the progress of the meeting

[1425] The server generates comments to support the progress of the meeting based on the results of sentiment analysis. Appropriate comments are generated using a generative AI model (OpenAI GPT-3).

[1426] Comment suggestion

[1427] The server transmits the generated comments to the terminal, and the terminal presents the comments to the user, who then proceeds with the conference by referring to the presented comments.

[1428] Specific examples

[1429] Specific examples of voice recognition

[1430] When a user says, "How is this project going?", the device's microphone picks up the speech. The server uses the Google Speech-to-Text API to convert this speech into text: "How is this project going?"

[1431] Specific examples of sentiment analysis

[1432] The server then sends the converted speech to the IBM Watson Tone Analyzer, which analyzes the user's emotional state. For example, emotions such as "excitement" or "anxiety" can be identified from the speech.

[1433] Examples of meeting support

[1434] Based on the results of the emotion analysis, the server generates comments to support the progress of the meeting. For example, if it determines that meeting participants are excited, it sends a prompt to the generative AI model saying, "Please generate a comment to encourage meeting participants to calm down when they are excited," and obtains a comment such as, "Everyone, let's calm down a bit."

[1435] Prompt Sentence Examples

[1436] "Generate comments to calm meeting participants when they get excited."

[1437] Using this prompt, the generative AI model can generate comments to effectively support the progress of the meeting.

[1438] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1439] Step 1:

[1440] When a user speaks during a conference, the microphone connected to the device picks up the voice, and the device transmits the captured voice data to the server.

[1441] Input: User's voice

[1442] Output: Audio data sent to the server

[1443] Specific operation: When a user says, "How is this project progressing?", the device's microphone captures the voice and sends it to the server as audio data.

[1444] Step 2:

[1445] The server passes the received voice data to speech recognition software (Google Speech-to-Text API) and converts it into text in real time, which is then passed on to the next processing step.

[1446] Input: Audio data

[1447] Output: Text data

[1448] Specific operation: The server sends the voice data to the Google Speech-to-Text API and receives the text data, "How is the progress on this project?"

[1449] Step 3:

[1450] The server sends the converted speech to emotion analysis software (IBM Watson Tone Analyzer) to analyze the emotional state. The analysis results are used to support the progress of the meeting.

[1451] Input: Text data

[1452] Output: Emotion analysis results

[1453] Specific operation: The server sends the text "How is the progress on this project?" to the IBM Watson Tone Analyzer and obtains emotional states such as "excited" or "anxious."

[1454] Step 4:

[1455] The server generates comments to support the progress of the meeting based on the results of sentiment analysis. Appropriate comments are generated using a generative AI model (OpenAI GPT-3).

[1456] Input: Sentiment analysis results

[1457] Output: Generated comments

[1458] Specific operation: If the server determines that the emotional state is "excited," it sends a prompt to the generative AI model saying, "Please generate a comment to encourage meeting participants to calm down when they are excited," and obtains a comment such as, "Everyone, let's calm down a bit."

[1459] Step 5:

[1460] The server transmits the generated comments to the terminal, and the terminal presents the comments to the user, who then proceeds with the conference by referring to the presented comments.

[1461] Input: Generated comment

[1462] Output: Comments presented to the user

[1463] Specific operation: The server sends a comment to the terminal saying, "Everyone, let's calm down a bit." The terminal then displays the comment to the user. The user then uses the displayed comment as a reference to proceed with the meeting.

[1464] (Application example 2)

[1465] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1466] Conventional conference systems lack the ability to convert statements made during a meeting into text in real time or to support the progress of the meeting. Furthermore, because the progress of the meeting is not adjusted taking into account the emotional state of the participants, the meeting can become stagnant or lack vitality. This reduces the efficiency of the meeting and takes time to create minutes.

[1467] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 2 is realized by the following means.

[1468] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of what was said in the meeting and creating minutes at the end of the meeting, means for recognizing the emotional states of the meeting participants, and means for adjusting the progress of the meeting based on the emotional states. This makes it possible to convert what is said during the meeting into text in real time and adjust the progress of the meeting based on the emotional states.

[1469] A "method for creating an agenda before a meeting" is a method for planning the agenda and progress of a meeting and providing it to participants before the meeting begins.

[1470] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text information in real time using voice recognition technology.

[1471] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[1472] "Methods for summarizing what was said at a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said at the end of a meeting and compiling it into an official record.

[1473] The "means for recognizing the emotional state of the conference participants" is a means for analyzing the emotions of the participants during the conference and grasping their state.

[1474] The "means for adjusting the progress of a meeting based on emotional states" refers to a means for appropriately changing the way the meeting is conducted in accordance with the recognized emotional states.

[1475] A "generative AI model" is a model that uses artificial intelligence technology to generate data and perform specific tasks.

[1476] A "prompt" is an instruction given to a generative AI model to obtain a specific output.

[1477] A system for implementing this invention has the following configuration: A server includes means for creating a draft agenda before a meeting begins, means for recognizing and converting the speaker's voice into text, means for supporting the progress of the meeting during the meeting, means for summarizing what has been said at the meeting and creating minutes at the end of the meeting, means for recognizing the emotional states of the meeting participants, and means for adjusting the progress of the meeting based on their emotional states.

[1478] Hardware and Software Configuration

[1479] Microphone: Used to pick up audio during the meeting.

[1480] Server: Performs various processes such as speech recognition, emotion recognition, progress support, and minutes creation.

[1481] Speech recognition software: Uses speech recognition libraries to convert speech to text.

[1482] Emotion recognition software: Analyzes emotions from text using the pipeline function from the transformers library.

[1483] Generative AI model: Used to generate progress support messages based on emotional state.

[1484] Processing flow

[1485] 1. Speech recognition: The server converts the voice data acquired from the microphone into text in real time using a speech recognition library.

[1486] 2. Emotion recognition: The server performs emotion analysis on the textual content of the speech using the pipeline function of the transformers library.

[1487] 3. Progression support: Based on the results of the sentiment analysis, the server inputs prompt sentences into the generative AI model to generate appropriate progress support messages.

[1488] 4. Creating minutes: At the end of the meeting, the server creates minutes by compiling the text of the remarks and the support messages for the meeting.

[1489] Specific examples

[1490] For example, if someone says "This project is going well" during a meeting, the server converts this speech into text and the result of the sentiment analysis is "POSITIVE." Based on this, the following prompt sentence is input into the generative AI model:

[1491] This is a lively meeting. Let's move on to the next item on the agenda.

[1492] The generated progress support message is "The meeting is lively. Let's move on to the next agenda item." and is presented to the meeting participants.

[1493] In this way, the server can facilitate the smooth progress of the meeting and realize efficient minutes creation.

[1494] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1495] Step 1:

[1496] The server acquires voice data from the microphone. When a user speaks during a conference, the voice is sent to the server through the microphone. The input is voice data, and the output is the acquisition of voice data.

[1497] Step 2:

[1498] The server converts the acquired voice data into text using a voice recognition library. Voice recognition technology is used to convert voice data into text data. The input is voice data, and the output is text data.

[1499] Step 3:

[1500] The server performs sentiment analysis on text data using the pipeline function of the transformers library. The text data is input into a sentiment analysis model to obtain the emotional state. The input is text data, and the output is the emotional state (e.g., POSITIVE, NEGATIVE, NEUTRAL).

[1501] Step 4:

[1502] Based on the results of the emotion analysis, the server inputs a prompt sentence into the generative AI model to generate a progress support message.The server inputs a prompt sentence according to the emotional state into the generative AI model to obtain an appropriate progress support message.The input is the emotional state and the prompt sentence, and the output is a progress support message.

[1503] Step 5:

[1504] The server presents the generated progress support message to the conference participants. The progress support message is communicated to the participants via a display device or an audio output device. The input is the progress support message, and the output is the message presented to the participants.

[1505] Step 6:

[1506] At the end of the meeting, the server compiles the textual content of the remarks and the support messages to create minutes. The server organizes the text data and support messages collected during the meeting and saves them in a file as minutes. The input is the text data and support messages, and the output is a minutes file.

[1507] Example 3

[1508] Next, a description will be given of Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1509] Conventional meeting support systems lacked specific support for smoothly progressing meetings, and did not provide sufficient means to encourage participants to speak or to effectively summarize the meeting content. Furthermore, they lacked the functionality to analyze the content of comments made during the meeting and the emotional state of participants in real time and generate appropriate comments based on that analysis, which could result in a decrease in meeting productivity. Furthermore, when automatically generating minutes at the end of a meeting, the emotional state of the speakers could not be reflected, making it difficult to understand the progress of the meeting in detail.

[1510] The specific processing by the specific processing unit 290 of the data processing device 12 in the third embodiment is realized by the following means.

[1511] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of the meeting comments and creating minutes at the end of the meeting, means for analyzing the contents of the comments and emotional state, means for generating draft comments to encourage comments, and means for generating a summary of the contents and closing content before the end of the meeting. This makes it possible to smoothly proceed with the meeting and provide an environment in which all participants can actively participate.

[1512] A "method for creating a draft agenda before a meeting" is a method for planning the agenda and order of proceedings before a meeting and providing them to participants.

[1513] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text data in real time using voice recognition technology.

[1514] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[1515] "A means for summarizing what has been said at a meeting and creating minutes at the end of the meeting" is a means for automatically generating minutes based on the text of what has been said at the end of the meeting.

[1516] The "means for analyzing speech content and emotional state" is a means for analyzing the textual content of speech and audio data and evaluating the emotional state of each speaker.

[1517] The "means for generating proposed comments to encourage speech" is a means for generating proposed comments to encourage participants and the facilitator to make the next statement according to the progress of the meeting.

[1518] "Means for generating a summary of the content and closing content before the end of the meeting" refers to a means for summarizing what has been said so far and generating closing content before the end of the meeting.

[1519] This invention is a system for smoothly progressing a conference and providing an environment in which all participants can actively participate. This system includes elements of a server, a terminal, and a user, and operates as follows.

[1520] Hardware and software used

[1521] Hardware: Servers, devices (PCs, tablets, smartphones)

[1522] Software: Generative AI models (e.g., GPT-4), emotion engines, speech recognition software (e.g., Google Speech-to-Text)

[1523] System Operation Overview

[1524] A method for drafting agendas before a meeting

[1525] The server plans the agenda and order of the meeting beforehand and generates a draft agenda, which is then provided to participants via their terminals.

[1526] A means of recognizing a speaker's voice and converting it into text

[1527] The server uses voice recognition software to convert speech during the meeting into text data in real time. The voice data is collected from microphones installed in the meeting room and participants' devices.

[1528] Support for ongoing meetings

[1529] The server uses a generative AI model to generate suggested comments that encourage participants to speak according to the progress of the meeting. The generated suggested comments are displayed on the device screen.

[1530] The server uses an emotion engine to analyze the textual content of comments and the audio data, and evaluates the emotional state of each speaker. The evaluation results are fed back to the progress support means.

[1531] A way to generate a summary or closing statement before the end of a meeting

[1532] Before the end of the meeting, the server summarizes what has been said so far and generates a closing statement. The summary and closing statement are displayed on the terminal screen.

[1533] A method for summarizing what was said at a meeting and creating minutes at the end of the meeting

[1534] The server automatically generates minutes based on the text of the remarks at the end of the meeting. The emotion engine records the emotional state of each speaker in the minutes and analyzes how those emotions influenced the discussion. The generated minutes are stored in a database and can be accessed by users.

[1535] Examples of specific examples and prompts

[1536] Examples:

[1537] Imagine a meeting to discuss the progress of a project. If the project leader's tone of voice sounds discouraged when reporting progress, the emotion engine will capture that information and provide feedback to the progress support tool. Based on that information, the progress support tool will suggest encouraging comments to other members. This will lighten the mood in the meeting and promote productive discussion.

[1538] Example prompt sentence:

[1539] "If a project leader sounds discouraged while reporting progress, what encouraging comments should you suggest to other team members?"

[1540] This system aims to facilitate smooth progress of the conference and provide an environment in which all participants can actively participate. The flow of the identification process in the third embodiment will be described with reference to FIG.

[1541] Step 1:

[1542] Collecting meeting audio data

[1543] The server collects audio data from the conference in real time, obtained from microphones installed in the conference room and from participants' devices.

[1544] Input: Audio data from the conference room microphone and participants' devices

[1545] Output: Audio data stored on the server

[1546] Specific operation: The server starts the audio stream at the beginning of the conference and collects the data.

[1547] Step 2:

[1548] Converting audio data to text

[1549] The server converts the collected voice data into text using voice recognition software (e.g., Google Speech-to-Text).

[1550] Input: Audio data

[1551] Output: Text data

[1552] Specific operation: The server converts the voice data into text in real time and stores the text data in a database.

[1553] Step 3:

[1554] Analysis of speech content and emotional state

[1555] The server analyzes the textual content of the comments and evaluates the emotional state of each speaker using an emotion engine.

[1556] Input: Text data, audio data

[1557] Output: Emotional state assessment result

[1558] Specific operation: The server inputs text data into the emotion engine and analyzes the emotional state (e.g., joy, sadness, anger). The evaluation results are stored in a database.

[1559] Step 4:

[1560] Generate comment suggestions to encourage participation

[1561] The server uses a generative AI model (e.g., GPT-4) to generate suggested comments that encourage participants to speak according to the progress of the meeting.

[1562] Input: Analysis results, meeting progress

[1563] Output: Comment proposal

[1564] Specific operation: Based on the analysis results and the progress of the meeting, the server inputs prompt sentences into the generative AI model to generate comment proposals, which are then sent to the device and displayed on the screen.

[1565] Step 5:

[1566] Generate summary and closing content

[1567] Before the end of the meeting, the server summarizes what has been said so far and generates a proposal for closing remarks.

[1568] Input: Text data

[1569] Output: Summary, closing contents

[1570] Specific operation: The server inputs the text data into the summarization algorithm to generate a summary, inputs the proposed closing sentence into the generative AI model as a prompt sentence, and sends the generated sentence to the device.

[1571] Step 6:

[1572] Automatic generation of meeting minutes

[1573] The server automatically generates minutes based on the text of the remarks at the end of the meeting.

[1574] Input: Text data, sentiment analysis results

[1575] Output: Meeting minutes

[1576] Specific operation: The server generates minutes based on the text data and the results of sentiment analysis. The generated minutes are stored in a database and made accessible to users.

[1577] (Application example 3)

[1578] Next, a description will be given of Application Example 3 of Form Example 3. In the following description, the data processing device 12 will be referred to as a "server" and the smart glasses 214 will be referred to as a "terminal."

[1579] Conventional meeting support systems lacked sufficient support for smooth meeting progress, particularly in terms of support that took into account the emotional state of the speaker, and lacked the ability to summarize the meeting content or generate closing comments. Furthermore, the functionality for automatically creating minutes at the end of a meeting was limited, creating challenges for improving meeting efficiency and productivity.

[1580] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 3 is realized by the following means. In this invention, the server includes a means for creating a draft agenda before the meeting begins, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for summarizing the contents of the meeting and creating minutes at the end of the meeting, a means for analyzing the emotional state of the speaker, a means for generating a summary of the meeting content, and a means for generating closing comments. This makes it possible to smoothly progress the meeting, support the progress taking into account the emotional state of the speaker, and generate a summary of the meeting content and closing comments. In addition, automatically creating minutes at the end of the meeting makes it possible to improve the efficiency and productivity of the meeting.

[1581] The "means for creating a draft agenda before a meeting" is a means for automatically creating an agenda and a schedule before a meeting begins.

[1582] "Means for recognizing the speaker's voice and converting it into text" refers to means for converting statements made during a meeting into text data using voice recognition technology.

[1583] The "means for supporting the progress of an ongoing meeting" are means for providing comments and summaries that encourage people to speak up in order to facilitate the smooth progress of the meeting.

[1584] The "means for summarizing the contents of statements made at a meeting and creating minutes at the end of the meeting" is a means for automatically summarizing the contents of statements made at the end of a meeting and creating minutes.

[1585] The "means for analyzing the emotional state of a speaker" is a means for analyzing the emotions from the speaker's voice or text and recording that state.

[1586] The "means for generating a summary of the contents of a meeting" is a means for summarizing the contents of what was said during the meeting and providing it in a concise form.

[1587] The "means for generating closing comments" is a means for automatically generating appropriate closing comments at the end of a meeting.

[1588] A system for carrying out this invention has the following configuration: A server includes means for creating a draft agenda before a meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for summarizing the contents of the meeting and creating minutes at the end of the meeting, means for analyzing the emotional state of the speaker, means for generating a summary of the meeting contents, and means for generating closing comments.

[1589] Hardware and Software Configuration

[1590] Hardware: microphone, speaker, robot body

[1591] Software: Python, speech recognition library, OpenAI API, TextBlob library

[1592] Processing flow

[1593] 1. Speech recognition: The server collects the audio during the meeting through a microphone and converts the audio into text using a speech recognition library.

[1594] 2. Summary generation: The server uses the OpenAI API to generate a summary of the meeting content. Specifically, based on the text data collected during the meeting, the server inputs prompt sentences into the generative AI model to obtain the summary.

[1595] 3. Sentiment Analysis: The server uses the TextBlob library to analyze the emotional state of the speaker from the text data, thereby understanding how the speaker's emotions affect the progress of the conference.

[1596] 4. Closing comment generation: The server again uses the OpenAI API to generate a closing comment based on the meeting content. It inputs a prompt and retrieves an appropriate closing comment.

[1597] 5. Creation of minutes: At the end of the meeting, the server automatically generates minutes based on the collected text data.

[1598] Specific examples

[1599] For example, imagine a meeting is being held to discuss improvements to a production line in a factory. During the meeting, when an engineer makes a new proposal, the emotion engine analyzes how other members feel about the proposal. If there are many negative emotions toward the proposal, the server uses that information to suggest a comment to the facilitator, such as, "Could you please explain this proposal in more detail?"

[1600] Prompt Sentence Examples

[1601] Please summarize the following meeting:

[1602] [Text of the meeting]

[1603] Please generate closing comments based on the following meeting content:

[1604] [Text of the meeting]

[1605] In this way, the system can smoothly advance meetings, provide support that takes into account the emotional state of the speakers, and generate a summary of the meeting content and closing comments. Furthermore, by automatically creating minutes at the end of a meeting, it is possible to improve meeting efficiency and productivity.

[1606] The flow of the specific processing in Application Example 3 will be described with reference to FIG.

[1607] Step 1:

[1608] The server collects audio during the conference via a microphone. The collected audio data is converted into text data using a speech recognition library. The input is audio data, and the output is text data. Through this process, the content of speech during the conference is recorded in text format.

[1609] Step 2:

[1610] The server generates a summary of the meeting content based on the text data. Specifically, it uses the OpenAI API to input prompts into the generative AI model and obtains the summary. The input is the text data collected during the meeting, and the output is the summary text. This process concisely summarizes the meeting content.

[1611] Step 3:

[1612] The server analyzes the emotional state of the speaker from the text data using the TextBlob library. The input is text data, and the output is the emotion analysis results. This processing allows us to understand how the speaker's emotions are affecting the progress of the conference.

[1613] Step 4:

[1614] The server suggests appropriate comments to the facilitator and participants based on the results of the sentiment analysis. For example, if there are a lot of negative emotions, it suggests a comment to the facilitator such as, "Could you please explain this proposal in more detail?" The input is the sentiment analysis results, and the output is the suggested comment. This process helps meetings proceed smoothly.

[1615] Step 5:

[1616] The server generates closing comments based on the meeting content. It again uses the OpenAI API to input prompts into the generative AI model and obtains the closing comments. The input is the text data collected during the meeting, and the output is the closing comments. This process ensures that the meeting is properly concluded.

[1617] Step 6:

[1618] The server automatically generates minutes based on the text data collected at the end of the meeting. The input is the text data collected during the meeting, and the output is the minutes text. This process records the details of the meeting and allows for later reference.

[1619] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[1620] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of a data generation model58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating voice, text data indicating text, and image data indicating an image is also input. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1621] Another example of generative AI is Gemini (internet search engine). <url: https: gemini.google.com ?hl="ja">) are listed.

[1622] In the above embodiment, an example in which the specific processing is performed by the data processing device 12 has been given, but the technology of the present disclosure is not limited to this, and the specific processing may be performed by the smart glasses 214.

[1623] [Third embodiment]

[1624] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[1625] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[1626] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32.

[1627] The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1628] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[1629] The microphone 238 receives instructions and the like from the user 20 by receiving voice uttered by the user 20. The microphone 238 captures the voice uttered by the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio in accordance with instructions from the processor 46.

[1630] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the surroundings of user 20 (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[1631] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[1632] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[1633] The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1634] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1635] In the headset type terminal 314, a reception output process is performed by the processor 46. A reception output program 60 is stored in the storage 50. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output process is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[1636] Next, the specific processing by the specific processing unit 290 of the data processing device 12 will be described.

[1637] "Example 1"

[1638] This embodiment of the present invention provides an interface for inputting the purpose and agenda of a meeting as a means for creating a draft agenda before the meeting. Specifically, a user inputs the purpose and agenda of the meeting as text, and a draft agenda is automatically generated based on that.

[1639] "Example 2"

[1640] In addition, speech recognition technology is used to recognize the speaker's voice and convert it into text. Specifically, what is said during a meeting is picked up by a microphone and converted into text in real time using speech recognition software. This converted text is used to support the progress of the meeting and to create minutes.

[1641] "Example 3"

[1642] As a means of supporting the progress of ongoing meetings, the system provides a means for suggesting comments to encourage participants to speak next, a means for suggesting comments for the facilitator to encourage speaking, and a means for providing a summary of the content and closing content before the end of the meeting so that the meeting can be wrapped up within the allotted time. Specifically, the system analyzes the progress of the meeting and the content of comments, and displays on the screen suggested comments to encourage speaking at the appropriate time. In addition, before the end of the meeting, it summarizes what has been said so far and provides suggested closing content.

[1643] "Example 4"

[1644] At the end of a meeting, we provide a means to compile meeting minutes by summarizing what was said during the meeting. Specifically, minutes are automatically generated based on the text of what was said during the meeting. These minutes can be used for reference after the meeting or to prepare for the next meeting.

[1645] The processing flow of each embodiment will be described below.

[1646] "Example 1"

[1647] Step 1: The user enters the purpose and agenda of the meeting.

[1648] Step 2: The system analyzes the input information and automatically generates a draft agenda.

[1649] Step 3: The generated agenda proposal is displayed to the user, who can make any necessary corrections or additions.

[1650] "Example 2"

[1651] Step 1: When the meeting starts, the microphone picks up what is being said during the meeting.

[1652] Step 2: The captured voice data is sent to speech recognition software and converted into text in real time.

[1653] Step 3: The transcribed remarks are used to support the progress of the meeting and to create minutes.

[1654] "Example 3"

[1655] Step 1: As the meeting progresses, the system analyzes the progress of the meeting and what is being said in real time.

[1656] Step 2: Based on the analysis results, the system displays suggested comments on the screen to encourage people to speak at the appropriate time.

[1657] Step 3: Before the meeting ends, the system summarizes what has been said so far and provides suggestions for closing remarks.

[1658] "Example 4"

[1659] Step 1: Once the meeting is over, the system automatically generates minutes based on the text of what was said during the meeting.

[1660] Step 2: The generated minutes are displayed to the user, who can make corrections or additions as needed.

[1661] Step 3: The final minutes are saved for post-meeting reference and to help prepare for the next meeting.

[1662] Example 1

[1663] Next, a description will be given of Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1664] In conventional conference systems, creating an agenda before a meeting was a time-consuming and difficult task, making it difficult to do efficiently. Supporting the progress of the meeting and creating minutes after the meeting also required manual work, which was time-consuming and labor-intensive. Furthermore, there was a lack of a way to automatically generate an appropriate agenda based on the purpose and topic of the meeting, which meant a lot of time was spent preparing for the meeting.

[1665] The specific processing by the specific processing unit 290 of the data processing device 12 in the first embodiment is realized by the following means.

[1666] In this invention, the server includes means for a user to input the purpose and agenda of a meeting, means for transmitting the input data to the server, means for the server to analyze the received data and create and input prompt sentences to the generative AI model, means for the generative AI model to generate a draft agenda, means for the server to return the generated draft agenda to the terminal, and means for the terminal to display the draft agenda to the user, thereby enabling users to easily automatically generate draft agendas for meetings.

[1667] The "means for creating a draft agenda before a meeting" is a means for automatically generating a draft agenda based on the purpose and agenda of the meeting.

[1668] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[1669] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to ensure the smooth progress of meetings.

[1670] "Methods for summarizing what was said at a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said at a meeting and compiling it into minutes after the meeting has ended.

[1671] The "means for users to input the purpose and agenda of the meeting" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[1672] The "means for transmitting input data to a server" refers to a communication means for transmitting data input by a user to a server.

[1673] "Means for analyzing data received by the server and creating and inputting prompt sentences to the generative AI model" refers to means for analyzing data received by the server and creating prompt sentences to be input to the generative AI model based on that data.

[1674] "Means by which a generative AI model generates a draft agenda" refers to means by which a generative AI model automatically generates a draft agenda based on a prompt sentence.

[1675] The "means for the server to return the generated agenda proposal to the terminal" refers to the means for the server to transmit the generated agenda proposal to the terminal.

[1676] The "means by which the terminal displays the agenda proposal to the user" refers to the means by which the terminal visually displays the received agenda proposal to the user.

[1677] This invention is a system that automatically generates a draft agenda before a meeting, supports the progress of the meeting during the meeting, and creates minutes after the meeting. This system includes: a means for a user to input the purpose and agenda of the meeting; a means for transmitting the input data to a server; a means for the server to analyze the received data and create and input prompt sentences to a generative AI model; a means for the generative AI model to generate a draft agenda; a means for the server to return the generated draft agenda to a terminal; and a means for the terminal to display the draft agenda to the user.

[1678] Users access a dedicated interface using a web browser and enter the purpose and agenda of the meeting in text format. This interface is implemented as an HTML form, making it easy for users to enter information. For example, a user can enter the purpose of the meeting, "Determine a market launch strategy for a new product," and the agenda, "Market analysis, competitive research, marketing strategy, and sales plan."

[1679] The device sends the data entered by the user to the server as an HTTP POST request. The data is sent in JSON format. The server analyzes the received data and creates a prompt to input into the generative AI model. For example, it generates a prompt such as, "The purpose of the meeting is to decide on a market launch strategy for a new product, and the agenda will include market analysis, competitive research, marketing strategy, and sales plan. Please create a draft agenda based on this."

[1680] The server inputs prompt sentences into the generative AI model to generate a draft agenda. The generated draft agenda is temporarily stored on the server, and the server sends it back to the device. The device analyzes the received draft agenda and displays it in a format that is easy for the user to view. For example, it is displayed on a web page in the following format:

[1681] Proposed agenda:

[1682] 1. Market analysis (10:00 - 10:30)

[1683] Review of current market conditions

[1684] Analysis of key market trends

[1685] 2. Competitive Research (10:30 - 11:00)

[1686] Strategies and trends of major competitors

[1687] Analysis of the strengths and weaknesses of competing products

[1688] 3. Marketing Strategy (11:00 - 11:30)

[1689] Identifying your target market

[1690] Marketing campaign planning

[1691] 4. Sales Planning (11:30 - 12:00)

[1692] Selection of sales channels

[1693] Setting sales targets

[1694] In this way, users can easily automatically generate meeting agendas. Support for meeting progress and the creation of meeting minutes after the meeting are also automated, allowing for efficient preparation, progress, and post-meeting processes.

[1695] The flow of the identification process in the first embodiment will be described with reference to FIG.

[1696] Step 1:

[1697] The user inputs the purpose and agenda of the meeting.

[1698] A user opens a web browser and accesses a dedicated interface. The interface has a text input field where the user inputs the purpose and agenda of the meeting. For example, the purpose is to "decide on a market launch strategy for a new product" and the agenda is to "market analysis, competitive research, marketing strategy, and sales plan." The input data is in text format.

[1699] Step 2:

[1700] The terminal sends the input data to the server.

[1701] When the user completes the input and clicks the submit button, the device sends the input data to the server as an HTTP POST request. The data sent is in JSON format, and looks something like this:

[1702] json

[1703] {

[1704] "Objective": "Determine a new product launch strategy",

[1705] "Agenda": "Market analysis, competitive research, marketing strategy, sales plan"

[1706] }

[1707] Step 3:

[1708] The server receives the data and creates and inputs prompts to the generative AI model.

[1709] The server analyzes the received data and creates a prompt to be input to the generative AI model. For example, it generates the following prompt:

[1710] "The purpose of the meeting is to decide on a market launch strategy for a new product, and the topics to be discussed will include market analysis, competitive research, marketing strategy, and sales plan. Please create a draft agenda based on this."

[1711] The input data is in JSON format, and the output data is a prompt in text format.

[1712] Step 4:

[1713] A generative AI model generates a draft agenda.

[1714] The server inputs a prompt to the generative AI model, which then generates a draft agenda based on the prompt. For example, the following draft agenda is generated:

[1715] 1. Market analysis (10:00 - 10:30)

[1716] Review of current market conditions

[1717] Analysis of key market trends

[1718] 2. Competitive Research (10:30 - 11:00)

[1719] Strategies and trends of major competitors

[1720] Analysis of the strengths and weaknesses of competing products

[1721] 3. Marketing Strategy (11:00 - 11:30)

[1722] Identifying your target market

[1723] Marketing campaign planning

[1724] 4. Sales Planning (11:30 - 12:00)

[1725] Selection of sales channels

[1726] Setting sales targets

[1727] The input data is a prompt, and the output data is a draft agenda in text format.

[1728] Step 5:

[1729] The server returns the generated agenda plan to the terminal.

[1730] The server returns the generated agenda in JSON format to the terminal. For example, the following JSON data is returned:

[1731] json

[1732] {

[1733] "agenda": [

[1734] {"Time": "10:00 - 10:30", "Topic": "Market Analysis", "Details": ["Review of the current market situation", "Analysis of key market trends"]},

[1735] {"Time": "10:30 - 11:00", "Topic": "Competitive Research", "Details": ["Strategies and Trends of Major Competitors", "Analysis of Strengths and Weaknesses of Competitive Products"]},

[1736] {"Time": "11:00 - 11:30", "Topic": "Marketing Strategy", "Details": ["Identifying Target Market", "Planning a Marketing Campaign"]},

[1737] {"Time": "11:30 - 12:00", "Item": "Sales plan", "Details": ["Sales channel selection", "Sales target setting"]}

[1738] ]

[1739] }

[1740] The input data is a draft agenda in text format, and the output data is a draft agenda in JSON format.

[1741] Step 6:

[1742] The terminal displays the proposed agenda to the user.

[1743] The device parses the received agenda proposal and displays it in a user-friendly format, for example on a web page in the following format:

[1744] Proposed agenda:

[1745] 1. Market analysis (10:00 - 10:30)

[1746] Review of current market conditions

[1747] Analysis of key market trends

[1748] 2. Competitive Research (10:30 - 11:00)

[1749] Strategies and trends of major competitors

[1750] Analysis of the strengths and weaknesses of competing products

[1751] 3. Marketing Strategy (11:00 - 11:30)

[1752] Identifying your target market

[1753] Marketing campaign planning

[1754] 4. Sales Planning (11:30 - 12:00)

[1755] Selection of sales channels

[1756] Setting sales targets

[1757] The input data is a draft agenda in JSON format, and the output data is a visual representation of the draft agenda.

[1758] (Application example 1)

[1759] Next, a description will be given of Application Example 1 of Form Example 1. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1760] In conventional conference systems, creating the agenda before the meeting, providing support during the meeting, and creating minutes after the meeting were often done manually, which was time-consuming and labor-intensive. It was also difficult to create an agenda efficiently based on the purpose and topic of the meeting, which sometimes prevented the meeting from proceeding smoothly.

[1761] The specific processing by the specific processing unit 290 of the data processing device 12 in the application example 1 is realized by the following means.

[1762] In this invention, the server includes a means for creating a draft agenda before the meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting during the meeting, a means for compiling the contents of the meeting remarks and creating minutes at the end of the meeting, an interface means for the user to input the purpose and agenda of the meeting as text, a means for automatically generating a draft agenda using a generative AI model, and a means for displaying and sharing the generated draft agenda with the user. This makes it possible to efficiently carry out everything from preparation to progress of the meeting and follow-up after it ends.

[1763] The "means for creating a draft agenda before a meeting" is a means for automatically creating a draft agenda for planning the progress of a meeting based on the purpose and agenda of the meeting.

[1764] The "means for recognizing the speaker's voice and converting it into text" refers to a means for recognizing the voice of a speaker during a conference in real time and converting it into text data.

[1765] "Means for supporting the progress of ongoing meetings" are means for providing appropriate comments and instructions to participants and facilitators in order to facilitate the smooth progress of meetings.

[1766] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to methods for organizing what was said during the meeting and compiling it into minutes after the meeting has ended.

[1767] The "interface means for users to input the purpose and agenda of the meeting as text" refers to a means for providing an interface for users to input the purpose and agenda of the meeting in text format.

[1768] "Means for automatically generating a draft agenda using a generative AI model" refers to means for automatically generating a draft agenda based on the input meeting purpose and agenda items using a generative AI model.

[1769] The "means for displaying and sharing the generated agenda draft with the user" refers to a means for displaying the generated agenda draft to the user and sharing it with other participants as necessary.

[1770] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what was said in the meeting and creating minutes at the end of the meeting, an interface means for users to input the purpose and agenda of the meeting as text, a means for automatically generating a draft agenda using a generative AI model, and a means for displaying and sharing the generated draft agenda with users.

[1771] System program processing explanation

[1772] Hardware used

[1773] Smartphone

[1774] server

[1775] Software used

[1776] Python

[1777] OpenAI API

[1778] Data processing and calculation

[1779] 1. User Input:

[1780] The user opens the smartphone application and inputs the purpose and agenda of the meeting in text format, thereby functioning as an interface means for the user to input the purpose and agenda of the meeting in text format.

[1781] 2. Use of generative AI models:

[1782] Based on the input information, a prompt is sent to a generative AI model (OpenAI's GPT-3). This is where the means to automatically generate a draft agenda using the generative AI model comes into play.

[1783] 3. Generate a draft agenda:

[1784] GPT-3 generates a draft agenda based on the prompt and returns it to the application, which then provides a way to display and share the draft agenda with the user.

[1785] 4. Speech Recognition:

[1786] During a meeting, the speaker's voice is recognized in real time and converted into text data. This is where the means for recognizing the speaker's voice and converting it into text comes into play.

[1787] 5. Progression support:

[1788] To facilitate smooth progress in the meeting, appropriate comments and instructions are provided to participants and facilitators. This is where ongoing meeting support measures come into play.

[1789] 6. Minutes Preparation:

[1790] After the meeting, the contents of what was said during the meeting are organized and compiled as minutes. This is where the means of compiling the contents of what was said during the meeting and creating minutes at the end of the meeting comes into play.

[1791] Specific examples

[1792] A store manager holds a sales strategy meeting for a new product, and uses a smartphone app to input the purpose and agenda of the meeting. For example, he might enter a prompt like this:

[1793] Meeting purpose: New product sales strategy

[1794] agenda:

[1795] market analysis

[1796] Competitive Research

[1797] Selection of sales channels

[1798] Promotion Plan

[1799] Create a draft agenda based on this information.

[1800] The application automatically generates an agenda that can be shared with all staff members, improving meeting efficiency. In this way, meetings can be efficiently conducted from preparation to progress, and follow-up after the meeting has ended.

[1801] The flow of the specific processing in the application example 1 will be described with reference to FIG.

[1802] Step 1:

[1803] A user opens a smartphone application and inputs the purpose and agenda of the meeting in text format. The input text data is generated as a prompt sentence including the purpose and agenda of the meeting. The input data is the "purpose of the meeting" and "agenda," and the output data is the prompt sentence.

[1804] Step 2:

[1805] The device sends the generated prompt to the server. The server receives the prompt and sends it to the generative AI model (OpenAI's GPT-3). The input data is the prompt, and the output data is a request to the generative AI model.

[1806] Step 3:

[1807] The server receives the response from the generative AI model. The generative AI model generates a draft agenda based on the prompt and returns the result to the server. The input data is the prompt, and the output data is the generated draft agenda.

[1808] Step 4:

[1809] The server sends the generated agenda plan to the terminal, which receives the agenda plan and displays it to the user. The input data is the generated agenda plan, and the output data is the agenda plan displayed to the user.

[1810] Step 5:

[1811] During a meeting, the device recognizes the speaker's voice in real time and converts it into text data. Speech recognition technology is used to convert the speaker's voice into text. The input data is the speaker's voice, and the output data is the text of what was said.

[1812] Step 6:

[1813] The server receives the textual content of speech and generates appropriate comments and instructions to support the progress of the meeting. This provides appropriate comments and instructions to participants and the facilitator. The input data is the textual content of speech, and the output data is the generated comments and instructions.

[1814] Step 7:

[1815] After the meeting ends, the server organizes the content of the meeting and compiles it into minutes. The generated minutes are sent to the terminal and displayed to the user. The input data is the text of the comments, and the output data is the generated minutes.

[1816] Example 2

[1817] Next, a description will be given of Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1818] With conventional conferencing systems, it was difficult to convert statements made during a meeting into text in real time and use it to support the progress of the meeting or to create minutes. Furthermore, there was a lack of efficient means to carry out a series of processes, such as collecting, transmitting, converting, displaying, and saving audio data. This resulted in reduced meeting efficiency and the time it took to create minutes.

[1819] The specific processing by the specific processing unit 290 of the data processing device 12 in the second embodiment is realized by the following means.

[1820] In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for compiling the contents of what was said at the meeting and creating minutes at the end of the meeting, means for collecting voice data, means for transmitting the voice data, means for converting the voice data into text data, means for displaying the text data in real time, and means for saving the text data. This makes it possible to convert what is said during the meeting into text in real time, and to efficiently support the progress of the meeting and create minutes.

[1821] A "method for drafting an agenda before a meeting" is a method for planning the agenda and progress of a meeting and sharing it with participants before the meeting begins.

[1822] "Means for recognizing the speaker's voice and converting it into text" refers to a means for converting statements made during a meeting into text in real time using voice recognition technology.

[1823] "Means for supporting the progress of a meeting in progress" are means for encouraging people to speak and supporting the progress of the agenda in order to ensure the smooth progress of the meeting.

[1824] "Methods for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a method for organizing what was said during the meeting and saving it as an official record when the meeting ends.

[1825] The "means for collecting voice data" refers to a means for collecting speeches made during a meeting as voice data using a device such as a microphone.

[1826] The "means for transmitting voice data" refers to a means for transmitting collected voice data to a server or other device.

[1827] The "means for converting voice data into text data" refers to a means for converting voice data into text data using voice recognition technology.

[1828] The "means for displaying text data in real time" refers to a means for displaying the converted text data in real time during a conference so that participants can check it.

[1829] The "means for saving text data" refers to a means for saving text data generated during a conference so that it can be referenced later.

[1830] MODE FOR CARRYING OUT THE INVENTION

[1831] The present invention relates to a system that converts statements made during a meeting into text in real time and uses the text to support the progress of the meeting and to create minutes. A specific embodiment of this system will be described below.

[1832] Hardware and software used

[1833] Hardware: microphone, device (computer, tablet, smartphone, etc.)

[1834] Software: speech recognition software (e.g., Google Speech-to-Text API), conferencing applications

[1835] System Overview

[1836] The system includes the following main means:

[1837] 1. A means for creating agendas before a meeting: A means for users to plan the agenda and progress of a meeting and share it with participants.

[1838] 2. A means of recognizing the speaker's voice and converting it into text: A means of converting statements made during a meeting into text in real time using voice recognition technology.

[1839] 3. Support measures for ongoing meetings: These are measures to encourage speaking and help move the agenda forward in order to ensure smooth progress in the meeting.

[1840] 4. A means of summarizing what was said during the meeting and creating minutes at the end of the meeting: This is a means of organizing what was said during the meeting and saving it as an official record when the meeting ends.

[1841] 5. Means of collecting audio data: A means of collecting speech during a meeting as audio data using a device such as a microphone.

[1842] 6. Means for transmitting audio data: A means for transmitting collected audio data to a server or other device.

[1843] 7. Means for converting voice data into text data: A means for converting voice data into text data using voice recognition technology.

[1844] 8. A means for displaying text data in real time: A means for displaying converted text data in real time during a meeting so that participants can check it.

[1845] 9. A means of saving text data: A means of saving text data generated during a meeting for later reference.

[1846] Specific examples

[1847] For example, if someone says "Let's move on to the next topic" during a meeting, the following processing will occur:

[1848] The audio is picked up by a microphone connected to the device.

[1849] The terminal transmits the voice data to the server.

[1850] The server converts the voice data into text data using speech recognition software (Google Speech-to-Text API).

[1851] The server displays text data such as "Let's move on to the next topic" in the conference application in real time.

[1852] The user checks the displayed text and supports the progress of the meeting.

[1853] After the meeting ends, the server saves the text data as minutes, allowing users to access them later.

[1854] Prompt Sentence Examples

[1855] Examples of prompts to be input to a generative AI model include:

[1856] "Please explain the process flow of a system that converts statements made during a meeting into text in real time and is used to support the progress of the meeting and create meeting minutes. Specifically, please clearly state what hardware and software are used and what data processing and calculations are performed for each step from the start of the meeting to collecting audio, transmitting the audio data, converting the audio into text, and using the text data."

[1857] By inputting this prompt into a generative AI model, a detailed description of the system can be obtained.

[1858] The flow of the identification process in the second embodiment will be described with reference to FIG.

[1859] Step 1:

[1860] A user starts a conference.

[1861] Input: A user launches a conferencing application and clicks the "Start Conference" button.

[1862] Data processing / calculation: The terminal notifies the server of the start of the conference and generates a conference ID.

[1863] Output: A conference ID is generated and the conference is started.

[1864] Specific operation: When a user launches a conferencing application and clicks the "Start conference" button, the terminal sends a request to start the conference to the server, and the server generates a conference ID.

[1865] Step 2:

[1866] The device collects the audio.

[1867] Input: Speeches made during the meeting are input into the microphone.

[1868] Data processing / computation: The microphone converts the voice into digital data, which the device stores in a buffer.

[1869] Output: Digital audio data is stored in a buffer.

[1870] How it works: A microphone connected to the device picks up what is being said during the meeting, converts the audio into digital data, and stores it in a buffer.

[1871] Step 3:

[1872] The terminal transmits the audio data.

[1873] Input: Buffered digital audio data.

[1874] Data processing / calculation: The terminal divides the voice data into packets and sends them to the server over the network.

[1875] Output: The audio data is sent to the server.

[1876] Specific operation: The device divides the audio data stored in the buffer into packets and sends them to the server over the network.

[1877] Step 4:

[1878] The server converts the audio into text.

[1879] Input: The audio data received by the server.

[1880] Data processing / calculation: The server converts the voice data into text data using speech recognition software (e.g., Google Speech-to-Text API).

[1881] Output: Text data is generated.

[1882] Specific operation: The server passes the received voice data to the voice recognition software, which converts the voice data into text data.

[1883] Step 5:

[1884] The server displays the text data in real time.

[1885] Input: Generated text data.

[1886] Data processing / calculation: The server sends the text data to the conferencing application and displays it in real time.

[1887] Output: The text data is displayed in the conferencing application.

[1888] Specific behavior: The server sends the generated text data to the conferencing application and displays it so that the user can view it in real time.

[1889] Step 6:

[1890] The server stores the text data.

[1891] Input: Generated text data.

[1892] Data processing / calculation: The server stores the text data in a database.

[1893] Output: Text data is saved.

[1894] Specific operation: The server stores the generated text data in a database so that users can access it after the meeting ends.

[1895] (Application example 2)

[1896] Next, a description will be given of Application Example 2 of Form Example 2. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1897] Conventional conferencing systems can convert statements made during a meeting into text in real time to support the progress of the meeting, but they cannot receive voice instructions at work sites such as factories and automatically carry out work based on those instructions. This reduces work efficiency at the site and increases the possibility of human error.

[1898] The specific processing by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for creating a draft agenda before the meeting begins, means for recognizing the speaker's voice and converting it into text, means for supporting the progress of the meeting during the meeting, means for summarizing the contents of the meeting remarks and creating minutes at the end of the meeting, and means for receiving work instructions by voice, converting them into text in real time, and automating work based on the text. This makes it possible not only to support the progress of meetings and create minutes, but also to receive work instructions by voice at work sites such as factories and automatically perform work based on those instructions.

[1899] A "method for creating a draft agenda before a meeting" is a method for creating an agenda and a schedule before a meeting begins.

[1900] "Means for recognizing a speaker's voice and converting it into text" refers to means for converting a speaker's speech into text data in real time using voice recognition technology.

[1901] The "means for supporting the progress of the ongoing conference" are means for supporting the smooth progress of the conference.

[1902] "Means for summarizing what was said during a meeting and creating minutes at the end of the meeting" refers to a means for summarizing what was said during the meeting and compiling it into minutes when the meeting ends.

[1903] "Means for receiving work instructions via voice, converting them into text in real time, and automating work based on that text" refers to means for converting work instructions given via voice into text in real time, and automatically performing work based on that text.

[1904] A system for implementing this invention includes a means for creating a draft agenda before a meeting, a means for recognizing the speaker's voice and converting it into text, a means for supporting the progress of the meeting while it is in progress, a means for summarizing what has been said at the meeting and creating minutes at the end of the meeting, and a means for receiving work instructions by voice, converting them into text in real time, and automating work based on that text.

[1905] The server uses speech recognition technology to convert the speaker's voice into text in real time. Specifically, it collects voice data using a microphone and converts it into text data using a speech recognition library. This text data is used to support the progress of the meeting and to create minutes.

[1906] The server also receives work instructions via voice at factories and other work sites, converts them into text in real time, and automatically performs tasks based on the text. For example, tasks such as assembling, inspecting, and transporting parts can be performed by voice command. This improves work efficiency at the site and reduces the occurrence of human error.

[1907] For example, if a voice command such as "Please assemble the parts" is given in a factory, the server will recognize the command and begin assembling the parts. At this time, voice recognition technology will be used to convert the command into text, and the robot will automatically carry out the work based on that text.

[1908] Examples of prompts to input to a generative AI model include:

[1909] Create a program that recognizes the voice command "Please assemble the parts" and causes the robot to begin assembling the parts.

[1910] In this way, by combining voice recognition technology and automation technology, the server can not only support the progress of meetings and create minutes, but also efficiently process and automate work instructions on-site.

[1911] The flow of the specific processing in the application example 2 will be described with reference to FIG.

[1912] Step 1:

[1913] The user inputs voice instructions through a microphone, and the input voice data is sent to the server.

[1914] Step 2:

[1915] The server converts the received voice data into text data using a voice recognition library. Specifically, it analyzes the voice data and generates corresponding text using voice recognition technology. The input is voice data and the output is text data.

[1916] Step 3:

[1917] The server analyzes the generated text data and understands the instructions. For example, it checks whether the instruction "Please assemble the parts" is included. The input is the text data, and the output is the analysis result.

[1918] Step 4:

[1919] The server selects an appropriate task based on the analysis results. For example, if the keyword "assembly" is included, it selects the task of assembling parts. The input is the analysis results, and the output is the selected task.

[1920] Step 5:

[1921] The server generates commands to execute the selected tasks and sends them to the robot. Specifically, it generates commands containing specific operation instructions for assembly tasks. The input is the selected task, and the output is the command to the robot.

[1922] Step 6:

[1923] The robot performs a task based on the command it receives, for example, starting an assembly task. The input is the command to the robot, and the output is the task performed.

[1924] Step 7:

[1925] The server monitors the progress of the work and issues further instructions as needed, for example, when the work is completed it issues the next instruction. The input is the progress of the work and the output is further instructions.

[1926] In this way, a series of steps is realized: the user issues instructions by voice, the server converts the instructions into text, analyzes them, selects an appropriate task, and the robot carries out that task.

[1927] Example 3

[1928] Next, a third embodiment of the third embodiment will be described. In the following description, the data processing device 12 will be referred to as a "server" and the headset type terminal 314 will be referred to as a "terminal."

[1929] Conventional conferencing systems lacked sufficient support for smoothly advancing meetings, and in particular, the timing of encouraging participants to speak, summarizi...

Claims

1. A means for recognizing and converting the speech of meeting participants into text; an emotion analysis means for analyzing emotions of the participants from text information of the converted voices of the participants; A conference progress support means for generating comments to support the progress of the conference based on the emotion analysis result of the participants and presenting the generated comments; A means for compiling the remarks of the participants at the end of the meeting and creating minutes of the meeting; A system including:

2. The progress support means includes: Present the generated comment proposal to the participant to encourage the participant to speak; Proposing to the facilitator a proposed comment for encouraging the facilitator to make the comment that has been generated; Generate and present at least one of a summary of the meeting contents and closing contents before the end of the meeting; The system of claim 1 .

3. The progress support means generates the comment using a generation AI based on a prompt sentence corresponding to the emotion analysis result of the participant. The system of claim 1 .

Images